swarmops derive#

Derive the eager derived values of new changesets, ahead of any reader.

Once, it derives the values of the changesets the journal records in up to 1,000 frames, from its first frame or after --after N, and gives the last frame it read as next: run it with --after next to go on. With --follow it runs until it is killed: from the journal's end (from its first frame with --from-start, or after --after N) it derives the changesets as they are recorded, looking again every --interval-ms once it has caught up, and prints one envelope a line for each run that derived something. A run that fails ends it with that run's envelope and exit code. Commits derive nothing.

Metadata

Group: store
Since: 0.1.0
Mutates: no
Identity required: no
MCP tool: none
Envelope op: derive
Exit codes: 0
Errors: none

Syntax#

swarmops derive [OPTIONS]

Required arguments#

None.

Optional arguments#

The command also takes the global options.

Syntax: --after <N>.

Go on after journal frame N, a previous run's next (default: the first frame; with --follow, the journal's end).

Syntax: --follow.

Keep deriving what the journal records, until killed.

Syntax: --interval-ms <N>. Default: 500.

With --follow: how many milliseconds to wait before looking again, once caught up.

Syntax: --from-start.

With --follow: start from the journal's first frame, not its end.

Examples#

swarmops --repo [tmp]/store derive

{
  "ok": true,
  "op": "derive",
  "api": 1,
  "result": {
    "derived": 2,
    "cached": 0,
    "next": 1
  },
  "meta": {
    "repo": "[tmp]/store",
    "elapsed_ms": "[elapsed_ms]",
    "resolved": {},
    "auth": "trust"
  }
}

Exit code 0. Recorded by the CLI tests. A name in square brackets, such as [tmp] or [c1], stands for what differs from run to run: the test's temporary directory, an id, a time.

Details#

derive computes the store's eager derived values ahead of the readers that would otherwise compute them, such as each changeset's first-parent skip list. A derived value is a cache: computed from immutable objects, stored in a file of its own under meta/derived/, never synced and named by no journal record (Derived value). So derive writes nothing but that cache: it needs no identity, it is not a mutation, and it has no MCP tool, being maintenance rather than an agent's work.

Nothing is lost without it. Commits derive nothing, and a read that needs a value not derived yet derives it and stores it. derive moves that work off the read path.

Once, or following the journal#

The journal records one frame per write transaction, a commit or a ref's move, numbered from 0 in the order they committed (Journal and frame).

Without --follow, derive reads the frames after --after N (from the first frame without it), at most 1,000 of them, derives the values of the changesets they record, prints one envelope and exits. next is the last frame it read: run it again with --after <next> to go on through a longer journal, or later to take what was recorded since. next is left out when no frame follows --after.
With --follow, it runs until it is killed. It starts at the journal's end, so it derives only what is recorded from then on; --from-start starts at the first frame instead, and --after N after frame N. Once it has read every frame it waits --interval-ms (500 by default) and looks again. It prints one envelope for each run that derived something, at once, and nothing for a run that derived nothing.

--after and --from-start exclude each other, and --interval-ms and --from-start need --follow; any of these on a command line the grammar refuses is E_INVALID_ARG (exit 2). --interval-ms is at least 1.

Reading the counts#

derived counts the values this run computed for the changesets its frames record; cached counts those the store held already. Values computed on the way, an older changeset's that a newer one needed, are stored but not counted. Run twice over the same frames, the second run derives nothing and counts the values as cached.

Patterns#

Beside a busy store, run swarmops derive --follow as a long-lived process, so that agents' reads find their values stored. Its output is one compact JSON envelope a line when stdout is not a terminal, which a log collector can read as it comes.
After removing meta/derived/ to make room, run swarmops derive once to compute the eager values for every changeset again (Derive them again).
To go through a large journal in steps, loop on --after <next> until next is left out.

Caveats#

A --follow run that fails ends the command with that run's envelope and its exit code; a store that does not open ends it at once the same way. Output that can no longer be written (the reader of its stdout gone) ends it with exit 1 and a line on stderr.
--after past the journal's last frame is not an error: there is nothing after it, so the run derives nothing and leaves next out.

Return information#

result is a DeriveResp (swarmops schema DeriveResp prints its JSON Schema).

Field	Type	Required	Description
`cached`	`integer`	yes	How many of those values the store held already.
`derived`	`integer`	yes	How many values it derived: of each eager derivation, one for each changeset its frames record whose value the store lacked. Values those needed on the way, an older changeset's, are derived too and not counted.
`next`	`integer or null`	no	The last frame it read: pass it as the next call's `after` to go on, through a journal longer than `limit` frames or through the frames recorded since. Omitted when it read no frame, as none follows `after`: it is at the journal's end. With `limit` 0, the journal's last frame, omitted while the journal holds none.

Envelope op: derive.

Exit codes:

0: Success.

Errors#

No golden step shows this command failing yet.

Any command that opens a store may also fail with a store error, such as E_NOT_A_REPO, E_IO or E_STORE_CORRUPT; a command line the grammar refuses is E_INVALID_ARG (exit 2).