Artifacts
All pages
Docs · StartMarkdown

The model

Term Is For checkpoints
Artifact A path you choose, slashes allowed: experiments/hoonshot/1234 One run's checkpoint location
Version An immutable, named set of files under an artifact, with a little JSON meta One checkpoint, step-00000510
Latest The greatest committed version name, which is always the last one written What --resume latest takes
Writer The one holder allowed to add versions The run; a new box resuming fences the old one
Full path o41://<artifact>/<version>; names have no slash, so the last segment is the version o41://experiments/hoonshot/1234/step-00000300
Note A small blob beside the versions, replaced whole, never part of one An evaluator's evaluated.json

Names

  • An artifact is 1 to 512 characters: slash-separated segments of letters, digits, ., _ and -, with no empty, . or .. segment.
  • A version is 1 to 128 letters, digits, ., _ or -, and not latest.
  • A file name is 1 to 256 of the same characters, with no slash, and not _manifest.json: an artifact path holds slashes, so a file name with one could name another artifact's file. A version holds up to 1000 files and up to 16 KiB of meta.

Versions only ascend

A new version is named above the latest, so the latest is always the last one written. Nothing is ever hidden and there is no lineage to track. Zero-pad numbered names so byte order is step order: step-00000510, not step-510.

One writer

An artifact has at most one writer at a time. Opening a writer takes the artifact over: it gets a new epoch, and every begin or commit from an older epoch is refused with 409 superseded from then on. A writer opens in one of three modes:

Mode On an empty artifact On an artifact with versions Checkpoint use
latest Opens; nothing to continue from Opens and returns the latest, signed --resume latest
new Opens 409 not_empty A run without --resume
fork Opens a fork, records the source, returns the source signed 409 not_empty --resume <full path> into a new artifact

There is no way to continue from anything but the latest in place. Any other version is a fork into a new, empty artifact. No bytes are copied: the fork reads its source where it is, and its first version names the source.

epoch 1  box A  aws us-east-1      100  200  300  400  500  [510 in flight]
epoch 2  box B  azure westeurope                       open latest → 500   510  520
                                                       A's commit of 510: 409 superseded

Readers never open a writer. Reading a version, listing versions and listing artifacts need only a key.

Why a used artifact takes nothing but a continuation

A run launched without --resume on a location another run already used would write its checkpoints between the old run's, while the latest stays the old run's highest step; its first preemption would then resume into the other run. So new on a used artifact is refused, before the first step.