All pages
The model
| Term | Is | For checkpoints |
|---|---|---|
| Artifact | A path you choose, slashes allowed: experiments/hoonshot/1234 |
One run's checkpoint location |
| Version | An immutable, named set of files under an artifact, with a little JSON meta | One checkpoint, step-00000510 |
| Latest | The greatest committed version name, which is always the last one written | What --resume latest takes |
| Writer | The one holder allowed to add versions | The run; a new box resuming fences the old one |
| Full path | o41://<artifact>/<version>; names have no slash, so the last segment is the version |
o41://experiments/hoonshot/1234/step-00000300 |
| Note | A small blob beside the versions, replaced whole, never part of one | An evaluator's evaluated.json |
Names
- An artifact is 1 to 512 characters: slash-separated segments of letters,
digits,
.,_and-, with no empty,.or..segment. - A version is 1 to 128 letters, digits,
.,_or-, and notlatest. - A file name is 1 to 256 of the same characters, with no slash, and not
_manifest.json: an artifact path holds slashes, so a file name with one could name another artifact's file. A version holds up to 1000 files and up to 16 KiB of meta.
Versions only ascend
A new version is named above the latest, so the latest is always the last one
written. Nothing is ever hidden and there is no lineage to track. Zero-pad
numbered names so byte order is step order: step-00000510, not step-510.
One writer
An artifact has at most one writer at a time. Opening a writer takes the
artifact over: it gets a new epoch, and every begin or commit from an older
epoch is refused with 409 superseded from then on. A writer opens in one of
three modes:
| Mode | On an empty artifact | On an artifact with versions | Checkpoint use |
|---|---|---|---|
latest |
Opens; nothing to continue from | Opens and returns the latest, signed | --resume latest |
new |
Opens | 409 not_empty |
A run without --resume |
fork |
Opens a fork, records the source, returns the source signed | 409 not_empty |
--resume <full path> into a new artifact |
There is no way to continue from anything but the latest in place. Any other version is a fork into a new, empty artifact. No bytes are copied: the fork reads its source where it is, and its first version names the source.
epoch 1 box A aws us-east-1 100 200 300 400 500 [510 in flight]
epoch 2 box B azure westeurope open latest → 500 510 520
A's commit of 510: 409 supersededReaders never open a writer. Reading a version, listing versions and listing artifacts need only a key.
Why a used artifact takes nothing but a continuation
A run launched without --resume on a location another run already used would
write its checkpoints between the old run's, while the latest stays the old
run's highest step; its first preemption would then resume into the other run.
So new on a used artifact is refused, before the first step.