Assets¶

Assets are how runs produce durable outputs (checkpoints, reports, datasets, environment snapshots) and how researchers attach research documents to experiments. methodic supports two upload paths depending on size and shape, plus convenience helpers on Run and chronicle.assets.

Inline (small structured payloads)¶

For research reports, environment snapshots, and other small payloads, post the content inline and Chronicle auto-finalizes:

# Worker side — auto-populates output_of from the bound run
run.upload_asset(
    asset_type="takeaways_report",
    content={
        "summary": "Loss converged at step 12000",
        "key_findings": [...],
    },
)

# Researcher side — explicit (no run context)
chronicle.assets.create_inline(
    asset_type="research_report",
    content={"notes": "..."},
)

Reserved asset types Chronicle knows about:

hypothesis_report — full hypothesis document, submitted at experiment creation
takeaways_report — post-experiment analysis
research_report — general research documents (literature review, synthesis)

You can use any other string for custom asset types.

Markdown / plain text¶

For human-readable docs (research reports, notes), pass the text as content and override the content type:

hypothesis = open("hypothesis.md").read()
chronicle.assets.create_inline(
    asset_type="hypothesis_report",
    content=hypothesis,
    content_type="text/markdown",
)

Binary (presigned URL flow)¶

For checkpoints, large datasets, or anything you'd rather not pipe through Chronicle, the upload goes directly to cloud storage. Three steps:

# 1. Register the asset and get presigned URLs (one per component)
upload_info = run.create_asset_presigned(
    asset_type="checkpoint",
    components=["model.safetensors", "optimizer.pt", "metadata.json"],
)

# 2. PUT each component to its presigned URL
for filename, url in upload_info.upload_urls.items():
    run.upload_component(url, local_path=Path(f"./out/{filename}"), content_type="application/octet-stream")

# 3. Finalize — asset transitions to `ready` and becomes immutable
run.finalize_asset(upload_info.asset_id)

For convenience, run.upload_directory_async() does all three steps for a directory of files in the background.

The R2 route¶

Run uploads (and chronicle.datasets.upload) ask Chronicle for the R2 route first: for the asset types and scopes it's enabled on, the bytes go to Cloudflare R2 through Scribe instead of to GCS. Checkpoints, datasets and images then live in R2 alone; other types (model snapshots) get a GCS copy after finalize. Same three steps — what changes is the URLs. upload_info.via is "r2", the URLs point at Scribe and take your API key, and a file larger than upload_info.max_part_bytes goes up as a multipart upload (aborted if a part keeps failing). upload_component handles both routes, so always upload through it rather than PUTting the URL yourself.

Lower-level callers opt in with chronicle.assets.create_with_presigned(..., upload_targets=methodic.assets.UPLOAD_TARGETS_R2_FIRST); without it you get presigned GCS URLs any HTTP client can PUT to.

Checkpoints, datasets and images take the R2 route for every account (an owner can turn assets.r2 off) and live only in R2: reading them costs no egress. A checkpoint expires 180 days after upload unless it's preserved (see Checkpoint expiry); datasets, images and model snapshots never expire. Reports, papers and the other documents search indexes stay in GCS. upload_component returns each file's SHA-256, and passing them to finalize(asset_id, sha256s) records the asset's content hash, which Chronicle keeps after the bytes are gone. Run uploads do this for you.

Once an asset has a ready R2 copy, its read URLs (presign, get(include_presigned=True)) point at R2, whether or not GCS has the bytes too. A private copy's URL is presigned straight against the R2 bucket for a day, like a GCS signed URL; any HTTP client can fetch it without your API key. (A server without R2's S3 API hands out Scribe links with a one-hour token instead.) A public copy's URL is its public address.

Moving existing assets into R2¶

Chronicle moves the checkpoints, datasets and images that are still in GCS into R2 on its own, a few at a time, deleting each GCS copy once R2 has the bytes. To move one now, call chronicle.assets.move_to_r2(asset_id) (Write on the asset): the copy runs in the background, and chronicle.assets.r2_status(asset_id) reports its state (copying, then ready). Reads keep working throughout, from GCS until the R2 copy is ready. The web app's asset page has the same action under Storage. Assets registered by reference, or external, aren't moved.

Checkpoint expiry¶

Keeping every checkpoint a training run saves would pile up storage nobody reads, so checkpoints expire: 180 days after upload, Chronicle deletes a checkpoint's bytes unless it has been preserved. The asset itself stays, in state expired, with its content hash (sha256). Nothing else expires.

A shorter TTL. Pass ttl_days (1 to methodic.CHECKPOINT_TTL_DAYS, 180) when you create a checkpoint, and it expires that many days after upload instead:
```
run.upload_directory_async(Path("./out/checkpoint-1000"), asset_type="checkpoint", ttl_days=30)
```
create_asset_presigned, upload_file, register_and_upload_async and chronicle.assets.create_with_presigned take it too. In menlo-park, set checkpoint_ttl_days: 30 at the top level of the experiment YAML. Other asset types refuse it (BadRequestError, kind ttl_not_supported). - Notices. The activity feed warns 14 and 3 days before checkpoints expire: one yellow notice per experiment, listing the checkpoints due, each with a Preserve button. A checkpoint whose whole TTL is shorter than a notice's lead time skips that notice. - Preserve the checkpoints worth keeping, such as a resume point or a fine-tuning base. chronicle.assets.preserve(asset_id) clears the expiry for good; the bytes move to permanent storage in the background (job_id in the response). It needs Write on the asset and is idempotent. The MCP tool is chronicle.preserve_asset. - Expired. Downloading or presigning an expired checkpoint raises AssetExpiredError (HTTP 410, kind asset_expired), and so does preserving one. The asset's sha256 still identifies the bytes it had.

Asset states¶

State	Meaning
`pending`	Created, components not all uploaded yet
`ready`	Finalized, immutable, available as an input to other variations
`abandoned`	Run failed before finalize; storage may be GC'd
`expired`	A checkpoint past its expiry: bytes deleted, row and `sha256` kept

Validity flags¶

Chronicle may mark an asset (set them yourself under Managing assets):

deprecated — soft warning, asset still usable
invalidated — hard block; Chronicle won't accept it as a new input without allow_invalid_assets: true. Outputs of a retracted experiment are invalidated automatically.

These flags don't change the lifecycle state; they're orthogonal annotations.

Downloading¶

chronicle.assets.download(asset_id, local_dir=Path("./checkpoint"))

Single GET request: chronicle.assets.get(asset_id, include_presigned=True) returns metadata + presigned read URLs in one call, which download consumes. Files stream straight to disk.

When the asset has a ready R2 copy (the r2 block on the asset, state: "ready"), download reads that instead: the copy's manifest, then each object it lists, with your API key (a public copy needs none). If the R2 read fails, it falls back to the presigned GCS URLs.

Managing assets¶

Beyond upload and download, chronicle.assets carries an asset's validity annotations, access control, and ownership. (Run-output approval — approve / reject / bulk_approve — lives in the attention & review guide.)

Deprecate, invalidate, delete¶

Three escalating ways to take an asset out of use, matching the validity flags above:

chronicle.assets.deprecate(asset_id, reason="superseded by the Re=2000 sweep")  # soft warning, still usable
chronicle.assets.invalidate(asset_id, reason="labeling bug in the source data") # hard block as a new input
chronicle.assets.delete(asset_id)                                               # hard delete — only if unlinked

deprecate leaves the asset usable but surfaces reason wherever it's linked ("superseded, but existing results stand"). invalidate blocks it as a new input — link it anyway with allow_invalid_assets=True — while keeping the row, links, and provenance ("wrong data, do not build on this"). delete is the orphan-cleanup path: it removes the row, ACLs, bytes, and search document, and is refused with 409 (asset_linked) while any experiment or variation references it. Reach for deprecate/invalidate on assets that are part of an experiment's record; delete only over-uploaded or abandoned orphans.

Access control¶

Assets carry generic ACLs — datasets are plain assets, so the same calls apply:

chronicle.assets.grant_access(asset_id, principal_id="user_abc", action="read")  # read | write | delete | administer
chronicle.assets.revoke_access(asset_id, principal_id="user_abc", action="read")
chronicle.assets.list_access(asset_id)                                           # who can reach it
chronicle.assets.share_with_scope(asset_id, scope_id="team_9")                   # grant a whole team/org read

principal_id is a user sub, a team/org id, or everyone; resolve team/org ids with chronicle.me.scopes(). grant_access/revoke_access are idempotent and require Administer on the asset (its creator has it). share_with_scope is a thin convenience over grant_access for sharing with an entire scope.

Visibility and ownership¶

set_visibility sets the asset's broadcast read grant independent of any experiment it's linked to; move transfers it into an organization (the asset analog of experiments.move):

chronicle.assets.set_visibility(asset_id, "public")   # "private" | "organization" / "team" | "public"
chronicle.assets.move(asset_id, organization_id="org_123", visibility="organization")

move requires Administer on the asset and membership of the target org; created_by is unchanged.