Bulk harvest

Harvest residual-stream activations over a whole prompt corpus as downloadable files — the batch counterpart to inline harvesting, for when you want thousands to millions of prompts (SAE training, large probing datasets) rather than a few activations in an HTTP response.

You submit a job, poll it, and download safetensors shards by URL. Same API key as everything else. The GPU work runs on a dedicated batch app per model; nothing you do here slows the completions engines.

Submit a job

POST https://infra.acsresearch.org/v1/harvest with your prompts:

curl -s "$ACS_API_BASE/harvest" \
  -H "Authorization: Bearer $ACS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-8b",
    "prompts": ["The quick brown fox jumps over the lazy dog.",
                "Interpretability research reads the residual stream."],
    "layers": [8, 16, 24]
  }'

{ "job_id": "071fa547-…", "status": "running", "model": "llama-8b", "n_prompts": 2 }

(Responses below are abridged — a real one carries a few more bookkeeping keys like run_id and created_at.)

Fields beyond model and prompts (a typo'd field name returns a 400 rather than silently using a default on a paid GPU job):

layers — which decoder-block outputs to keep, e.g. [8, 16, 24]. Omit it for the default quartile subset (blocks at ~25/50/75% depth — [8, 16, 24] on llama-8b); pass the string "all" (lowercase, exact) to keep every block. Unlike the inline endpoint, the harvester subsets server-side, so the 3-layer default on a 32-layer model writes ~10× less data than "all". Same convention as inline: layer k is the output of block k.
shard_size — prompts per output file (default 32).
batch_size — forward-pass batch inside the job. Leave it unset: the default vLLM-Lens backend batches continuously on its own; the value only shapes the HF reference path (HARVEST_USE_VLLM=0).

Poll it

GET https://infra.acsresearch.org/v1/harvest/<job_id> with the same key. status moves from running to done (or failed, with a short error message):

curl -s "$ACS_API_BASE/harvest/071fa547-…" -H "Authorization: Bearer $ACS_API_KEY"

{
  "job_id": "071fa547-…",
  "status": "done",
  "model": "llama-8b",
  "n_prompts": 2,
  "result": {
    "n_shards": 1,
    "layer_indices": [8, 16, 24],
    "manifest_url": "https://…/manifest.json?…",      // presigned — no key needed
    "shard_urls":   ["https://…/shard_00000.safetensors?…"],
    "stats_url":    "https://…/stats.safetensors?…",
    "timings": { "forward_s": 1.59, "upload_s": 4.05, "tokens": 21, /* … */ }
  },
  "urls_expire_at": "2026-07-24T20:31:37+00:00"
}

A small llama-8b job goes submit → done in about a minute, most of it engine start-up. Poll every 20–30 s; there's no webhook.

Download and read the shards

The URLs are presigned and work in a plain browser, curl, or wget — no API key. They expire at urls_expire_at (7 days after completion; the response adds "urls_expired": true once they have).

Each shard holds two tensors per prompt — the activations and the input token ids, so you can line activations up with tokens without re-tokenizing:

import os, requests
from safetensors.torch import load_file

API_BASE = os.environ["ACS_API_BASE"]           # e.g. https://…/v1
KEY = os.environ["ACS_API_KEY"]

job = requests.get(f"{API_BASE}/harvest/{job_id}",
                   headers={"Authorization": f"Bearer {KEY}"}).json()

open("shard0.safetensors", "wb").write(requests.get(job["result"]["shard_urls"][0]).content)
shard = load_file("shard0.safetensors")

shard["prompt_0"]   # bf16, [n_layers_kept, n_tokens, hidden] — e.g. [3, 11, 4096]
shard["tokens_0"]   # int32, [n_tokens] — same order as the tensor's token axis

manifest_url points to a JSON index of every shard (which prompts are in which file, shapes, the layer convention spelled out). stats_url is a small safetensors file with per-layer token mean and std over the whole run — the normalization statistics SAE training wants, with each prompt's BOS token left out of the statistics because its outlier norms would skew them.

Limits & practicalities

One running job per key (the default). A second submit while one is running returns 429 harvest_concurrency_exceeded — poll the first to completion. Your key may also carry a monthly job quota (429 harvest_quota_exceeded).
Corpus caps. Up to 4,096 prompts and ~2M estimated tokens per job, and request bodies up to 50 MB (413 beyond that). For more, split into several jobs.
Prompts longer than the model's context window are truncated to fit, and the token axis you get back matches what actually ran.
Big models are slow to start. llama-405b streams ~810 GB of weights on a cold start — the first job can spend ~50 minutes loading before any prompt runs. The load is paid once per job, so batch your corpus into one large job instead of many small ones.
503 harvest_unavailable means the harvest app for that model isn't reachable right now; the response records a job id you can show us.

Inline or bulk?

Inline (output_residual_stream) answers in the same HTTP response and is right for probing and steering workflows up to ~10k examples. Bulk trades latency for throughput: the capture itself runs ~13× faster than the inline path's transport ceiling, keeps only the layers you ask for, and hands you files you can re-download for a week.