Understanding the envelope

Every read of a job returns one envelope, and which of its keys are filled in depends on status. A client switches on status and finds everything it needs in the same object, without asking a second endpoint what happened.

#When to use it

Read this page before you handle job results in code: it is the contract between Kaabist and your client. If you only need the final data, an SDK's wait helper (at the end of this page) gives you a finished or parked job and you read data, error, input_request or attention from it.

#Creating a job

POST /v1/jobs takes exactly one of three forms:

{ "program": "acme-report@production", "inputs": { "report_id": "42" }, "tags": ["customer:42"] }

{ "url": "https://shop.example.com/p/1", "extract": { "title": "h1" } }

{ "definition": { "kaab": "1", "steps": [ { "goto": "https://shop.example.com/p/1" }, { "extract": { "title": "h1" } } ] } }

The first runs a stored program by reference (name, name@3, name@production, prg_… or pv_…). The second is the shorthand: it compiles to a two-step program at request time. The third sends a whole program inline without storing it. All three are linted, run and billed the same way, and all three answer 202 with the envelope, or 200 with a finished one when you send Prefer: wait. Every form also takes session, options, policies, webhook_url, tags, dry_run and cache.

Two neighbours take the same body:

• POST /v1/jobs/estimate validates it and answers { credits_min, credits_max, tier_expected, cache } without creating anything.
• POST /v1/jobs/batch takes one program (or url + extract) and up to 1 000 items, each with its own inputs or url and tags. It is all or nothing: if one item does not validate, or the balance does not cover the batch, no job is created.

#The envelope

{
  "id": "job_…", "object": "job", "mode": "live",
  "status": "succeeded", "reason": null,
  "program": { "id": "prg_…", "name": "acme-report", "version": 3, "version_id": "pv_…" },
  "inputs_hash": "sha256:…",
  "data": {}, "raw": {}, "warnings": [],
  "meta": {},
  "error": null, "input_request": null, "attention": null, "progress": null,
  "tags": ["customer:42"], "webhook_url": null,
  "created_at": "…", "started_at": "…", "finished_at": "…", "expires_at": "…"
}

#What each status carries

queued and running are on their way; awaiting_input and needs_attention are parked and wait for you; succeeded, failed and canceled are final.

#meta

meta is how the run describes itself. It is present on every status, filled in as the job progresses:

#warnings

warnings[] says "you got an answer, and here is what you should know about it": a null marker parsed as empty, a list shorter than expected, a locator that had to be healed, a job that moved up a tier. Each is { code, field?, raw?, step?, message?, detail?, count?, of? } from a closed set of codes. A warning about every item of a list is aggregated onto the list instead of repeated:

{ "code": "null_marker", "field": "products[].price", "count": 3, "of": 40 }

One warning is not about the job it is on. fill_rate_drop is attached to every job of a program version whose recent runs show a field that has stopped filling; the data in this job may look fine, and there is probably a proposal waiting in your review queue. See Validation and warnings.

#error

On a failed job:

{ "code": "blocked", "message": "challenge page after 3 attempts", "tier": "stealth",
  "retryable": true, "step": "steps[2]", "snapshot": "/v1/jobs/job_…/snapshot", "details": {} }

code is from the closed set of error codes; retryable says whether running the same job again could succeed.

#input_request

On an awaiting_input job: { name, prompt, type, pattern, keep_alive_until, expires_at, collect, screenshot, resolve_url }. Answer it with POST /v1/jobs/{id}/input before expires_at. resolve_url is set when the step collects the value on a hosted page.

#attention

On a needs_attention job: { reason, step, detail, screenshot, snapshot, checkpoint, expires_at, proposals, resolve_url, actions }. reason is one of obstacle, drift, unknown_page, session_expired, side_effect_ambiguous and captcha, and is repeated at the top level of the envelope. actions lists what the job accepts. See Human in the loop.

#Worked example

The site at https://portal.example.com signs in with an e-mail and password, asks for a one-time code, then shows a 40-row report. A program for it, with the code collected by an await_input step, passes through most statuses. The envelopes below are trimmed to the keys that change.

Created (202):

{ "id": "job_c2", "object": "job", "status": "queued", "program": { "name": "portal-report", "version": 1 }, "created_at": "…" }

Running, on step 3 of 8, before any checkpoint:

{ "id": "job_c2", "status": "running", "progress": { "step": 3, "of": 8, "checkpoint": null } }

Parked for the code. The fifth step, steps[4], is a when block whose first step asks for the code:

{ "id": "job_c2", "status": "awaiting_input",
  "progress": { "step": 5, "of": 8, "checkpoint": null },
  "input_request": { "name": "otp", "prompt": "Doğrulama kodu", "type": "string", "pattern": "^[0-9]{6}$",
    "keep_alive_until": "…", "expires_at": "…", "collect": "api",
    "screenshot": "/v1/jobs/job_c2/screenshot?step=steps[4].do[0]", "resolve_url": null } }

You send POST /v1/jobs/job_c2/input with { "otp": "123456" }, and the job is running again in the same browser page.

Succeeded, with the report read:

{ "id": "job_c2", "status": "succeeded",
  "data": { "rows": [ { "date": "…", "description": "…", "amount": { "amount": 53.5, "currency": "TRY" } } ] },
  "warnings": [],
  "meta": { "tier": "browser", "session": { "id": "ses_…", "name": "portal-login", "saved": true },
    "credits": { "reserved": 12, "settled": 7, "breakdown": { "compute": 5, "proxy": 2, "llm": 0, "captcha": 0 } } } }

Needs attention instead, on a later run after the site moved the amount column and the healer was not confident of the new one:

{ "id": "job_c3", "status": "needs_attention", "reason": "drift",
  "attention": { "reason": "drift", "step": "steps[7]", "detail": "extract.rows[].amount: 0/40 matched, landmarks ok",
    "checkpoint": "logged_in", "expires_at": "…",
    "proposals": [ { "id": "prp_2", "kind": "relocate", "target": "extract.rows[].amount", "confidence": 0.86 } ],
    "actions": ["apply", "skip_step", "retry_from", "takeover", "abort"] } }

Failed, on a different site that kept refusing the job:

{ "id": "job_x", "status": "failed",
  "error": { "code": "blocked", "message": "challenge page after 3 attempts", "tier": "stealth", "retryable": true,
    "snapshot": "/v1/jobs/job_x/snapshot" },
  "meta": { "credits": { "reserved": 40, "settled": 25 } } }

Canceled with POST /v1/jobs/{id}/cancel: status: "canceled", its timings and credits, and nothing else. The reservation is released minus what the job had already spent.

#Evidence

GET /v1/jobs/{id}/steps is the timeline: one row per step attempt, with timings, the tier, what the healer did, warnings and links to that step's snapshot and screenshot. GET /v1/jobs/{id}/snapshot, /screenshot and /artifacts/{name} each redirect (302) to a link that is valid for fifteen minutes. Values of secret inputs never appear in any of them.

#Waiting for a job

Prefer: wait=<seconds>, from 1 to 60, holds one request open. Beyond a minute, each SDK has a wait helper that long-polls GET /v1/jobs/{id} until the job is final, and can answer it on the way: an onInput handler for awaiting_input (its value is posted to /input) and an onAttention handler for needs_attention (its action is posted to /resume, or it leaves the job to somebody else).

// Or start it and wait separately: `jobs.wait` long-polls until the job ends.
const started = await kaab.jobs.create({
  url: `${site}/p/2`,
  extract: { title: "h1" },
});
const done = await kaab.jobs.wait(started.id, { timeoutMs: 120_000 });
console.log(`${done.id}: ${done.status} ${JSON.stringify(done.data)}`);

# Or start it and wait separately: jobs.wait long-polls until the job ends.
started = kaab.jobs.create(url=f"{site}/p/2", extract={"title": "h1"})
done = kaab.jobs.wait(started.id, timeout=120)
print(f"{done.id}: {done.status}")

// Or start it and wait separately: Jobs.Wait long-polls until the job ends.
body, _ = kaab.Body[client.CreateJobJSONBody](map[string]any{"url": site + "/p/2", "extract": map[string]any{"title": "h1"}})
started, err := c.Jobs.Create(ctx, body)
if err != nil {
    log.Fatal(err)
}
done, err := c.Jobs.Wait(ctx, started.ID, kaab.WaitOptions{Timeout: 2 * time.Minute})
if err != nil {
    log.Fatal(err)
}
fmt.Printf("%s: %s\n", done.ID, done.Status)

Every helper returns at once on succeeded, failed or canceled. A parked job with no handler for its state is returned as it is, because it is a state your code has to act on. Each handler is asked once per park. With the CLI, kaab jobs run … --wait exits 2 for a failed or canceled job and 3 for a parked one.

#Pitfalls

• Switching on data. Switch on status. data is present on succeeded, and on failed only as a partial result.
• Ignoring warnings on success. A succeeded job with null_marker or list_short returned an answer you may not want; with policies.strict: true any warning fails the job instead.
• Treating parked as stuck. An awaiting_input job fails with input_expired once input_request.expires_at passes, and a needs_attention job fails at attention.expires_at (seven days by default). Answer, resume or cancel it before then.
• Long-polling forever. A wait helper with no onInput returns a job parked for input; loop on it only after you have answered.
• Expecting 409 to be a bug. 409 job_not_waiting means the verb does not fit the job's state; 409 fence_open means the job is inside an unconfirmed side effect and cannot be canceled or rewound past it.

#Reference

• Get a job, create a job, estimate, create a batch
• List a job's steps, cancel a job
• The job object and the step record
• Job statuses, attention reasons, warnings, errors
• SDK reference