Snapshots

Every step of a job leaves a snapshot: the page as Kaabist saw it, plus the network, console and timing that explain how it got there. Snapshots are what you look at when a run did something you did not expect, what the builder renders when you pick a field, what healing compares against when a site changes, and what you can re-run a new program version against without fetching anything.

#When to use it

Open a step's snapshot when a job failed, parked, or returned a value you did not expect, to see the page the step actually had in front of it. Replay a draft version against a stored snapshot when you change an extract schema, so you can check the result before you publish or spend credits. Read the "never contains" section before you automate a site with sensitive data.

#What a snapshot contains

• The page. The HTML after the page settled, not the HTML the server sent. Stylesheets are inlined and every link, image and form action is rewritten to an absolute address, so the stored copy renders faithfully weeks later with no access to the site. Content behind an open shadow root is included; content behind a closed one cannot be read by anything, including Kaabist, and is only counted.
• Same-origin frames. Each is stored separately, with the path to the <iframe> that holds it. A cross-origin frame is recorded as present but its content is not stored.
• Network activity. Every request the page made, with its method, status, type, size and timing. Response bodies are kept only for JSON and plain text, and only up to 512 KB each; that is what makes reading the API the page already calls possible.
• Console output, truncated to 2 KB per message.
• Embedded data. An index of the machine-readable payloads found in the page (__NEXT_DATA__, a Nuxt or Redux store, ld+json), so extraction can read structured data instead of scraping it.
• Page facts: the URL asked for and the final URL after redirects, the HTTP status, a few response headers, the tier, timings, and counts of nodes, frames, forms, scripts and shadow roots.
• A screenshot, when the step failed, the job needs your attention, or you asked for one.

#What a snapshot never contains

Cookies, storage state, passwords and one-time codes are never written. Password fields and inputs whose name looks like a secret (token, otp, code, cvv and similar) are emptied before the page is serialised, and marked so you can see that a value was there. Set-Cookie is removed from the stored headers, and request and response headers in the network log are limited to content-type and accept. This is structural, not a filter applied afterwards: the stored format has no place to put them.

#Fetching a step's snapshot

GET /v1/jobs/{id}/steps lists every step a job ran, each attempt separately, with its status, timings, warnings, any healing or obstacle record, and links to its snapshot and screenshot:

{
  "data": [
    {
      "step_path": "steps[1]", "attempt": 0, "action": "extract", "status": "ok", "tier": "http",
      "started_at": "…", "finished_at": "…", "duration_ms": 41,
      "snapshot": "/v1/jobs/job_01J…/snapshot?step=steps[1]",
      "screenshot": null, "error": null, "warnings": [], "healed": null, "obstacle": null,
      "result_preview": null
    }
  ]
}

GET /v1/jobs/{id}/snapshot?step=steps[1] answers 302 to a link to that step's snapshot JSON, valid for fifteen minutes; without step it is the final step's. GET /v1/jobs/{id}/screenshot?step=… does the same for the screenshot. A finished job's envelope carries the final snapshot's link as meta.snapshot, and a failed job's error.snapshot points at the page it failed on. From the command line, kaab jobs snapshot JOB_ID --step steps[1] downloads it.

#Replaying a program against a snapshot

POST /v1/programs/{id}/versions/{n}/replay runs a version's extraction against a stored page instead of fetching one:

{ "snapshot": "job_01J…:steps[1]" }

It answers { "data", "raw", "warnings" }, exactly what a job would have returned, and costs no credits. Use "snapshot": "ground_truth" to replay against the pages the version's fields were confirmed on. The version may be a draft, so you can try a change before you publish it:

// Run version 1 once: every step stores a snapshot of the page.
const run = await kaab.jobs.create({ program: `${name}@1`, wait: 30 });

// Version 2 reads one more field. Try it on the stored page: no fetch, no credits.
const v2 = await kaab.programs.versions.create(program.id, {
  publish: false, // a draft until the replay looks right
  definition: {
    kaab: "1",
    name,
    options: { max_tier: "http" },
    steps: [
      { goto: `${site}/p/3` },
      {
        extract: {
          title: "h1",
          price: { selector: ".price", type: "money" },
          sku: { selector: ".sku", regex: "SKU:\\s*([A-Z0-9-]+)" },
        },
      },
    ],
  },
});
const replay = await kaab.programs.versions.replay(program.id, v2.version, {
  snapshot: `${run.id}:steps[1]`,
});
console.log(`v${v2.version} on ${run.id}'s page: ${JSON.stringify(replay.data)}`);

#In the panel

On a job's page, the Steps tab lists each step with its status, duration and a thumbnail of its screenshot. Expanding a step shows Open the stored page, which opens the snapshot beside the list. The page is scaled to fit the pane, and when the program version has ground truth for its fields, the elements those fields were confirmed on are outlined, with the note "Outlined: fields with ground truth". From the Error tab, the same viewer opens the page the job failed on.

The viewer renders a sanitised copy in a sandboxed frame: scripts, inline event handlers, javascript: links and nested frame sources are removed, and a policy that forbids scripts entirely is applied on top. The page looks the same and nothing in it runs. The builder uses the same copy when you pick fields on a stored page.

#How long they are kept

Snapshots and screenshots follow your plan's retention: 7, 30 or 90 days, see Plans and pricing. The snapshots a program version's fields were confirmed against are exempt and live as long as the version is in use, because they are what healing compares against. A builder draft's own timeline is kept for 7 days. See Data retention for everything else.

#Worked example

A program reads a shop's product page https://shop.example.com/p/3 at the http tier, with title and price.

1. Run it once. GET /v1/jobs/{id}/steps shows two steps; steps[1], the extract, has a snapshot link.
2. Follow GET /v1/jobs/{id}/snapshot?step=steps[1]: the snapshot JSON has final_url ending in /p/3, status: 200, tier: "http", and an embedded_json entry of kind next_data for the page's __NEXT_DATA__ script.
3. Create version 2 as a draft (publish: false) that also reads sku with the regex SKU:\s*([A-Z0-9-]+).
4. Replay version 2 against "<job id>:steps[1]". The answer's data has title, price and sku, read from the page the first job stored. Nothing was fetched and nothing was charged.
5. Publish version 2 when the replay looks right.

#Pitfalls

• The link expires. A snapshot link is valid for fifteen minutes; ask the API for a fresh one rather than storing it.
• Network bodies are capped. A JSON response over 512 KB is recorded without its body, so a network locator cannot read it.
• A closed shadow root is empty in the snapshot. It is empty to every script, and a field inside one cannot be extracted at any tier.
• Replay reads; it does not act. It runs extraction against a stored page. A change to steps that click or fill needs a real run, or a test run in the builder.
• Retention is per plan. A snapshot past your plan's retention is gone; replay against something recent.

#Reference

• List a job's steps, get a step's snapshot, get a screenshot
• Replay a program version
• Step record schema
• kaab jobs snapshot and kaab jobs steps