Concepts

Each concept page explains one idea Kaabist is built on: what it is, why it behaves the way it does, and what it looks like on a real job. Read them in order the first time; afterwards, each page stands alone.

When to use it

Read a concept page when you want to understand why a job did what it did, or before you design a program that depends on that behaviour. For a step-by-step task, go to the guides; for exact fields and codes, go to the reference.

How programs run

• Programs and versions: a program is versioned JSON; versions never change, aliases point at them, and lint runs before you publish.
• Tiers and escalation: the three ways Kaabist fetches a page, where a job starts, why it moves up, and what Kaabist remembers about a site.
• The http tier: one request and no browser. What it can read, and what makes a job need more.
• The browser tier: a real Chromium per job, with isolation, asset blocking, real clicks and the limits that bound it.
• Sessions: staying signed in between jobs, with storage state, locking, validity checks, expiry and refresh programs.
• Human in the loop: parking, the five resume actions, inputs, hosted pages, takeover, checkpoints and expiry.

When pages change or get in the way

• Self-healing: what Kaabist does when an element moves, and why a change to your program is always a proposal.
• Obstacles: dialogs, consent banners, tours and captchas, and how each is dismissed or handed to a person.
• Validation and warnings: how a page becomes typed data, and the warnings you get instead of a guess.
• Locators: how a field points at an element, and why it keeps several candidates.
• Snapshots: what Kaabist stores for each step, and what it never stores.
• Documents and PDFs: the security gate every PDF passes before it is stored or read, and the codes it refuses a file with.
• How research finds and verifies data: the model decides where, Kaabist extracts; why every research value is verbatim and sourced.

Running at scale

• Credits and billing: what a job costs, how credits are reserved and settled, and what happens when they run out.
• Limits: request rates, concurrency, test-mode quotas and what each plan includes.
• Egress and proxies: how traffic reaches a site, proxy pools and countries, and why a request was refused.
• Queueing: why a job starts when it does, through fairness between organisations, concurrency and recovery.

Accounts and data

• Accounts and roles: organisations, members, the five roles and two-factor authentication.
• Data retention: what Kaabist stores about a job, where, and when it is deleted.

Worked example

One job touches most of these pages. A program that reads a report behind the login of https://portal.example.com runs at the browser tier because it fills a form; it parks for a one-time code; it saves a session so the next run skips the login; when the site ships a redesign that moves the amount column from td.amount to td[data-col=amount], healing proposes the new locator; and the run is billed in credits for its browser time and proxy traffic.

Pitfalls

• Reading concepts as API reference. Concept pages describe behaviour and name the fields involved; the exact shapes and every allowed value are in the generated reference.
• Skipping the tier and parking pages. Most surprises (a job that cost more than expected, a job that stopped) are explained by those two.

Reference

• API reference
• DSL reference
• Codes