# Tiers and escalation

Which tier a job starts at and why it moves up, what Kaabist learns about each site, when a discovered API makes a job cheaper, and the shared cache.

Kaabist fetches pages in three ways that differ widely in cost, and it picks the cheapest one that can read your page. A job starts as low as its program allows, moves up when the page turns out to need more, and Kaabist remembers what each site needed, so later jobs against the same site start at the right tier.

## When to use it

You do not choose a tier for each job; read this page to understand what a job cost and why. Set `options.max_tier` to cap spending, `options.prefer_tier` when you already know a site needs a browser, and `options.cache` when many jobs read the same public page.

## The three tiers

| Tier | What it is | Compute credits |
|---|---|---|
| `http` | one request with a real browser's TLS fingerprint, one parsed document, no JavaScript | 1 per request |
| `browser` | a real Chromium, with JavaScript, clicks and typing | 5, plus 1 per started 10 s beyond the first 30 s |
| `stealth` | a hardened browser on a residential address | 20, plus 1 per started 10 s beyond the first 30 s |

Proxy traffic is billed on top, per megabyte by pool: datacenter is the default, residential costs ten times as much. See [The http tier](https://docs.kaab.ist/concepts/http-tier), [The browser tier](https://docs.kaab.ist/concepts/browser-tier) and [Credits and billing](https://docs.kaab.ist/concepts/credits-and-billing).

## Where a job starts

In order:

1. **What your program does.** A program with a `click`, `fill`, `press`, `hover`, `select`, `check`, `uncheck`, `upload`, `scroll`, `wait_for`, `evaluate`, `screenshot`, `download`, `paginate`, `await_input`, `set_session`, `capture_response` or `loop_until` cannot run without a browser, so it starts at `browser`. A program that only navigates and extracts starts at `http`.
2. **`options.prefer_tier`.** A floor, not a ceiling: ask for `browser` and the job does not start lower.
3. **What Kaabist has learned about the hostname.** When the site profile is confident that a document is not enough, the job starts at the tier the site needs and does not pay for an attempt that would fail.
4. **`options.max_tier`** caps all of the above, and your plan caps `max_tier`. A job that needs a tier above its cap fails with `max_tier_exceeded` rather than spending what you did not agree to.

The envelope records the start in `meta.tier_requested` and the tier the job finished at in `meta.tier`.

## Why a job moves up

When a step fails, Kaabist classifies the page before reacting, and only some classes mean "this tier cannot read this page":

| Class | What it means | What happens |
|---|---|---|
| `empty_shell` | the document is a mount point and a script bundle; the content does not exist yet | moves to `browser` |
| `placeholder_content` | the elements are there but print `-`, an em dash (U+2014), `Loading` or a skeleton | at `http`, moves to `browser` once; if the browser reads the same, the value is treated as empty |
| `challenge_page` | an anti-bot interstitial (Cloudflare, DataDome, PerimeterX, Akamai, Imperva) | handled by `policies.captcha`, like a puzzle, and recorded on the site profile, so later jobs start at the tier and on the proxy pool the site needs (residential addresses at `stealth`) |
| `captcha` | a puzzle: reCAPTCHA, hCaptcha, Turnstile, GeeTest, Arkose | does not move up, because a better address does not solve a puzzle; follows `policies.captcha`: `park` (the default), `solve` or `fail` |

Moving up is not a failure. Kaabist takes a checkpoint, requeues the job at the next tier and carries on from there; the job keeps its id, inputs and envelope. Each hop is recorded:

```json
{
  "meta": {
    "tier": "browser",
    "tier_requested": "http",
    "escalations": [{ "from": "http", "to": "browser", "reason": "empty_shell" }]
  },
  "warnings": [{ "code": "escalated", "step": "steps[0]", "message": "empty_shell" }]
}
```

`reason` is the class, sometimes followed by the signature that identified it (`challenge_page:<signature>`), or `tier_insufficient:<action>` when a tier was asked to do something it cannot.

Everything else is about your program, not the tier: a moved element goes to the [healer](https://docs.kaab.ist/concepts/self-healing), an expired session to its [refresh program](https://docs.kaab.ist/concepts/sessions), a dialog to the [obstacle layer](https://docs.kaab.ist/concepts/obstacles), and a page nobody recognises parks the job for a person ([Human in the loop](https://docs.kaab.ist/concepts/human-in-the-loop)).

## What Kaabist remembers about a site

Per hostname, Kaabist keeps the tier that works and how confident it is, the proxy pool the site needs, the endpoints it has been seen to serve, the obstacles it shows, and when it last refused a job. `GET /v1/sites/{hostname}` shows the part that is yours to see:

```json
{ "hostname": "shop.example", "object": "site_profile",
  "tier_required": "browser", "tier_confidence": 0.9,
  "known_obstacles_count": 2, "api_endpoints_count": 1,
  "last_blocked_at": "2026-09-13T08:41:00Z" }
```

A hostname nothing is known about returns an empty profile, not a `404`. Two rules shape what is learned:

- **A profile is re-tested.** Confidence rises slowly with successes and jumps on a refusal. A profile that has not been re-tested for a week is stale, and one job in ten deliberately starts a rung lower to find out whether the site got easier. A site that fixed its rendering stops costing you a browser.
- **What a job teaches depends on what the job was.** A public page teaches the platform's profile, which every customer benefits from. A job carrying your session, your own proxy or an `evaluate` step teaches only your organisation's profile: what a site does behind your login is not a fact about the site.

## When a discovered API makes a job cheaper

A page that renders from JSON got that JSON from somewhere. After a successful browser-tier run, Kaabist compares the values it extracted with the JSON responses the page fetched, and when one explains the page it writes the locators that would read it directly. It then checks them by running the extraction against that endpoint alone and comparing the result with what the browser produced; anything that does not match exactly is discarded.

What survives becomes a **proposal** on your program, of kind `api_discovery`, in the review queue. Nothing changes until you approve it. Once approved, the program reads the endpoint instead of the rendered page and can run at the `http` tier: the same data, without a browser.

## The shared cache

Two jobs that read the same public page inside the same window can share one fetch. Set `cache.max_age` (1 to 86400 seconds) on the job, and Kaabist serves a stored copy when it has one. `meta.cache` is `hit` or `miss` on every job, and a hit adds the warning `cache_hit`. `POST /v1/jobs/estimate` tells you in advance whether a job could be served from the cache (`cache: "possible"`).

A job is eligible only when it is a plain public fetch:

- the program is exactly one `goto` and one `extract` into `data`, with no `expect`, no `if` and no side effect;
- the URL carries no template;
- the job has no session and no inputs.

`cache` on a job that is not eligible is ignored, not refused. The key is the normalized URL, the tier, the hash of your extract schema and the job's mode, so test and live jobs never share entries. Normalizing drops the fragment, the default port, the case of the scheme and host, the order of query parameters and campaign parameters (`utm_*`, `fbclid`, `gclid`, `_ga`, `mc_cid`, `mc_eid`); it keeps everything else, including a trailing slash. Pages that answered with an error, and pages Kaabist had to diagnose, are never stored. A hit pays its compute credit and no proxy traffic.

## Worked example

The single-page app at `https://app.example.com` serves an empty `#root` shell: its script fetches the product list as JSON half a second after load and draws the cards. Store a program that waits for the cards and reads them:

```json
{
  "kaab": "1",
  "name": "app-products",
  "options": { "max_tier": "browser" },
  "steps": [
    { "goto": "https://app.example.com/", "expect": { "count": { "target": "#products .product-card", "min": 1 } } },
    { "extract": { "products": { "selector": "#products .product-card", "many": true, "min_items": 1,
        "fields": { "title": ".title", "price": { "selector": ".price", "type": "money", "locale": "tr", "currency": "TRY" } } } } }
  ]
}
```

**First run.** Nothing in the program needs a browser, so the job starts at `http`. The document has no cards, the `expect` fails, and the page is classified `empty_shell`. The job moves to `browser`, where the script runs and the cards appear, and it succeeds. The envelope shows `tier_requested: "http"`, `tier: "browser"`, one entry in `meta.escalations` with reason `empty_shell`, and an `escalated` warning. It paid for one http request and one browser run.

**Later runs.** The failed `http` attempt is recorded on the site's profile. Once the profile is confident that the site needs a browser, jobs start there: `tier_requested` is `browser`, `escalations` is empty, and no http request is wasted.

**After discovery.** The browser run saw the page fetch its product list as JSON, and the values in it match what was extracted. An `api_discovery` proposal appears in your review queue, replacing the CSS locators with `network` locators on that endpoint. Approve it with `promote: true` and a new version is published that reads the JSON directly; once nothing in the program needs the rendered page, it runs at the `http` tier, one compute credit instead of five.

## Pitfalls

- **A `max_tier` that is too low.** A program capped at `http` against a JavaScript site fails `max_tier_exceeded` on its first run. The free plan is capped at `http`.
- **Assuming the first run's cost is the steady state.** The first job against a new site may pay for a failed lower tier; later jobs do not.
- **Expecting a captcha or a challenge to escalate.** Both follow `policies.captcha`, which parks by default. Set `policies.captcha: "solve"` to use a solver, which is billed per solve.
- **Expecting the cache to share personal pages.** Any session or input makes a job ineligible, by design.

## Reference

- [Program options](https://docs.kaab.ist/reference/dsl/options) (`max_tier`, `prefer_tier`, `cache`)
- [Get a site profile](https://docs.kaab.ist/reference/api/sites/get-site-profile) and [the site profile object](https://docs.kaab.ist/reference/api/schemas/site-profile)
- [Estimate a job](https://docs.kaab.ist/reference/api/jobs/estimate-job)
- [Tiers](https://docs.kaab.ist/reference/codes#tiers), [diagnose classes](https://docs.kaab.ist/reference/codes#diagnose-classes), [warnings](https://docs.kaab.ist/reference/codes#warnings)
- [Approve a proposal](https://docs.kaab.ist/reference/api/proposals/approve-proposal)