Tiers and escalation

Kaabist fetches pages in three ways that differ widely in cost, and it picks the cheapest one that can read your page. A job starts as low as its program allows, moves up when the page turns out to need more, and Kaabist remembers what each site needed, so later jobs against the same site start at the right tier.

#When to use it

You do not choose a tier for each job; read this page to understand what a job cost and why. Set options.max_tier to cap spending, options.prefer_tier when you already know a site needs a browser, and options.cache when many jobs read the same public page.

#The three tiers

Proxy traffic is billed on top, per megabyte by pool: datacenter is the default, residential costs ten times as much. See The http tier, The browser tier and Credits and billing.

#Where a job starts

In order:

1. What your program does. A program with a click, fill, press, hover, select, check, uncheck, upload, scroll, wait_for, evaluate, screenshot, download, paginate, await_input, set_session, capture_response or loop_until cannot run without a browser, so it starts at browser. A program that only navigates and extracts starts at http.
2. options.prefer_tier. A floor, not a ceiling: ask for browser and the job does not start lower.
3. What Kaabist has learned about the hostname. When the site profile is confident that a document is not enough, the job starts at the tier the site needs and does not pay for an attempt that would fail.
4. options.max_tier caps all of the above, and your plan caps max_tier. A job that needs a tier above its cap fails with max_tier_exceeded rather than spending what you did not agree to.

The envelope records the start in meta.tier_requested and the tier the job finished at in meta.tier.

#Why a job moves up

When a step fails, Kaabist classifies the page before reacting, and only some classes mean "this tier cannot read this page":

Moving up is not a failure. Kaabist takes a checkpoint, requeues the job at the next tier and carries on from there; the job keeps its id, inputs and envelope. Each hop is recorded:

{
  "meta": {
    "tier": "browser",
    "tier_requested": "http",
    "escalations": [{ "from": "http", "to": "browser", "reason": "empty_shell" }]
  },
  "warnings": [{ "code": "escalated", "step": "steps[0]", "message": "empty_shell" }]
}

reason is the class, sometimes followed by the signature that identified it (challenge_page:<signature>), or tier_insufficient:<action> when a tier was asked to do something it cannot.

Everything else is about your program, not the tier: a moved element goes to the healer, an expired session to its refresh program, a dialog to the obstacle layer, and a page nobody recognises parks the job for a person (Human in the loop).

#What Kaabist remembers about a site

Per hostname, Kaabist keeps the tier that works and how confident it is, the proxy pool the site needs, the endpoints it has been seen to serve, the obstacles it shows, and when it last refused a job. GET /v1/sites/{hostname} shows the part that is yours to see:

{ "hostname": "shop.example", "object": "site_profile",
  "tier_required": "browser", "tier_confidence": 0.9,
  "known_obstacles_count": 2, "api_endpoints_count": 1,
  "last_blocked_at": "2026-09-13T08:41:00Z" }

A hostname nothing is known about returns an empty profile, not a 404. Two rules shape what is learned:

• A profile is re-tested. Confidence rises slowly with successes and jumps on a refusal. A profile that has not been re-tested for a week is stale, and one job in ten deliberately starts a rung lower to find out whether the site got easier. A site that fixed its rendering stops costing you a browser.
• What a job teaches depends on what the job was. A public page teaches the platform's profile, which every customer benefits from. A job carrying your session, your own proxy or an evaluate step teaches only your organisation's profile: what a site does behind your login is not a fact about the site.

#When a discovered API makes a job cheaper

A page that renders from JSON got that JSON from somewhere. After a successful browser-tier run, Kaabist compares the values it extracted with the JSON responses the page fetched, and when one explains the page it writes the locators that would read it directly. It then checks them by running the extraction against that endpoint alone and comparing the result with what the browser produced; anything that does not match exactly is discarded.

What survives becomes a proposal on your program, of kind api_discovery, in the review queue. Nothing changes until you approve it. Once approved, the program reads the endpoint instead of the rendered page and can run at the http tier: the same data, without a browser.

#The shared cache

Two jobs that read the same public page inside the same window can share one fetch. Set cache.max_age (1 to 86400 seconds) on the job, and Kaabist serves a stored copy when it has one. meta.cache is hit or miss on every job, and a hit adds the warning cache_hit. POST /v1/jobs/estimate tells you in advance whether a job could be served from the cache (cache: "possible").

A job is eligible only when it is a plain public fetch:

• the program is exactly one goto and one extract into data, with no expect, no if and no side effect;
• the URL carries no template;
• the job has no session and no inputs.

cache on a job that is not eligible is ignored, not refused. The key is the normalized URL, the tier, the hash of your extract schema and the job's mode, so test and live jobs never share entries. Normalizing drops the fragment, the default port, the case of the scheme and host, the order of query parameters and campaign parameters (utm_*, fbclid, gclid, _ga, mc_cid, mc_eid); it keeps everything else, including a trailing slash. Pages that answered with an error, and pages Kaabist had to diagnose, are never stored. A hit pays its compute credit and no proxy traffic.

#Worked example

The single-page app at https://app.example.com serves an empty #root shell: its script fetches the product list as JSON half a second after load and draws the cards. Store a program that waits for the cards and reads them:

{
  "kaab": "1",
  "name": "app-products",
  "options": { "max_tier": "browser" },
  "steps": [
    { "goto": "https://app.example.com/", "expect": { "count": { "target": "#products .product-card", "min": 1 } } },
    { "extract": { "products": { "selector": "#products .product-card", "many": true, "min_items": 1,
        "fields": { "title": ".title", "price": { "selector": ".price", "type": "money", "locale": "tr", "currency": "TRY" } } } } }
  ]
}

First run. Nothing in the program needs a browser, so the job starts at http. The document has no cards, the expect fails, and the page is classified empty_shell. The job moves to browser, where the script runs and the cards appear, and it succeeds. The envelope shows tier_requested: "http", tier: "browser", one entry in meta.escalations with reason empty_shell, and an escalated warning. It paid for one http request and one browser run.

Later runs. The failed http attempt is recorded on the site's profile. Once the profile is confident that the site needs a browser, jobs start there: tier_requested is browser, escalations is empty, and no http request is wasted.

After discovery. The browser run saw the page fetch its product list as JSON, and the values in it match what was extracted. An api_discovery proposal appears in your review queue, replacing the CSS locators with network locators on that endpoint. Approve it with promote: true and a new version is published that reads the JSON directly; once nothing in the program needs the rendered page, it runs at the http tier, one compute credit instead of five.

#Pitfalls

• A max_tier that is too low. A program capped at http against a JavaScript site fails max_tier_exceeded on its first run. The free plan is capped at http.
• Assuming the first run's cost is the steady state. The first job against a new site may pay for a failed lower tier; later jobs do not.
• Expecting a captcha or a challenge to escalate. Both follow policies.captcha, which parks by default. Set policies.captcha: "solve" to use a solver, which is billed per solve.
• Expecting the cache to share personal pages. Any session or input makes a job ineligible, by design.

#Reference

• Program options (max_tier, prefer_tier, cache)
• Get a site profile and the site profile object
• Estimate a job
• Tiers, diagnose classes, warnings
• Approve a proposal