# Migrating from other tools

Move Playwright or Puppeteer scripts, scraping-API calls and no-code scrapers to Kaabist programs, with a concept map and a worked conversion.

Most automation you already have is a script that drives a browser, a call to a scraping API, or a
point-and-click scraper. This guide maps each of them onto Kaabist's programs, locators, tiers and
schedules, and converts a small Playwright script step by step.

## When to use it

Use this guide when you have working automation and want Kaabist to run it: to stop maintaining
browsers and proxies yourself, to get typed and validated data back, or to have a changed page
parked and healed instead of breaking silently.

Kaabist runs programs written in its own JSON language, not your scripts. If your script depends on
arbitrary code at every step (complex in-page computation, custom protocols, a desktop app), a
program can still call `evaluate`, but that step is opaque to healing, and you may be better served
keeping that part where it is.

## Concept map

### From Playwright or Puppeteer

| In your script | In a program |
|---|---|
| `page.goto(url)` | [`goto`](https://docs.kaab.ist/reference/dsl/actions/goto) |
| `locator.fill(value)`, Puppeteer `page.type` | [`fill`](https://docs.kaab.ist/reference/dsl/actions/fill) |
| `locator.click()` | [`click`](https://docs.kaab.ist/reference/dsl/actions/click) |
| `keyboard.press("Enter")` | [`press`](https://docs.kaab.ist/reference/dsl/actions/press) |
| `locator.hover()` | [`hover`](https://docs.kaab.ist/reference/dsl/actions/hover) |
| `selectOption`, `check`, `uncheck` | [`select`](https://docs.kaab.ist/reference/dsl/actions/select), [`check`](https://docs.kaab.ist/reference/dsl/actions/check), [`uncheck`](https://docs.kaab.ist/reference/dsl/actions/uncheck) |
| `setInputFiles` | [`upload`](https://docs.kaab.ist/reference/dsl/actions/upload), from files uploaded to your organisation |
| `textContent`, `getAttribute`, `$$eval` | an [`extract`](https://docs.kaab.ist/reference/dsl/actions/extract) schema: fields, types, lists |
| `waitForSelector`, `expect(locator).toBeVisible()` | a step's `expect`, or [`wait_for`](https://docs.kaab.ist/reference/dsl/actions/wait-for) |
| `waitForTimeout(ms)` | [`sleep`](https://docs.kaab.ist/reference/dsl/actions/sleep), which lint warns about: wait for a condition instead |
| `waitForResponse` + `response.json()` | [`capture_response`](https://docs.kaab.ist/reference/dsl/actions/capture-response), or a `network` field |
| `page.evaluate(fn)` | [`evaluate`](https://docs.kaab.ist/reference/dsl/actions/evaluate) (paid plans; the step cannot be healed) |
| `waitForEvent("download")` | [`download`](https://docs.kaab.ist/reference/dsl/actions/download) |
| `page.screenshot()` | [`screenshot`](https://docs.kaab.ist/reference/dsl/actions/screenshot) |
| `context.storageState()` | a [session](https://docs.kaab.ist/concepts/sessions), saved by a `checkpoint` with `save_session` |
| a `for` loop over "next page" | [`paginate`](https://docs.kaab.ist/reference/dsl/actions/paginate) |
| `while (…)` | [`loop_until`](https://docs.kaab.ist/reference/dsl/actions/loop-until), with a required `max` |
| `if (…)` | [`when`](https://docs.kaab.ist/reference/dsl/actions/when), or a step's `if` |
| `prompt()` for a one-time code | [`await_input`](https://docs.kaab.ist/reference/dsl/actions/await-input): the job parks until you answer |

### Selectors become locators

A Playwright locator names one way to find an element. A Kaabist target can name several, as
`candidates`, tried in order until one matches exactly one element:

| Playwright | Target |
|---|---|
| `page.getByTestId("price")` | `{ "testid": "price" }` |
| `page.getByRole("button", { name: "Next" })` | `{ "role": "button", "name": "Next" }` |
| `page.getByText("Sign in", { exact: true })` | `{ "text": "Sign in", "exact": true }` |
| `page.locator(".price")` | `".price"` or `{ "css": ".price" }` |
| `page.locator("xpath=…")` | `{ "xpath": "…" }` |
| `.nth(2)` | `"nth": 2` |
| `page.frameLocator("#pay")` | `"frame": ["#pay"]` |

A match on more than one element is a failure (`target_ambiguous`), not "the first one". Give the
most stable locator first: a test id, then a role and name, then text, then CSS. See
[locators](https://docs.kaab.ist/concepts/locators).

### Retries become tiers and healing

Your script probably wraps steps in retries and sleeps. Kaabist decides why a step failed and
reacts to the cause: a page that needs JavaScript moves the job to a browser
([tiers](https://docs.kaab.ist/concepts/tiers-and-escalation)), a moved element is relocated from its ground truth
([self-healing](https://docs.kaab.ist/concepts/self-healing)), a cookie banner is dismissed
([obstacles](https://docs.kaab.ist/concepts/obstacles)), and network errors are retried with backoff. A step-level
`retry` exists for the rest, except on steps that change something
([side-effect safety](https://docs.kaab.ist/guides/side-effect-safety)).

### Cron becomes schedules

A cron job that runs your script becomes a [schedule](https://docs.kaab.ist/guides/schedules-and-change-detection) on a
program, with a time zone, an overlap policy and optional change detection that tells you only
when the result differs.

### From a scraping API

A call that sends a URL and selectors maps onto the `url` + `extract` shorthand of
`POST /v1/jobs`: one request, one job, typed `data` back. Kaabist starts on the cheapest tier
and escalates on its own, so you do not choose "render JavaScript" per call; cap the cost with
`max_tier`. When you call the same page shape repeatedly, save it as a program instead and run it
by name.

### From a no-code scraper

Point-and-click selection is the panel's picker: open the page in the
[program builder](https://docs.kaab.ist/panel/builder), click a value, press `s` to turn one item into a list, and
confirm. See [picking and recording](https://docs.kaab.ist/panel/picking-and-recording). The result is an ordinary
program you can export as JSON and run from the API.

## Worked example

This Playwright script reads the product cards of a shop's category list, page after page:

```ts
import { chromium } from "playwright";

const browser = await chromium.launch();
const page = await browser.newPage();
await page.goto("https://shop.example.com/c/shoes?page=1");

const products = [];
for (let n = 0; n < 5; n++) {
  for (const card of await page.locator(".product-card").all()) {
    const price = await card.locator(".price").innerText(); // "1.337,40 TL"
    products.push({
      title: await card.locator(".title").innerText(),
      price: Number(price.replace(" TL", "").replaceAll(".", "").replace(",", ".")),
      sku: (await card.locator(".sku").innerText()).replace("SKU: ", ""),
    });
  }
  const next = page.locator("a[rel=next]");
  if ((await next.count()) === 0) break;
  await next.click();
  await page.waitForLoadState();
}
await browser.close();
```

The same work as a program:

```json
{
  "kaab": "1",
  "name": "shoe-catalogue",
  "options": { "max_tier": "http" },
  "steps": [
    {
      "goto": "https://shop.example.com/c/shoes?page=1",
      "expect": { "count": { "target": ".product-card", "min": 1 } }
    },
    {
      "paginate": { "next": "a[rel=next]", "max_pages": 5, "stop_when": { "absent": "a[rel=next]" } },
      "extract": {
        "products": {
          "selector": ".product-card",
          "many": true,
          "fields": {
            "title": ".title",
            "price": { "selector": ".price", "type": "money", "locale": "tr", "currency": "TRY" },
            "sku": { "selector": ".sku", "regex": "SKU:\\s*([A-Z0-9-]+)" }
          }
        }
      }
    }
  ]
}
```

What changed:

- **The loop is `paginate`.** It follows `a[rel=next]` for at most five pages, extracts on each,
  and concatenates the `products` lists.
- **Parsing is declared.** `type: "money"` with `locale: "tr"` reads `1.337,40 TL` as
  `{ "amount": 1337.4, "currency": "TRY" }`; a value that does not parse becomes a warning, not a
  crash. The `regex` keeps only the SKU code.
- **No browser is launched for this page.** `max_tier: "http"` keeps it on plain HTTP requests,
  which this shop allows.
- **The `expect` replaces `waitForLoadState`.** The step succeeds only when product cards are
  present.

Save it as a program and publish it:

### TypeScript

```ts
// Version 1, published: alias `latest` points at it.
const program = await kaab.programs.create({ name, definition, publish: true });

// Version 2 adds a field; publishing it moves `latest` to 2.
const v2 = await kaab.programs.versions.create(program.id, {
  definition: {
    ...definition,
    steps: [
      definition.steps[0],
      { extract: { ...definition.steps[1]?.extract, sku: { selector: ".sku" } } },
    ],
  },
  notes: "read the SKU too",
  publish: true,
});
console.log(`${program.name}: v${v2.version} is ${v2.status}`);
```

Then run it. The first item of `products` is `Kulaklık Klasik 101`, priced 1337.4 TRY, SKU
`HP-1037`. A one-off page that you have not saved yet runs with the shorthand:

### curl

```bash
response=$(curl -sS --fail-with-body "${KAAB_BASE_URL:-https://api.kaab.ist/v1}/jobs" \
  -H "Authorization: Bearer $KAAB_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait=30" \
  -d '{
    "url": "'"${SITE_URL:-https://shop.example.com}"'/p/1",
    "extract": {
      "title": "h1",
      "price": { "selector": ".price", "type": "money", "locale": "tr", "currency": "TRY" }
    }
  }')
echo "$response"
```

### TypeScript

```ts
// One call: create the job and hold the request up to 30 s for its result (`Prefer: wait`).
const job = await kaab.jobs.create({
  url: `${site}/p/1`,
  extract: {
    title: "h1",
    price: { selector: ".price", type: "money", locale: "tr", currency: "TRY" },
  },
  wait: 30,
});
if (job.status !== "succeeded") throw new Error(`job ${job.id} is ${job.status}`);
console.log(`${job.id}: ${JSON.stringify(job.data)}`);
```

### Python

```python
# One call: create the job and hold the request up to 30 s for its result (Prefer: wait).
job = kaab.jobs.create(
    url=f"{site}/p/1",
    extract={"title": "h1", "price": {"selector": ".price", "type": "money", "locale": "tr", "currency": "TRY"}},
    wait=30,
)
if job.status != "succeeded":
    raise SystemExit(f"job {job.id} is {job.status}")
print(f"{job.id}: {job.data.to_dict() if job.data else None}")
```

### Go

```go
// One call: create the job and hold the request up to 30 s for its result (Prefer: wait).
body, err := kaab.Body[client.CreateJobJSONBody](map[string]any{
	"url": site + "/p/1",
	"extract": map[string]any{
		"title": "h1",
		"price": map[string]any{"selector": ".price", "type": "money", "locale": "tr", "currency": "TRY"},
	},
})
if err != nil {
	log.Fatal(err)
}
job, err := c.Jobs.Create(ctx, body, kaab.WithWait(30))
if err != nil {
	log.Fatal(err)
}
if job.Status != "succeeded" {
	log.Fatalf("job %s is %s", job.ID, job.Status)
}
fmt.Printf("%s: %v\n", job.ID, *job.Data)
```

### CLI

```bash
data=$(kaab jobs run \
  --url "${SITE_URL:-https://shop.example.com}/p/1" \
  --extract '{"title": "h1", "price": {"selector": ".price", "type": "money", "locale": "tr", "currency": "TRY"}}' \
  --wait 30)
echo "$data"
```

## Pitfalls

- **Porting the sleeps.** `sleep` is capped at 10 seconds and lint warns about it. Replace each with
  an `expect` or `wait_for` on what you were waiting for.
- **Positional selectors.** `nth-child` chains and structural XPaths are the first thing to break,
  and lint warns when a target has nothing better. Prefer test ids, roles and text, and add
  candidates.
- **"First match" habits.** Playwright's strict mode is close, but code that used `$` or `.first()`
  silently picked one element. In a program, an ambiguous target fails; add `nth` only when the
  position is the meaning.
- **Moving logic into `evaluate`.** It works, but healing cannot see inside it and it needs a paid
  plan. Use extract types, transforms and conditions first.
- **Passwords as plain strings.** Declare them as `secret` inputs (see
  [security and data handling](https://docs.kaab.ist/guides/security-and-data)).
- **Assuming a browser.** Kaabist starts on HTTP and escalates when the page needs it. If your
  script relied on browser-only behaviour (clicks, typing), the program runs in a browser
  automatically; a read-only program may not.

## Reference

- [DSL reference](https://docs.kaab.ist/reference/dsl), [steps](https://docs.kaab.ist/reference/dsl/steps), [targets](https://docs.kaab.ist/reference/dsl/targets)
- [`paginate`](https://docs.kaab.ist/reference/dsl/actions/paginate) and [`extract`](https://docs.kaab.ist/reference/dsl/actions/extract)
- Field options: [`many`](https://docs.kaab.ist/reference/dsl/fields/many), [`fields`](https://docs.kaab.ist/reference/dsl/fields/fields),
  [`type`](https://docs.kaab.ist/reference/dsl/fields/type), [`regex`](https://docs.kaab.ist/reference/dsl/fields/regex),
  [`locale`](https://docs.kaab.ist/reference/dsl/fields/locale)
- [Create a program](https://docs.kaab.ist/reference/api/programs/create-program), [create a job](https://docs.kaab.ist/reference/api/jobs/create-job)