# Side-effect safety

Mark the steps that change something, checkpoint after them, and send an idempotency key, so an order is never placed twice.

A side effect is a step the world remembers: an order placed, a message sent, a form submitted.
Kaabist cannot undo those, so it guards each one with a fence, refuses to repeat it, and parks the
job for a person whenever it cannot tell whether the step happened.

## When to use it

Mark a step `side_effect: true` whenever running it twice would do harm: a submit that creates
something, a button that sends, pays, books or deletes. Pair it with an `Idempotency-Key` on the
request that creates the job whenever your own code may retry that request.

A program that only reads pages (`goto` and `extract`) needs none of this. Do not mark a step as a
side effect "to be safe": a fenced step cannot be retried, cannot be optional, and turns an
interruption into a parked job that a person has to resolve.

## What `side_effect: true` changes

### The fence

Before a side-effect step acts, Kaabist records that the step is *about to execute*. After the
step's `expect` holds, it records that the step *executed*. The next `checkpoint` closes the fence.

If the run is interrupted between those two records (the worker restarts, the browser crashes, the
connection drops while the site is still answering), nobody can know whether the action went
through. Kaabist never guesses: instead of running the step again, it parks the job in
`needs_attention` with the reason `side_effect_ambiguous`, and a person decides.

While a fence is open, the job cannot be canceled: `POST /v1/jobs/{id}/cancel` answers
`409 fence_open`.

### No retries, no optional

A side-effect step may not carry `retry` or `optional: true`. Both would let the engine run the
action again, or carry on as if it had not happened, so lint refuses them with
[E011](https://docs.kaab.ist/reference/dsl/lint/e011) and the program cannot be published. If the submit is flaky,
make the step's `expect` precise and let the job park instead.

### A checkpoint after every side effect

End every side effect with a `checkpoint` before the next one. Without it, lint warns with
[W004](https://docs.kaab.ist/reference/dsl/lint/w004): the job has no safe place to restart from after the action, and a
later problem could only be resolved by rewinding past it, which Kaabist refuses.

### Resuming a parked job

A parked job restarts from a checkpoint (see [human-in-the-loop](https://docs.kaab.ist/concepts/human-in-the-loop)), and
the fences decide where it may restart:

- **`retry_from`** may name only a checkpoint after the last side effect that executed. A checkpoint
  at or before it would place the order again, so the request is refused with `409 fence_open`.
- **An open fence** (the step was dispatched but never confirmed) blocks every restart at or before
  that step: a resume that would start there parks the job `side_effect_ambiguous` again instead of
  touching the site.
- **`skip_step`** on a `side_effect_ambiguous` park restarts *after* the fenced step: you are telling
  Kaabist that the action went through.
- **`abort`** cancels the job.

A run that replays from an earlier checkpoint, for example because the checkpoint's page no longer
opens, never replays across an executed side effect. The data extracted before the park stays in
the result.

To resolve `side_effect_ambiguous`, check the site first. If the action went through, `skip_step`
tells Kaabist so and the job carries on after the fenced step. If it did not, `abort` the job and
create a new one, with a new idempotency key: within 24 hours the old key returns the aborted job.

### Dry runs

`dry_run: true` on job creation runs the program until the first side-effect step and stops there.
The job ends `succeeded` with `meta.dry_run = true` and nothing changed on the site. The program builder's
**Dry run** does the same.

## Idempotency keys

A fence protects the steps inside a job. An idempotency key protects the request that creates the
job: a client that times out and tries again must not start a second job that places a second
order.

Send `Idempotency-Key: <string of at most 255 characters>` on `POST /v1/jobs` or
`POST /v1/jobs/batch`. For 24 hours, per organisation:

- the same key with the same body returns the original response, marked
  `Idempotent-Replayed: true`, and queues nothing;
- the same key with a different body is refused with `409 idempotency_conflict`.

Use one key per thing you mean to do (one order, one message) and store it with that thing on your
side, so a retry after a crash or a deploy sends the same key. The SDKs send a fresh key on every
`jobs.create` and `jobs.batch` when you give none, which makes their own automatic retries safe;
pass your own key to make *your* retries safe too.

## Worked example

Take an order form at `https://checkout.example.com` that places orders through a `POST /orders`
that is not idempotent: every submission is a new order. The program fills the order form, fences the submit, checkpoints
right after it, and reads the confirmation page:

### TypeScript

```ts
const definition = {
  kaab: "1",
  name,
  inputs: { sku: "string", qty: { type: "number", default: 1, min: 1, max: 40 } },
  options: { max_tier: "browser" },
  steps: [
    { goto: `${site}/orders/new`, expect: { visible: "#order-form" } },
    { fill: "input[name=sku]", value: "{{inputs.sku}}" },
    { fill: "input[name=qty]", value: "{{inputs.qty}}" },
    {
      // Never retried or replayed blindly: if the run is cut here, the job parks instead.
      click: "#order-form button[type=submit]",
      side_effect: true,
      expect: { url: "**/orders/*" },
    },
    // Anything after this resumes from here, not from the start: the order is not placed again.
    { checkpoint: "order-placed" },
    {
      extract: { order: "h1", sku: "#order-sku", qty: { selector: "#order-qty", type: "integer" } },
    },
  ],
};
await kaab.programs.create({ name, definition, publish: true });
```

The click on the submit button carries `side_effect: true` and an `expect` on the order URL, so the
fence closes only once the confirmation page has loaded. The `order-placed` checkpoint is where any
later restart begins: a problem while reading the confirmation never places the order again.

Creating the job with an idempotency key makes the request itself safe to repeat:

### TypeScript

```ts
// One key per order you mean to place; store it with the order on your side.
const idempotencyKey = `order-${name}`;
const request = { program: name, inputs: { sku: "HP-1037", qty: 2 } };

const first = await kaab.jobs.create({ ...request, idempotencyKey });
// A retry (after a timeout, a crash, a deploy) sends the same key and the same body…
const again = await kaab.jobs.create({ ...request, idempotencyKey });
// …and gets the same job back: nothing new is queued, no second order.
console.log(`${first.id} === ${again.id}: ${first.id === again.id}`);

const done = await kaab.jobs.wait(first.id, { timeoutMs: 120_000 });
console.log(`${done.id}: ${done.status} ${JSON.stringify(done.data)}`);
```

The second `jobs.create` sends the same key and the same body, gets the first job back, and the
site receives one order, not two. A `dry_run` of the same program fills the form and stops before
the submit, and the site receives no order at all.

## Pitfalls

- **An `expect` that is too loose.** The fence closes when the step's `expect` holds. An `expect`
  that holds before the site has answered (a spinner, the same URL) closes it too early; one that
  never holds parks the job even though the order went through. Wait for the page or element that
  only exists after success.
- **Several side effects without checkpoints between them.** Lint warns (W004), and an interruption
  after the second one leaves no restart point between the two.
- **A new idempotency key per retry.** A key generated inside the retry loop protects nothing. Make
  it once, from your own record of the order.
- **Reusing a key for different work.** Within 24 hours the same key with a different body is
  `409 idempotency_conflict`, not a new job.
- **Expecting cancel to work at any moment.** Inside an open fence it is refused with `409
  fence_open`; cancel again once the step has finished.

## Reference

- [Steps: common keys](https://docs.kaab.ist/reference/dsl/steps) and the [`checkpoint` action](https://docs.kaab.ist/reference/dsl/actions/checkpoint)
- Lint [E011](https://docs.kaab.ist/reference/dsl/lint/e011) and [W004](https://docs.kaab.ist/reference/dsl/lint/w004)
- [Create a job](https://docs.kaab.ist/reference/api/jobs/create-job), [resume a job](https://docs.kaab.ist/reference/api/jobs/resume-job),
  [cancel a job](https://docs.kaab.ist/reference/api/jobs/cancel-job)
- [Attention reasons](https://docs.kaab.ist/reference/codes#attention-reasons) and [error codes](https://docs.kaab.ist/reference/codes#errors)