> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sedum.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# TypeScript tests

A `*.test.ts` file holds one or more tests. Each test is an async function:
plain-English steps run through `ai`, and anything else is ordinary
TypeScript: Playwright calls on `page`, `expect` assertions, API requests,
test data from any library, loops, and helper functions.

```ts theme={null}
// tests/checkout.test.ts
import { faker } from "@faker-js/faker";
import { test, expect, secret } from "sedum-cli";

test(
  "a new customer checks out",
  { url: "/", tags: ["checkout"] },
  async ({ page, ai, env }) => {
    await ai.group(
      "Log in",
      [
        "type {{user}} in the username field",
        "type {{password}} in the password field",
        "click the login button",
      ],
      { user: "standard_user", password: secret(env.SAUCE_PASSWORD!) },
    );

    await ai("click the Add to cart button for {{product}}", {
      product: "Sauce Labs Backpack",
    });
    await expect(page.locator(".shopping_cart_badge")).toHaveText("1");

    await ai("click the shopping cart link");
    await ai("click the Checkout button");
    await ai("type {{first}} in the First Name field", {
      first: faker.person.firstName(),
    });
    await ai("type {{last}} in the Last Name field", {
      last: faker.person.lastName(),
    });
    await ai("type {{zip}} in the Zip/Postal Code field", {
      zip: faker.location.zipCode(),
    });
    await ai("click the Continue button");

    const total = await ai.extract("the order total");
    expect(total).toContain("$");
    await ai("click the Finish button");
    await ai("verify the order was placed and a confirmation message is shown");
  },
);
```

```sh theme={null}
npx sedum run                                  # every test under tests/
npx sedum run tests/checkout.test.ts           # every test in one file
npx sedum run --id "tests/checkout.test.ts#a new customer checks out"   # one test
```

Install `sedum-cli` in the project (`npm install -D sedum-cli`) so your editor
has its types. Sedum compiles the file itself with
[jiti](https://github.com/unjs/jiti); you need no build step or `tsconfig.json`
to run tests. Types are not checked at run time; run `tsc --noEmit` for that.

## Declaring tests

```ts theme={null}
test(title, body);
test(title, options, body);
```

| Option | Meaning |
| - | - |
| `url` | Entry URL, absolute or relative to `baseUrl`. Omitted: `baseUrl`, or a blank page without one. |
| `id` | Stable identity. Default: `<file>#<title>`, such as `tests/checkout.test.ts#a new customer checks out`. |
| `tags` | Labels for `sedum run --labels`. |

Titles must be unique within a file and ids unique in the project. Each test
gets a fresh browser context and runs on its own, in parallel lanes and with
`--retries` like a YAML test unless it enters a goal (see below). Top-level code
in the file runs once when Sedum imports it, so keep per-test setup inside the
body.

The body receives:

| Name | What it is |
| - | - |
| `ai` | Runs plain-English steps; see below. |
| `page` | The Playwright [`Page`](https://playwright.dev/docs/api/class-page) the steps run on. |
| `context` | Its [`BrowserContext`](https://playwright.dev/docs/api/class-browsercontext), for cookies, storage, and routes. |
| `env` | Configured `variables`, the project `.env`, and the process environment, as `sedum run` resolves them. |
| `testInfo` | `{ id, title, file, tags, attempt }`. |

`expect` is Playwright's, with its auto-retrying matchers such as
`toHaveText` and `toHaveURL`, and the usual `toBe` and `toEqual`.

## Steps with `ai`

```ts theme={null}
await ai("click the login button");
await ai("type {{email}} into the Email field", { email });
await ai(["click the Cart link", "click the Checkout button"]);
```

A sentence is one step, written exactly as in a YAML test and classified and
resolved the same way: `click`, `type`, `press`, `goto`, `scroll`, `wait`,
`verify`, `measure`, and `remember` (see
[step classification](classification.md)). A list runs its sentences in order,
sharing one values object.

### Goals and separate verification

`ai.goal(goal, values?, options?)` asks the planner to complete a bounded task on the
current page:

```ts theme={null}
await ai.goal("Create a new account and complete its profile");
await ai('verify the page displays the confirmation message "Profile saved"');
await expect(page.getByRole("status")).toHaveText("Profile saved");
```

The promise resolves when the planner reports `DONE` and that decision passes
the normal confidence and freshness checks. This means **planner-reported
completion, not independent verification**: `ai.goal` does not call the Judge
or add a hidden assertion. A goal-only test can pass and is reported as
completed, not verified. Add a separate `ai("verify ...")` step for a semantic
check, a Playwright `expect` for an exact check, or both. A later check proves
only what it asserts; it does not retroactively verify every part of the goal.
There is no `ai.act`, `ai.assert`, or `ai.verify` API.

Goals, authored `ai` steps, Playwright calls, and assertions can be interleaved
on the same page. Each goal starts with fresh planning history, counters,
references, and generated-value memory, while observing the page left by the
preceding code:

```ts theme={null}
const customer = {
  email: faker.internet.exampleEmail(),
  firstName: faker.person.firstName(),
};

await ai.goal("Create an account for {{email}}", customer);
await expect(page).toHaveURL(/\/profile/);

await page.getByLabel("Marketing emails").uncheck();
await ai.goal(
  "Complete the profile for {{email}} using {{firstName}}",
  customer,
);
await ai('verify the page displays the confirmation message "Profile saved"');
await expect(page.getByRole("status")).toHaveText("Profile saved");
```

The optional values argument follows the same rules as `ai(sentence, values)`.
Every explicit `{{placeholder}}` must have a supplied binding; a missing one is
an authoring error. With the TypeSafe provider, a goal may also choose a local,
allowlisted Faker generator for a fill that has no applicable binding. Faker
runs locally for each invocation. Automatically generated values can be reused
within that goal (for example, in a confirmation field), but are not shared
with another goal. To reuse an identity, generate it in TypeScript as above and
pass it explicitly to every goal; wrap sensitive values with `secret()`.

Automatic generation defaults to enabled. Set `generateData: false` in the
third argument to restrict fills to supplied or remembered values:

```ts theme={null}
await ai.goal("Complete the profile for {{email}}", customer, {
  generateData: false,
});
// With no values, allow goals that do not need generated input.
await ai.goal("Open billing settings", undefined, { generateData: false });
```

This setting applies only to that invocation. It disables automatic Faker
generation, not Faker calls you make in TypeScript. Missing data cannot be
generated; a blocked goal rejects. Completion still depends on the planner,
so keep independent checks for the outcome you need.

A goal uses the unchanged defaults of 24 requests, 18 dispatched actions, and
120 seconds. Value-selection requests count toward the same request budget.
`BLOCKED`, abstention, no progress, budget exhaustion, cancellation, and action
errors reject the goal. Once a valid goal enters planning, Sedum does not
automatically retry the whole test, even if the goal succeeds and a later
`expect`, verify step, or cleanup fails. This prevents replaying side effects;
manual reruns are still possible. Tests that never enter a goal retain normal
`--retries` behavior.

Goal actions are limited to the operations offered by goal mode (currently
click and type); use deterministic Playwright or authored `ai` steps for other
work. Goals are still exclusive, tracked `ai` operations: always `await` them,
and do not overlap them with another `ai` call. Multiple goals are sequential,
not parallel.

**Pass values separately, not with `${}`.** Write `{{name}}` in the sentence
and give the value in the second argument. The sentence text stays the same on
every run, so its classification is cached and reused, and `sedum validate` can
check it before a run. A sentence built with `${}` or `+` is classified again
for every distinct value, and `validate` warns about it.

Values can be strings, numbers, and booleans. Before Sedum locates a target or
judges a claim, it puts plain values into the sentence, so
`click the Add to cart button for {{product}}` is resolved as the named
product. The value of a `type` step is typed, never shown to the model.

**Secrets.** Wrap a sensitive value in `secret()`. It is typed into the page,
but it stays a `{{placeholder}}` in model requests and is redacted from report
text and `result.json`. Printing a secret shows `[secret]`. Sedum learns a
secret when a step first passes it, so page text judged by an earlier step can
still contain it; screenshots are never redacted. List origins that show
secrets with `--sensitive-origin`.

```ts theme={null}
await ai("type {{password}} into the Password field", {
  password: secret(env.SHOP_PASSWORD!),
});
```

`remember <target> as {{name}}` stores page text for later steps in the same
test, as in YAML.

**Always `await`.** Steps run one at a time on one page. A step started while
another is still running stops the test with an error that names the line.

## Groups

`ai.group` names a block of steps. Every step inside it is reported under the
name, such as `Checkout › click the Continue button`, and a failure names the
group it happened in. Groups can nest.

```ts theme={null}
await ai.group(
  "Log in",
  ["type {{user}} in the username field", "click the login button"],
  { user },
);

await ai.group("Checkout", async () => {
  await ai("click the Checkout button");
  await page.getByLabel("First Name").fill(first); // code runs inside a group too
  await ai("click the Continue button");
});
```

## Reading from the page

`ai.holds(claim, values?)` asks whether a claim holds on the page now, so a
test can branch when a screen appears only sometimes. It returns `true` or
`false`, never fails the test just because the claim is false, and is recorded
without a pass/fail verdict. Unlike a verify step, it does not use the
configured verify grace period.

```ts theme={null}
if (await ai.holds("the passkey enrollment screen is shown")) {
  await ai("click the Skip button");
}
```

Pass a claim, not an action or a wait instruction. Claims are classified and
judged through the same deterministic and model-backed paths as verify steps at
run time. Offline validation accepts readable single-clause claim forms such as
`the dialog is shown`, `the prompt appears`, or `the total matches {{total}}`
without classification. It rejects malformed placeholders, actions, waits,
ambiguous sentence shapes, and call shapes it cannot inspect.

`ai.extract(description)` returns the text of the element the description
names, found the same way as a `remember` target. Pass a parser, such as a zod
schema, to convert it:

```ts theme={null}
const text = await ai.extract("the order total"); // "Total: $32.39"
const total = await ai.extract("the order total", {
  parse: (value) => Number(String(value).replace(/[^0-9.]/g, "")),
});
```

Extracted text is treated as page data: it is redacted from reports, like a
remembered value.

## Code between steps

Anything Playwright or Node can do works between steps. Use it where a
sentence is the wrong tool:

* **Skip slow setup.** Sign in through an API, or set a session cookie with
  `context.addCookies`, then `page.goto` the page under test.
* **Seed and inspect state.** Call your backend, or read `localStorage` with
  `page.evaluate`, and check it with `expect`.
* **Exact checks.** A count, a URL, or an element that must be absent is often
  clearer as a locator assertion than as a claim, for example
  `await expect(page.locator(".cart_item")).toHaveCount(0)`.
* **Generate data.** Use any library, such as `@faker-js/faker`, and pass the
  values to `ai`.
* **Share steps.** A module is a function: `export async function login(ai, user) { ... }`.
  A helper can also wrap `test()` itself, for example to add shared tags; the
  test belongs to the file that calls the helper.

The page settles before each `ai` step, so a step after code sees the page
that code left.

## Results

* **A failed step stops the test.** A failed `verify`, or a target that cannot
  be found, is recorded as a failed step, and the rest of the body does not
  run. The test fails even if your code catches the error.
* **An exception from your code fails the test.** A failed `expect` or any
  thrown error is recorded as a failed step with `operation: "code"`, at the
  file and line that threw, under its group. This includes a promise the test
  never awaits, such as an unawaited `page.click` or `expect`, and an
  `ai.group` or `ai.extract` whose code throws while nobody awaits it. To let a
  block fail on purpose, handle it: `await ai.group(...).catch(() => {})`.
* **A promise left behind fails the run, not another test.** Sedum blames a
  test for an unhandled rejection only when the error's stack points inside
  that test's lines while it runs. A promise that rejects after its test
  finished, in a helper any test could have called, in lines that several
  tests declared in a loop share, or in a file's top-level code, is reported
  for the whole run with its file and line, and the command exits 3. If it
  rejects after the report is written, Sedum prints it and still exits 3.
* **Misusing `ai` is an invalid test.** A missing value, an invalid values
  object, or a missing `await` stops the run as an invalid test, like a YAML
  file with an error, with the file, line, and a fix.

Supplied secret and generated goal values join the attempt-wide redaction
history, so later goals, verification, diagnostics, reports, and cleanup do not
echo known values. They do not become fill choices for later goals. Redaction
cannot mask screenshots, video, arbitrary application logging, or values Sedum
never learned; use disposable accounts and the existing evidence controls for
sensitive pages.

Reports name a TypeScript test by its file and title, and rerun commands add
`--id` to select just that test.

## Validation and listing

`sedum list` imports each `*.test.ts` file and lists one row per `test()`.
`sedum validate` also finds the sentences passed to `ai(...)`, `ai([...])`,
`ai.goal(...)`, and `ai.group(name, [...])`, in the test file and in the local modules it
imports, and checks them offline, or online with `--online`, exactly like YAML
steps. A list or sentence held in a `const` in the same file is read too, when
the file declares that name once and never reassigns it.
Validation recognizes the `ai` name.

For `ai.goal`, validation checks literal and supported `const`/local-helper
shapes, placeholder syntax, and statically readable argument shapes. Required
bindings and planner availability are checked at run time. Validation
does not execute the body, generate Faker data, ask a model whether the goal is
feasible, or classify the goal as one authored step. Dynamic expressions and
aliases it cannot resolve produce the same honest validation warnings as other
`ai` calls; runtime validation remains authoritative.

An argument validation cannot read, such as a variable, a function call, or a
sentence built with `${}`, is reported as a warning. So is any way of running
steps that validation cannot follow: `ai` aliased or renamed, `ai` read off the
context (`ctx.ai`, `t["ai"]`), or `ai`, `{ ai }`, or the test's context handed
to a helper outside the project or to a helper name the file declares more
than once. Pass `ai` to a helper in the test file or a local module as a
parameter named `ai`, such as `login(ai, user)`; validation follows the call
and checks the helper's steps. A list passed to `ai.group` must be a literal, a
`const` in the same file, or a function body. When it warns, `validate` does
not call the project fully valid and exits 1, as for a YAML sentence it could
not check offline.

Importing a file runs its top-level code. Keep network calls and other side
effects inside test bodies.

## Limits

* `validate` reads your code without running it. It vouches for the
  sentences it can see in the shapes above and warns about the rest; a
  sentence it cannot see is still checked when its step runs, and an invalid
  one stops that test with the file and line.
* There are no `beforeEach` or `afterEach` hooks yet; use a helper function
  and `try`/`finally` for cleanup.
* `import ... from "sedum-cli"` always resolves to the CLI that runs the test,
  whichever version is installed in the project.
* A `*.test.ts` file cannot use YAML modules, and a YAML test cannot call
  TypeScript.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.