# bitti: the whole documentation > bitti runs plain-text web tests (.bt files, and Gherkin .feature files) on real Chrome through navigera, in-process: one native binary, no Node, no driver process, no model at run time; API tests run with no browser at all. Pages in site order; each is also at https://andrey-usa.github.io/bitti/.md. # bitti [![CI](https://github.com/andrey-usa/bitti/actions/workflows/ci.yml/badge.svg)](https://github.com/andrey-usa/bitti/actions/workflows/ci.yml) [![PyPI](https://img.shields.io/pypi/v/bitti.svg)](https://pypi.org/project/bitti/) [![npm](https://img.shields.io/npm/v/@getbitti/bitti.svg)](https://www.npmjs.com/package/@getbitti/bitti) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![Documentation](https://img.shields.io/badge/documentation-site-blue.svg)](https://andrey-usa.github.io/bitti/) Plain-text web tests that run on real Chrome. One native binary drives the browser in-process through [navigera](https://github.com/andrey-usa/navigera): no Node, no transpile step, no driver process, and no model at run time. ```text # tests/shop.bt base http://127.0.0.1:8000 before each goto /shop click --selector "#cookie-accept" scenario "Adding a product counts it in the cart" @smoke click --text "Add to cart" expect --selector "#cart-count" text is 1 goto /cart expect text "Your cart" ``` ```console $ bitti run tests --junit report.xml PASS tests/shop.bt:7 Adding a product counts it in the cart (557 ms) 1 passed, 0 failed in 0.9 s ``` API tests are `.bt` files too: a scenario of `http` steps and their checks runs with no browser at all, in milliseconds ([API tests without a browser](guides/api-tests.md)). ## Start - [Install](start/install.md): Every way to install bitti, and how to check it worked. - [Quick start](start/quick-start.md): One .bt file, one scenario, one run. - [Worked examples](start/tutorial.md): The example suites in the repository, as they are: a shop, a sign-in, data rows and shared flows, and a Kubernetes Job. ## Guides - [Test an API without a browser](guides/api-tests.md): Scenarios of http and exec steps that start no browser, and what each request costs. - [Run Gherkin feature files](guides/gherkin.md): Run .feature files as they are, beside .bt files, with their steps defined in .bt. - [Run bitti in GitHub Actions or a container](guides/ci.md): The GitHub Action, the container image, and a Kubernetes Job. ## Concepts - [Flows, sign-ins and hooks](concepts/flows.md): Steps shared between scenarios: flows, optional steps and conditions, signing in once per run, the hooks around scenarios, and handing a step to a person. ## Reference - [The .bt format](reference/format.md): Every top-level line, step and variable a .bt file can hold. - [Checks: expect](reference/checks.md): The expect table: every check a step can make, and how each comparison reads. - [Checks: soft, ax and screenshots](reference/checks-soft.md): `--soft` failures that fail a scenario at its end, matching the accessibility tree, and screenshot baselines and diffs. - [Running](reference/running.md): bitti run and its flags: which scenarios run, and in which browsers and apps. - [Reports and evidence](reference/reports.md): What a run prints and saves: console, jsonl, JUnit, Allure, the HTML report, video, traces, watch and shards. - [Failures and fixes](reference/failures.md): What a failure prints, fixing locators with bitti fix, and the tools that write scenarios: navigera, from-session, record, mcp. - [Config: bitti.toml](reference/config.md): What a run takes by default from bitti.toml, the app it starts first, and named environments. - [bitti --help](reference/help.md): The commands and each one's options, exactly as this build prints them. ## Agents - [For AI agents](agents/overview.md): What an agent gets: the skill, the MCP server with the browser and the test tools, and these docs as markdown. - [Skill](agents/skill.md): The guide `bitti skill` gives an agent, as this version prints it. ## Benchmarks - [Results](benchmarks/results.md): bitti against Playwright Test on Node and on Bun, on Playwright's own TodoMVC suite, from one CI run: time, CPU, and runner and browser memory. - [Where bitti is not ahead](benchmarks/losses.md): Each measure on which Playwright beat bitti in the same run. - [The 40-scenario suite](benchmarks/suite40.md): bitti's 40-scenario suite and its Playwright Test twin, on a shop and a gauntlet of awkward pages, which CI runs on three operating systems. ## Project - [Changelog](project/changelog.md): What changed in each release, and what is on the way. - [Contributing](project/contributing.md): How to build bitti against navigera, run its tests, and keep its performance gate green. - [Security](project/security.md): What bitti protects and how, and how to report a problem. Built from commit [c612c9e](https://github.com/andrey-usa/bitti/commit/c612c9e), 2 commits after v0.3.0, of 2026-11-13; `bitti --version` says `bitti 0.3.1`. These docs are ahead of the 0.3.0 release: what the [changelog](project/changelog.md#unreleased) lists under Unreleased is not in 0.3.0 yet. # Install Every way to install bitti, and how to check it worked. 0.3.1 ships the same prebuilt binary on every channel: | Channel | Command | |---|---| | pip / uv | `pip install bitti` · `uv tool install bitti` | | npm | `npm install -g @getbitti/bitti` | | Homebrew | `brew install andrey-usa/tap/bitti` | | Scoop | `scoop bucket add andrey-usa https://github.com/andrey-usa/scoop-bucket` then `scoop install bitti` | | GitHub release | download the archive for your platform from [v0.3.1](https://github.com/andrey-usa/bitti/releases/tag/v0.3.1) | Verify it: ```console $ bitti --version bitti 0.2.0 ``` --- Made from [README.md](https://github.com/andrey-usa/bitti/blob/next/README.md). # Quick start One .bt file, one scenario, one run. A scenario is a `.bt` file. Point `base` at your app, list the steps, and run the folder: ```text # tests/shop.bt base http://127.0.0.1:8000 scenario "Adding a product counts it in the cart" goto / click --text "Add to cart" expect --selector "#cart-count" text is 1 ``` ```console $ bitti run tests PASS tests/shop.bt:3 Adding a product counts it in the cart (119 ms) 1 passed, 0 failed in 219 ms ``` Every step, check and variable: [The .bt format](../reference/format.md); every command and flag: [Running](../reference/running.md). --- Made from [README.md](https://github.com/andrey-usa/bitti/blob/next/README.md). # Worked examples The example suites in the repository, as they are: a shop, a sign-in, data rows and shared flows, and a Kubernetes Job. The suites in the repository's `examples/bench-site/`, as they are: CI runs them against navigera's bench site, a small shop. ```console $ python3 ../navigera/bench/site/server.py --port 8000 & $ bitti run examples/bench-site --base-url http://127.0.0.1:8000 ``` ## shop.bt A cart, a search and a coupon field inside a web component. ```text {{#include ../../../examples/bench-site/shop.bt}} ``` ## account.bt Signing in, and the orders table behind it. ```text {{#include ../../../examples/bench-site/account.bt}} ``` ## data.bt Data rows, flows from another file, optional steps, page-state checks. ```text {{#include ../../../examples/bench-site/data.bt}} ``` ## flows/account.bt The flows `data.bt` takes with `use`. ```text {{#include ../../../examples/bench-site/flows/account.bt}} ``` ## A Kubernetes Job `examples/kubernetes/job.yaml`: the tests from a ConfigMap, JUnit and evidence in `/work/out`, a memory-backed `/dev/shm`. ```yaml {{#include ../../../examples/kubernetes/job.yaml}} ``` --- Made from [docs/examples.md](https://github.com/andrey-usa/bitti/blob/next/docs/examples.md). # Test an API without a browser Scenarios of http and exec steps that start no browser, and what each request costs. A scenario whose steps never touch a page (`http`, `exec`, their checks, checks of saved values and of cookies, `save … = json|header`, `print` of an answer, `retry`, `wait --ms`, a `flow` or an `auth` made of those) runs with no browser at all, and a run of only such scenarios starts none: nothing to install on the machine, and each request costs about a millisecond on a kept-alive connection. ```text base http://127.0.0.1:8000 auth api http POST /api/login --form user=ada --form "password=${secret:PW}" scenario "An order can be placed and ships" @api as api http POST /api/orders --json '{"sku": "sku-1", "qty": 2}' expect status 201 expect json $ schema order.schema.json expect time < 500ms save order = json $.id retry --timeout-ms 20000 http GET /api/orders/${order} expect json $ contains '{"status": "SHIPPED", "items": [{"sku": "sku-1"}]}' ``` ```console $ bitti run tests/api PASS tests/api/orders.bt:6 An order can be placed and ships (412 ms) 1 passed, 0 failed in 415 ms (no browser) ``` - Such a scenario keeps its cookies the way a browser would (domain, path, `Secure`, expiry; a cookie set on a redirect goes with the next hop): what its sign-in sets goes with its later requests, and `expect cookie sid` checks one. - An `auth` of `http` steps signs in with no browser too, and saves its cookies in navigera's `state save` format: a browser scenario `as` the same auth starts signed in, and an API scenario `as` an auth that signed in through the page takes its cookies. - A failure shows the request and what came back: ```text FAIL orders.bt:6 An order can be placed and ships (31 ms) line 10: expect json $ schema order.schema.json expected json $ to match schema order.schema.json; it doesn't: $.total: expected number, got string "36.00" last http: POST http://127.0.0.1:8000/api/orders -> 201 (9 ms, 61 B, application/json) {"id": "ORD-7", "status": "NEW", "total": "36.00", "items": [...]} ``` - `--trace` records each step with the requests it made, and `--html`, `--junit`, `--allure` report these scenarios as any other. - The schema check reads what an API's contract uses, in drafts 4 to 2020-12 alike: `type` (and OpenAPI 3.0's `nullable`), `enum`, `const`, `properties`, `required`, `additionalProperties`, `patternProperties`, `propertyNames`, `min/maxProperties`, `dependentRequired`, `items` (or a list of them), `prefixItems`, `contains`, `min/maxItems`, `uniqueItems`, `min/maxLength`, `pattern`, `format` (`date-time`, `date`, `time`, `email`, `uuid`, `uri`, `ipv4`, `ipv6`), `minimum`/`maximum` and their exclusive forms, `multipleOf`, `allOf`, `anyOf`, `oneOf`, `not`, `if`/`then`/`else`, and `$ref` within the file (`#/$defs/…`, `#/definitions/…`). Other keywords are ignored, as validators do. --- Made from [docs/api-tests.md](https://github.com/andrey-usa/bitti/blob/next/docs/api-tests.md). # Run Gherkin feature files Run .feature files as they are, beside .bt files, with their steps defined in .bt. `.feature` files run as they are, beside `.bt` files (no generated code): `Feature`, `Background` (before each scenario), `Scenario`/`Example`, `Scenario Outline` (or `Scenario Template`) with `Examples` (or `Scenarios`) tables, several allowed, each with its own tags (`` filled in; one scenario per row, named `Name (col=value, …)` when the name has no `<…>`), `Rule` (a `Background` under it runs before that rule's scenarios only), tags (a feature's go on all its scenarios), and `Given`/`When`/`Then`/`And`/`But`/`*` steps, all alike. English keywords only. A step is one of the run's own `step` blocks, from any `.bt` file it loads: ```text step "I sign in with {password}" fill --label Password ${password} click --text "Sign in" ``` `{name}` takes a `"…"` or `'…'` value or one word and is `${name}` in the steps under it. A line two `step` blocks match is an error. Else it is a phrase of the built-in book: `I open "/cart"` (`go to`, `navigate to`, `visit`, `am on`), `I click "Apply"` (`click on`; by visible text), `I fill "Email" with "a@b.c"` (`fill in`, `enter`; by label), `I type "x" into "Search"`, `I select "Canada" from "Country"` (or `in`), `I check "Terms"`, `I uncheck …`, `I press "Enter"`, `I wait for "Saved"`, `I see "Saved"` (`I should see`), `I don't see "Error"` (`should not`, `shouldn't`, `do not`), `the URL is|contains|starts with|ends with|matches "…"` (`url`, `page URL`), `the title is|contains … "…"` (`page title`). An icon button or a field with no label needs a `step` block (`click --role …`, `--placeholder`). A step nothing matches is a file error that says how to write it. Data tables and doc strings under a step aren't supported yet. `--cucumber ` writes Cucumber JSON for the `.feature` files, step by step (passed, failed, skipped, with times), for test-management importers. `--junit`, `--html`, `--allure` and the console report them like any scenario. --- Made from [docs/gherkin.md](https://github.com/andrey-usa/bitti/blob/next/docs/gherkin.md). # Run bitti in GitHub Actions or a container The GitHub Action, the container image, and a Kubernetes Job. ## GitHub Actions ```yaml - uses: andrey-usa/bitti@next with: paths: tests base-url: http://127.0.0.1:3000 args: --workers auto --retries 1 --junit report.xml ``` It runs on the runner's Chrome (Linux, macOS, Windows). `version` picks a release (`latest` by default, or a tag). With no release binary for the runner, it builds bitti from the action's own commit (`source` asks for that), with navigera at `navigera-ref`. Failures are annotations on the lines that failed. The `evidence` output is the directory holding their screenshots, for `actions/upload-artifact`. ## Container `Dockerfile` builds an image with bitti and Debian's headless Chromium, amd64 or arm64, from the directory holding both checkouts (see Building): ```console $ docker build -f bitti/Dockerfile -t bitti . $ docker run --rm --network host -v "$PWD/tests:/tests:ro" bitti run /tests --base-url http://127.0.0.1:3000 ``` It runs as an unprivileged user (any uid, so OpenShift's random ones too), with a read-only root filesystem if asked (`HOME=/tmp`, a `/tmp` to write to, and a writable working directory: `.bitti/` with the last run, timings and evidence goes there, `/work` by default), and without Chrome's own sandbox: a container without user namespaces can't give Chrome one, and the container is the boundary. `examples/kubernetes/job.yaml` is a Job: the tests from a ConfigMap, JUnit and evidence in `/work/out`, a memory-backed `/dev/shm`. --- Made from [docs/guides/ci.md](https://github.com/andrey-usa/bitti/blob/next/docs/guides/ci.md). # Flows, sign-ins and hooks Steps shared between scenarios: flows, optional steps and conditions, signing in once per run, the hooks around scenarios, and handing a step to a person. - `flow name(a, b)` at the top level holds steps; `run name x y` runs them with `${a}` and `${b}` set. `use other.bt` takes another file's flows and `set`s. A failure inside a flow names the flow's own line. - `optional `: a step that may not apply (a cookie banner that only sometimes shows); its failure is noted, not fatal. - `if ` with steps indented under it runs them when the clause holds now (or within its `--timeout-ms`): `if --selector "#banner" visible`, then `click --text Accept` indented below. - `auth shopper [ttl 30m]` (`s`, `m`, `h`, `d`, or plain seconds) holds steps that sign in; `as shopper` in a scenario starts it signed in. The block runs once per run, in a browser context of its own, the first time a scenario needs it (the other workers wait for it), and its cookies and storage are saved under `.bitti/auth/` (keep that out of git). With a `ttl`, the saved state is reused by later runs until it is that old. - `before all` runs once for the file, before its first scenario, and every scenario of the file sees what it saves. If it fails, none of them run. `after all` runs once after the last one (a failure there is reported as `(after all)`). - `handoff until `: the steps only a person can do (2FA, a CAPTCHA). When the clause already holds, nothing happens; else the browser comes on screen, the run waits for the clause (5 minutes, or its `--timeout-ms`) or Enter, and the browser goes back off screen. Plain `handoff` waits for Enter. With no screen it fails at once and says so. --- Made from [docs/format.md](https://github.com/andrey-usa/bitti/blob/next/docs/format.md). # The .bt format Every top-level line, step and variable a .bt file can hold. Checks have their own pages: [Checks: expect](checks.md), [Checks: soft, ax and screenshots](checks-soft.md). ## Files, steps and variables - Top-level lines: `base `, `set NAME = value`, `use other.bt`, `flow name(a, b)`, `step ""` (a Gherkin step's steps, see below), `auth name [ttl 30m]`, `before all|each`, `after each|all`, `scenario "name" @tag…`. Steps are indented under them. - A step is a navigera command without the `navigera -s ` prefix (`goto /cart`, `click --text Add`): `goto`, `click`, `fill`, `select`, `press`, `check`, `hover`, `scroll`, `upload`, `wait`, `eval`, `set`, `route`, `state load`, and the rest. Targets use navigera's locators (`--role` with `--name`, `--label`, `--text`, `--placeholder`, `--testid`, `--selector`, `--near ` for the one in the card or row that shows the text, `--nth`). Refs from `ax` (`click 12`) are an error: they change between runs. `check --label Terms` ticks a box; `check ""` with no locator asks navigera a yes/no question about the page (a model call). - `set` steps are navigera's: `set viewport 390 844`, `set device "iPhone 16 Pro"` (55 devices), `set geo 52.52 13.405`, `set locale de-DE`, and `set permissions grant|deny|reset geolocation notifications camera microphone clipboard` (also `midi`, `sensors`, `background-sync`), which a page then sees as granted, denied or still to be asked. (A `set NAME = value` at the top of a file is a variable.) - A relative path a step reads (`upload`, `state load`, `eval --file`, `expect js --file`, `init-script --file`, `route --body-file`, `route --har`) is found next to the `.bt` file, as `use` and `for each` files are, so a suite runs the same from any directory. Files a step writes (`screenshot`, `state save`) land in the current directory. - Network: `route "*/api/*" --body '{…}'` answers in the server's place, `route --har api.har` replays a recorded HAR (navigera's `network --har` writes one; requests it has no entry for fail, or reach the network with `--not-found continue`), and `route "*/api/me" --modify --replace free pro` changes the real response (`--set-header 'Name: value'` too). `tests/fixtures/har.bt` has both. - Words split like a shell's, with no expansion; inside double quotes `\"` and `\\` stand for themselves and `\n`, `\r` for a line break (a textarea's value). `${NAME}` and `${NAME:-default}` come from `save`, a flow's arguments, the data row, `before all`, `--var`, `set`, then the environment; `$$` (or `\$`) is a literal `$`, and quotes don't stop `${…}` (`'${x}'` is filled in too). Also `${run.id}` (one per run: unique emails), `${scenario.name}`, and `${download.path}`/`${download.name}` after `expect download`. - `${secret:NAME}`: from the environment, else `--secrets ` (`NAME=value` lines; the environment wins). Its value is masked as `***` in everything the run prints and saves (messages, the page's URL and title, saved values, JUnit and JSON lines). - `${totp:NAME}`: the code an authenticator app shows right now, for a sign-in's second step (`fill --label "Authentication code" ${totp:SHOP_TOTP}`). NAME is found as a secret's is and holds the authenticator's base32 key or the `otpauth://` URI of the site's QR code; the key is masked like a secret. A code with under 3 s left waits for the next one, so it can't run out before the site checks it. - A default with spaces is one word: quote it whole, `"${GREETING:-hello there}"` (words split before `${…}` is filled in). - A `set` value can hold `${…}` too (a default, a `--var`, another `set`, a secret, the environment): it is filled in once, when a scenario starts. - `save NAME = ` keeps a command's result (`get text …`). - `print ` shows a read command's result in the run's output (`print get text --testid total`, `print eval "…"`, `print ax --selector main`): no check and no wait, for a look at the page or a value while a test is being written. `print http` shows the last `http` answer (its status, headers and body), `print body`, `print json <$.path>` and `print header ` a part of it (JSON indented), and `print "order ${id}"` a line with its variables filled in. - `exec [--timeout-ms ] ` runs a program (no shell: `sh -c "…"` or `pwsh -c "…"` for one) in the `.bt` file's directory, with `BITTI_BASE_URL` set, for what a browser can't reach: seed a database (`psql`, `sqlplus`), call a script. Its exit code and output become `${exec.exit}`, `${exec.stdout}` and `${exec.stderr}`, and `expect exit 0`, `expect stdout contains SHIPPED`, `expect stderr …` check them (the first 16 MB of each). A nonzero exit fails the step, with the last line of its stderr, unless a step indented under it checks `exit`. It is stopped after 60 s (`--timeout-ms`), and that fails the step whatever is under it. ```text exec ./scripts/order-status.sh ${order} expect exit 0 expect stdout contains SHIPPED ``` ## HTTP requests - `http [--json | --body | --json-file | --body-file | --form name=value… | --multipart name=value|name=@file…] [--query name=value]… [--user name:password | --bearer ] [--header "Name: value"]… [--no-follow] [--insecure] [--timeout-ms ]` makes a request outside the page (an API to set up or check what the UI did, or an API test of its own: see [API tests](../guides/api-tests.md); a relative URL takes the base URL). `--form` sends an urlencoded form, `--multipart` a `multipart/form-data` one (`@file` uploads the file beside the `.bt`, `;type=image/png` after it sets its type), `--query` adds an encoded `name=value` to the URL, and `--user ada:${secret:PW}` or `--bearer ${secret:TOKEN}` sets the `Authorization` header (Basic or Bearer). The scenario's cookies go with it and the cookies it sets come back to the browser, so an API sign-in signs the page in. Its answer is `${http.status}`, `${http.body}` and `${http.ms}` (how long it took); `expect status 201`, `expect json $.role is shopper`, `expect header Location ends-with /u/1`, `expect body contains …`, `expect time < 300ms` check it, and `save id = json $.id` (or `header `) keeps a value. A 4xx or 5xx fails the step, with the start of the body, unless a step indented under it checks `status`. A timeout fails it whatever is under it. Redirects are followed (up to 20, a cookie set on the way going to the next hop and to the browser). `--no-follow` gives the 3xx itself. `--insecure` accepts a certificate nothing vouches for (a local server's own). The client is built in (rustls, the operating system's certificates, the proxy from `https_proxy`/`http_proxy`/`no_proxy`): no process per request, and connections are kept alive between requests, so a call to a local server costs about a millisecond and a secret never reaches a command line. `--graphql ` (or `--graphql-file `) with `--variables ` and `--operation ` sends a GraphQL request as its JSON body. An answer with `errors` (GraphQL sends them with a 200) fails the step, unless a step under it checks `json $.errors`. ```text http POST /api/users --json '{"email": "${email}", "role": "shopper"}' expect status 201 expect json $.role is shopper save user = json $.id ``` ## Retries, pauses and the clock - `retry [--timeout-ms ]` with steps indented under it runs them again until all of them pass, a little later each time (0.1, 0.25, 0.5, then every 1 s), for what happens in the background: a job that finishes, an order that ships, a mail that is sent. Each `expect` in it looks once (unless it says `--timeout-ms`): the block is the wait. It gives up after its `--timeout-ms` (the run's expect timeout, 5 s, by default) with the last try's failure. ```text retry --timeout-ms 20000 http GET /api/orders/${order} expect json $.status is SHIPPED ``` - `pause` stops the run there, the browser as the steps before left it, open to a second navigera session (the command to attach is printed), until Enter. With no terminal it waits `BITTI_PAUSE_S` seconds, and under CI not at all unless that is set. A run with a `pause` in it runs one worker. - `clock install [--time 2024-02-02T08:00:00] [--paused]` puts the page's time under the scenario's control (navigera's `clock`): `Date`, `performance.now` and timers start there, in every page the scenario opens; `clock run-for 15m` (or `mm:ss`) advances it, firing the timers due on the way in order; `clock fast-forward`, `clock set