Command runners
Run your repository's own commands — tests, lint, build — on a machine you own, so agents can ask for them by name. The application server does not execute repository code.
A command runner lets a factory request tests, lint, builds and page checks on a machine your team operates. The runner connects out to CodeBook. Register approved commands for controlled execution, or enable autonomous execution on both the factory and runner when your team wants agents to propose their own shell lines.
An operator can separately enable a hosted proof service backed by containers or Modal. The incident-check runner is a different download used in your repository’s CI. Choose this command runner when you want a machine you control to execute work. Incident-check CI setup
How it works
- A person registers a runner for a factory and lists the commands it may run: a short name and a line, such as
test→npm test. - They start the runner program on their own laptop, workstation or server. It connects out to CodeBook; nothing connects in and no port is opened.
- An agent — working a card, or asked in a conversation — requests a command by name on a pull request.
- The runner checks out that exact commit with the machine's own git access, runs the line, and reports the exit code and the last of the output.
- The agent reads the result and uses it as evidence for its outcome.
Where execution happens
The agent stays in CodeBook and asks the runner on your machine to execute work. In controlled mode it can request only commands you approved by name. Autonomous mode also allows eligible agents to send shell lines, subject to the runner’s sandbox, network, permissions and approval settings.
Set it up
Open Factory settings → Runtimes → Command runners.
- Add a runner and name it — the machine it will run on is a good name.
- Optionally give a line to run before each command, such as
npm ci. - List the commands: a name, what it runs, and a time limit in seconds (10 to 3600; 900 by default). For example:
| Name | Runs |
|---|---|
test |
npm test |
lint |
npm run lint |
typecheck |
npx tsc --noEmit |
build |
npm run build |
You do not have to type them from memory. Find commands in the repository reads the repository's own files — package.json, a Makefile, pyproject.toml, Cargo.toml, go.mod, a Maven or Gradle build — and proposes commands for you to tick. A script's own text is never proposed, only <package manager> run <name>, and nothing is approved until you add it and save:
CodeBook discovers → you approve → agents can ask for it.
- Choose Create runner and save its one-time token in the runner machine’s secret store as
CODEBOOK_RUNNER_TOKEN. Use the CodeBook web origin shown by your deployment forCODEBOOK_URL. Use a directory where this machine’s runner program and its replacement are kept.
On macOS or Linux, with the token supplied to the process environment, run:
export CODEBOOK_URL=https://app.codebook.work
curl --fail --location --output codebook-runner.mjs "$CODEBOOK_URL/runner/codebook-runner.mjs"
node codebook-runner.mjs --check
node codebook-runner.mjs
PowerShell uses the same program. Supply the token in the process environment, then run:
$env:CODEBOOK_URL = 'https://app.codebook.work'
Invoke-WebRequest "$env:CODEBOOK_URL/runner/codebook-runner.mjs" -OutFile codebook-runner.mjs
node .\codebook-runner.mjs --check
node .\codebook-runner.mjs
The machine needs Node 20 or newer, Git and clone access as the runner’s OS user. For sandboxed commands install or enable a supported execution backend. Linux can use bubblewrap, Podman or Docker. macOS can use its system sandbox, Docker or Podman. Windows uses a supported container engine for sandboxed commands. The program can download Chromium for page checks. Leave the polling process running and confirm that the registered runner appears online before requesting a command.
Verify or upgrade a runner
- During an upgrade, let the current command finish, stop the old process and download a reviewed replacement from the same CodeBook origin. Keep the existing environment and work directory. The customer command runner shipped with this release is 2.1.0.
- Run
node codebook-runner.mjs --check. Read the individual Git, CodeBook, sandbox and browser lines. With the URL and token set, the CodeBook line should identify the registered runner. Use--setupto prepare its browser ahead of time. - Start the replacement as the same OS user. Request one approved command on a known test commit and check Recent runs for the command, commit, exit result and output. A browser-capable runner can also run a check against a permitted test URL.
- If it stays offline, compare the URL and registration, check the outbound connection and replace the stored token after a rotation. If Git cannot fetch the commit, test read access as the runner user. For a sandbox refusal, inspect the backend reported by
--check, its permissions and the runner’s configured requirements.
A work order with a deadline or spend cap holds commands on outside runners that cannot enforce those limits. Inspect Last operation held on the work order. Work order limits
Actions, not shell strings
Each command is stored as a structured action — an executable and its arguments — and is started without a shell, so nothing in an argument is ever interpreted. Under Who may run it, approval, folder and environment each command can also have:
- A folder inside the repository to run in, such as
packages/api. - Environment variables, as
NAME=value. These are stored as written: not for secrets. - Run through a shell — only needed for a pipe,
&&, a redirect, a variable or a wildcard. A person has to tick it. - Agents that may run it. Leave empty for any agent of the factory; tick some to limit it, for example
e2eto the QA agent only. - A person approves every run — for commands that change something, such as a migration. The run waits in the settings screen until you approve or decline it, and is cancelled after a day.
What agents can do
When a factory has a runner with at least one command, agents are offered three tools — both while working a card and in a conversation, so you can simply ask an agent in chat to run the tests on a pull request:
- List the commands — the approved names, what each runs, and whether the runner is online.
- Run a command — one name, on a pull request's head commit or on the default branch. It is queued and comes back at once with a run id.
- Read a run — queued, running, or how it ended, with the last of what it printed. One read can wait up to 50 seconds for the result.
A run ends one of these ways:
| Result | Meaning |
|---|---|
passed |
The command exited 0. |
failed |
It exited with another code. This is evidence about the code. |
error |
It could not be run: the checkout failed, or the setup line failed. |
timeout |
It ran past its time limit and was stopped. |
lost |
No runner took it within 30 minutes, or a runner took it and never reported. |
error, timeout and lost say nothing about the code, and agents are told so.
Policy over the runner
Two optional lists under Factory settings → Policies & budgets use the runner without any agent having to remember to:
- Commands that must pass before CodeBook merges. CodeBook runs them on your runner against the pull request merged with the main branch as it is now — nothing is committed or pushed — and waits. If another merge moves main first, the earlier answer no longer counts and they run again: a merge queue. A failure sends the card back to the developer with the output. A command that cannot be run at all leaves the pull request for a person. See Pull requests.
- Commands that verify a deploy. After a production deploy succeeds, CodeBook runs them on the deployed commit. See Production monitoring.
Autonomous mode
A runner is controlled by default: named commands only. That is the recommended way to run one.
Set to autonomous, agents that are allowed to write to the repository can also send their own shell lines — install, build, run, inspect — in the pull request's checkout. The checkout is kept between lines while the agent stays on the same pull request, so it can work the way a developer does at a terminal.
It is off unless a person turns it on twice:
- The runner is set to Autonomous in Factory settings.
- The program is started with
--autonomouson the machine.
Either one alone runs no shell line. You can also limit which agents may send lines, and require that a person approves every line.
A fixed blocklist refuses lines that:
- ask for administrator rights (
sudo); - delete or change permissions outside the checkout;
- stop the machine or write to a disk device;
- pipe a download into a shell;
- read credentials on the machine (
~/.ssh,~/.awsand the like); - push to the repository or publish a package;
- print the machine's environment;
- leave something running after the line ends.
The sandbox
Approved named commands and setup steps use a native sandbox when available, with a container fallback. On macOS the native backend is Seatbelt. On Linux it is bubblewrap. Selecting Docker or Podman explicitly runs these commands in that engine. An autonomous shell line prefers a container in automatic mode and can use an available native backend.
| Condition | What happens |
|---|---|
| Container backend selected | A temporary container mounts the checkout and can use the runner’s scoped dependency cache. The image defaults to node:22-bookworm. |
| Native backend selected | Commands use installed tools within the OS sandbox, with a limited environment and writable paths. |
| No usable sandbox | Approved commands refuse by default. Direct execution requires starting the machine’s runner with --sandbox none. A shell line for which the factory requires a sandbox is refused. |
The setup step can reach the network to install dependencies. Other sandboxed commands use the runner’s network setting. Checkout and cache directories are separated by organization and repository. Autonomous lines also have a five-minute limit and a rate limit of 120 lines per factory per hour. The runner records the line and output. Repository changes still go through the pull request workflow.
Browser checks
A command that exits 0 does not tell you the page works. When you list addresses an agent may open, agents can open a page in a real browser on your runner and report what a person would see:
- the final address and title, and the HTTP status;
- the text on the page;
- console errors and failed requests;
- whether each step and expectation held;
- a screenshot, kept for you under Recent runs.
Steps are a small fixed set — go to an address, click, fill a field, press a key, wait for something, expect some text, expect some text to be absent. There is no free-form script.
Two rules keep this safe:
- Only the addresses you listed can be opened. A link that leads anywhere else is stopped and reported. An agent cannot point the browser at an arbitrary site or at your internal network.
- Secrets are typed by name. List a variable name such as
TEST_PASSWORDunder Variables on the machine that may be used by name, set it in the environment where the runner runs, and an agent can fill a field with it without ever seeing the value. Only the name is stored in CodeBook.
Nothing extra to install. The runner brings its own browser: the first time your factory wants page checks from a machine, it downloads Chromium into its own folder (about 150 MB, once). Run the runner with --setup to do that ahead of time. --no-browser-install stops it from ever downloading one.
More steps are available than the basics: choose an option in a list, expect an element to be visible, expect the address to contain something. No JavaScript is ever taken from an agent.
Recordings, and other files from a run
The runner records browser checks and sends the recording back with the result. It plays on the card's Evidence, beside the steps. The recording is made at 960×600 to keep it small, is removed from your machine once sent, and is kept in your organization's file storage in Bookbag — CodeBook's own servers keep no files — where it counts against your storage like any other upload.
The same road carries other files a run produces: a trace, a log, a coverage or test report, a screenshot. A runner can add a file only to a run it ran, within the hour; a run keeps at most five files of 15 MB each; and a file is refused if its contents are not what its type says.
If a successful local run has no recording, inspect its artifact upload result and the organization’s file storage connection in Bookbag. On machines with a native sandbox, the runner can launch the browser child inside it. Elsewhere the browser runs in its own process with a limited environment and the configured address rules.
Capabilities and health
Under Browser checks, secrets by name and kept folders, and in the autonomous section, each runner has its own settings:
| Setting | What it controls |
|---|---|
| Sandbox, image, network | Where a shell line runs and what it can reach |
| Variables used by name | Which of the machine's variables a sandboxed line or a browser step may use |
| Addresses an agent may open | The browser's allow-list |
| Folders kept between runs | What survives the clean before each named command — node_modules, .venv, vendor, target by default |
The runtime check
Every time the program asks for work it reports its machine, and the settings screen turns that into a check per runner:
| Line | Ready when |
|---|---|
| Connected | The program is running and reaching CodeBook |
| Git | git is on the machine |
| Approved commands | At least one command is listed |
| Page checks | Addresses are listed and the runner has its browser |
| Sandbox | The machine has Docker, Podman, macOS Seatbelt or Linux bubblewrap |
| Autonomous shell | The runner is autonomous, the program was started --autonomous, and a sandbox is there if required |
| Variables by name | Every variable you listed is set on the machine (names only are reported) |
| Machine | Enough free disk |
Each line that is not ready says what to do. The capabilities are independent: a laptop with a browser and no container tool still checks pages; a build server with containers and no browser still builds and tests.
On the machine itself, node codebook-runner.mjs --check prints the same check, and --setup also downloads the browser:
CodeBook runner 2.1.0 — runtime check
Git ✓ git version 2.50.1
Node ✓ v24.19.0
Browser ✓ ready (the runner's own Chromium)
Sandbox ✓ the macOS sandbox
Disk ✓ 109.5 GB free
CodeBook ✓ connected as "Build box"
Ready for approved commands, page checks and sandboxed autonomous work.
What it cannot do
- Controlled mode accepts command names. The agent chooses a name from your list. It cannot add arguments, a command line or environment variables. For a repository recipe, the registered line must also match the saved recipe. If that registration changes after you request a proof, review the change and request a fresh run. Check a recipe against a runner
- The runner's token is narrow. It takes and reports its own factory’s runs and uploads artifacts for runs it performed. It reads no card, setting or other run. Switching a runner off or making a new token stops the old one at once.
- CodeBook holds no repository credential for it. The code is fetched by your machine's git.
- It runs nothing else. The program prints every line before running it. It is one file with no dependencies, so you can read it before you run it.
Options
| Option | Meaning |
|---|---|
--url |
Your CodeBook address (or the CODEBOOK_URL environment variable) |
--token |
The runner's token (or CODEBOOK_RUNNER_TOKEN) |
--workdir |
Where checkouts are kept. Default ~/.codebook-runner |
--allow test,lint |
Accept only these command names on this machine, whatever the list says |
--ssh |
Clone over SSH instead of HTTPS |
--autonomous |
Also accept shell lines agents write — only if the runner is set to autonomous in settings |
--sandbox auto |
Where a shell line runs: auto (containers or the available native sandbox), docker, podman, native, or none |
--setup |
Check the machine, download the runner's browser, and exit |
--check |
Show what the machine has and whether it reaches CodeBook, and exit |
--no-browser-install |
Never download a browser |
--once |
Take at most one run and exit — for cron or a scheduled job |
--dry-run |
Print what would run; run nothing |
Each repository gets one checkout in the work directory, reset to the exact commit before every run. Dependency folders such as node_modules are kept between runs so installs stay fast.
Container timeouts
The runner removes the step’s Docker or Podman container when its command times out or fails. If the engine cannot confirm cleanup, the report names the container and the runner stops accepting work. Check the engine and remove that container before restarting the runner. A required but unavailable sandbox refuses the shell line; it does not execute it on the host.
Safety
Use a dedicated runner user and keep its Git access scoped to the repositories it needs. Commands approved for direct execution run with that user’s rights. A pull request can change what npm test does, exactly as it can on any CI system: review who can open pull requests on the repository.
Next
- Tool connections — connect a hosted testing service instead of, or as well as, a runner.
- Pull requests — who merges finished work, and what happens after the merge.
To verify which repository commands your runner should execute, set up and dry-run the recipe. Repository reports and proof runs