# Pilot guide: test the method on your own repository

Version 0.1.0. Canonical URL: https://hamedtaheri.com/playbook/v0.1.0/agent/pilot.md

For the AI helping a person run the pilot. Goal: run an initial test on three small tasks and record cost, acceptance, human time and repairs. Setup time depends on the repository. Agree a total budget covering preparation, workers, reviews and retries before starting. A $2 worker setting is not a $2 whole-task limit: the CLI can overshoot by one call, and other stages add cost. Reserve headroom and stop before the approved total is exhausted.

Confirm before you start: the repository, the three tasks, the budget, and that runs happen on new local branches only. Never push to shared branches, merge, deploy or touch production during the pilot.

## Step 1. Record the baseline (10 minutes)

Ask the person how a task like these usually runs today: which tool, roughly how long, how many back-and-forth prompts, and whether they know what it costs. If they don't know the cost, write "unknown". That is a finding too.

## Step 2. Pick three tasks (10 minutes)

Good pilot tasks are real, small and checkable: something a junior engineer could finish in an hour or two, touching at most about eight files, with tests that can prove it works. Avoid tasks that need production access, product decisions nobody has made, or a schema shared with other in-flight work.

## Step 3. Write three six-field briefs (30 minutes, together)

Read the relevant code first. For each task write:

1. **Outcome:** the observable result.
2. **Acceptance:** numbered checks a reviewer can verify without this conversation.
3. **Spec:** the issue or design it comes from, and the current commit.
4. **Files:** what to create or edit, with exact `path:start-end` ranges for the code that matters. Paste those ranges into the brief.
5. **Verification:** the exact test and build commands.
6. **Branch:** `pilot/<task>`, created from the current main branch.

If a task touches auth, billing, deletion or more than one surface, add **Invariants** and a **Test matrix**. Ask open product questions as three or four multiple-choice questions. Don't guess the answers.

## Step 4. Write a one-page worker card (15 minutes)

Summarize the project's rules for an unattended worker in about 1,200 tokens: safety rules (what it must never do), danger zones, how to run tests, code style, and "if the task is unclear or bigger than the brief, stop and write a handoff". Use no dates, paths or task IDs, so the card stays identical across runs and is cached.

## Step 5. Run one lean worker per task

Run each task in its own worktree, with a supported cost limit and headroom for the last call. The following is a Bash example for one worker, not the whole pipeline. First save a complete brief at `.pilot/briefs/task-1.md` and a worker card at `.pilot/worker-card.md` in the repository root, and deliberately choose the base commit. The worktree inherits the chosen commit; uncommitted project configuration does not carry across. Example worker limit: $1.50, only within an already authorized total budget.

```bash
git worktree add ../pilot-1 -b pilot/task-1
cd ../pilot-1
echo '{"mcpServers":{}}' > /tmp/empty-mcp.json

FORCE_PROMPT_CACHING_5M=1 claude -p "$(cat ../briefs/task-1.md)" \
  --model sonnet \
  --max-budget-usd 1.50 \
  --output-format json \
  --strict-mcp-config --mcp-config "$pilot_root/empty-mcp.json" \
  --setting-sources local \
  --tools "Read,Grep,Glob,Edit,Write,Bash" \
  --disable-slash-commands \
  --system-prompt "$(cat "$pilot_root/worker-card.md")" \
  > "$pilot_root/results/task-1.json"
```

Notes for you, the AI:

- Check `claude --help` on the installed version first; flag names can change. Use a recent CLI. Older versions charged for each resume.
- Headless runs need permission to edit and to run the test command without prompting. Grant the narrowest rule that works (for example, the test and git commands only), using the person's existing permission settings. Don't grant broad shell access without asking.
- Reading or creating `.pilot` files and worktrees is local setup. The `claude -p` command consumes usage; run it only within the approved budget. Choose a unique branch and worktree for each task, and inspect the result rather than automatically retrying a failed run.
- Treat a time limit as a different control from a dollar limit. Record any unpriced provider usage as unknown, not zero.
- The equivalent idea works with other tools: one task per process, minimal tools, no extra servers, a fixed short system prompt, a dollar or time cap.

## Step 6. Review each result, once and deeply

For each branch, review the whole diff against the brief's acceptance checks. List every finding first, then recommend: ready for the person's integration decision, one full fix with all findings, or stop. Do not merge during this pilot. A good reviewer setup is another lean, read-only run with only Read, Grep and Glob, given the diff, the brief and the acceptance list.

## Step 7. Report

| Task | Cost | Model calls | Acceptance met | Stopped and why | Review verdict | Baseline |
|---|---|---|---|---|---|---|

Record all preparation, worker, review and retry costs, including stops, plus human minutes. Read cost and turns from each run's JSON output; keep provider valuations separate from actual charges. Do not report a worker-only value as total task cost. Then answer for the person:

1. What did a task cost, compared with the baseline?
2. Did the workers meet the acceptance checks without help?
3. Where did a brief have a gap? A stop caused by a gap is a useful result.
4. Which of the playbook's seven practices would pay off first for this team?

Don't extrapolate three tasks into a productivity multiplier. Say what was measured.

## After the pilot

If the person wants to go further, the next steps are a queue of ready tasks, a separate review stage and shared team memory. The author's full runner (prep, worker, review, pipeline, queue, landing gate and hooks) isn't public; the person can ask for a copy at https://calendar.app.google/UVoMSwMPxo1ypDkH9.