What got through—and can I trust it?

Back to the playbook · Measurement instructions for your AI · Aggregate data

Updated October 20, 2026. Observation window: October 1–19, Vancouver time, ending at midnight on October 20. Earlier cost studies and snapshots retain their own dates.

Read the results in three questions

What work got through? 185 work items have every recorded part merged. Across the full cohort, 275 implementation units merged. A work item is the requested outcome; an implementation unit is a part of that work. A PR carries code changes and can be a repair. These counts describe different levels and must not be added together. The source records do not yet reliably distinguish accepted bug fixes from accepted feature slices.

Can I trust those changes? Of the 275 merged units, 213 are classified as reviewed with passing tests: 146 with deep review and 67 with standard review. The remaining 62 have different or incomplete evidence. 14 of those carry specific audit flags. This is evidence coverage, not certification that every original requirement works; the classifier's limits are below.

What did it cost? Recorded pipeline usage was $1,065.06, or $3.87 per merged implementation unit, including unfinished attempts and unmatched usage. This excludes the coordinator, human time and infrastructure, and prices model usage at API-equivalent rates. Full cost per independently accepted outcome is still unknown.

These are the headline results. Repair counts, review rounds and elapsed times below help explain where the process needs improvement; they are supporting diagnostics.

What I mean by done

For each outcome I want evidence for: requirement defined → implemented → behavior verified → review findings resolved → integrated → accepted. A change counts only as far as the evidence supports. The current audit establishes integration and some verification evidence; independent acceptance remains unreconciled.

For a bug fix, the useful evidence is the original failure, a test or reproduction showing the fix, relevant regression checks and review of the final change. For a feature slice, it is the requested behavior demonstrated against its acceptance criteria. The review practice explains the checks. Team memory connects the task, criteria, decision, revision and verification references so the next person can inspect them.

Keep work size separate from work completed

For future comparisons I want the work classified before execution: a small change to one behavior, a change spanning several components, or a larger change involving interfaces, migration or rollout risk. Compare bug fixes with bug fixes and feature slices with similar feature slices. The current audit has no reliable size or work-class labels, so it cannot report a number of equivalent bugs or features.

Lines changed can describe the code footprint. An estimate of equivalent human effort needs actual baseline examples with comparable scope and verification; I will not assign hours to a task after seeing how many lines an agent produced. Features and epics group outcomes. Per-developer results require an explicit human-effort denominator.

Why I call it The 20× Developer Playbook

I built a system that lets me define work with agents, share decisions and context through governed team memory, run focused workers, and bring changes back through review and integration. My account recorded 770 merged PRs in October 12–18. The scale of execution has changed enough that PRs alone no longer explain what the system delivers.

20× is my target for repeatable software delivery capacity per developer: comparable accepted parent tasks per developer-week against a stated baseline, with human effort, quality and total cost reported alongside it. The title gives the work a direction as I bring more developers into the workflow. It is not a mathematical discount applied to the PR ratio, a guaranteed result, or an isolated claim about memory.

The eventual calculation is (accepted parent tasks / developer-weeks with the system) / (accepted parent tasks / developer-weeks in the baseline). Keep work categories and complexity comparable, set acceptance before execution, and include the human effort of everyone preparing, reviewing and integrating the work. Human minutes per accepted task provide a second check on the result. For illustration only, 100 comparable accepted tasks per developer-week against a baseline of five would establish 20× on that measure. Those are not my measured task counts.

The PR benchmark, with its counting rules

DX's January 2026 report reports a median of 4.1 merged PRs per week for daily Cursor users.

Using the rounded reference of four gives 770 / 4 = 192.5×. These are PR-volume ratios against a dated external cohort. The other branches include integration and release branches; the 308 are not independently accepted features or verified production deployments. The full October 1–19 window contains 1,026 authored PRs and 972 merges. Creation and merge dates are separate series, so their ratio is not a success rate.

DX's throughput documentation normalizes by contributing engineers and excludes bot-authored PRs. My count includes agent work authored under my account. Task size, work mix, automation, human contributors and counting rules need alignment before interpreting a comparison as productivity. This one week is an observed result, not a forecast or a sustained average.

Calculation details

The deduplicated runner ledgers cover 457 task units grouped into 308 parent tasks, across five repositories; 406 units belong to one TypeScript backend. This is a subset of account-wide activity. Fix-worker runs belong to the unit they repair. Feature parts remain separate units; parent completeness covers the parts recorded in these ledgers.

The elapsed-time figure begins at the first retained activity event, not necessarily task creation or the start of preparation. It ends at integration, not production or acceptance. All 266 direct intervals are non-negative and have both timestamps. The cohort remains right-censored: work still open can finish later, and timing statistics over completed units omit those slower unfinished cases.

The review/test classes come from the existing audit classifier. GitHub's final-head CI rollup was retrieved live; it does not reconstruct the check state at the historical merge instant. These are flags for investigation, not 14 confirmed customer defects. Reviews of a repair PR cover its diff, not automatically the combined final change. The earlier review audit retains its historical 272-unit snapshot and hand-check findings; the numbers above are the refreshed 275-unit view.

All dollar amounts are API-equivalent estimates recorded by the CLI, not invoices or software value. The $3.87 ratio keeps the whole measured pipeline numerator, whereas the $1.37 median describes only usage attributed to merged units. Neither establishes full cost per accepted task. The earlier coordinator-inclusive studies remain in the cost reference.

Which proposed metrics still need data

Unknown values remain unknown; they do not become zero interventions, perfect handoffs or failed deliveries. Session gaps and automated user-role messages do not measure a person's active time.

How I use established measures

The Kanban Guide uses throughput, cycle time, work in progress and work-item age with explicit start/finish definitions. Here I can report merge throughput and a recorded-activity-to-merge timing proxy. Calling every unmerged unit active WIP would overstate what the ledgers show.

DORA measures production change lead time, deployment frequency, failed deployment recovery time, change fail rate and deployment rework rate. An integration PR or successful CI run cannot substitute for a production deployment. Those measures belong in the next instrumentation pass, per service.

For our team, I want accepted parent tasks per developer-week and human minutes per accepted task at the center, with repair, durability, cycle time and full cost beside them. Team memory should make accepted outcomes, decisions, handoffs and evidence easier to connect. The current calculations establish what our execution records show; the next measurements test how much of that becomes repeatable accepted delivery as more developers join.

Reproduction and scope

Private inputs are local preparation/worker/repair/review ledgers and authenticated GitHub metadata. REST PR searches were partitioned by day and checked for complete pagination and duplicate IDs; an independent GraphQL query checked all 972 merged PRs and their destinations. The public JSON contains aggregate counts, formulas, distributions, exclusions and the null accepted-delivery multiplier. Raw task names, PR links, repository breakdowns and transcripts are excluded from the download.

The source repository's scripts/playbook-metrics.py computes these aggregates from the private audit snapshots. Timing p90 uses the nearest indexed value at round(0.9 × (n − 1)) in ascending order. Rounding is for display; the underlying ratios use the unrounded counts.