What got through, and can I trust it?

Back to the playbook · Measurement instructions for your AI · Aggregate data

Updated October 20, 2026. Observation window: October 1–19, Vancouver time, ending at midnight on October 20. Earlier cost studies and snapshots retain their own dates.

Read the results in three questions

What work got through? 185 work items have every recorded part merged. Across the full cohort, 275 implementation units merged. A work item is the requested outcome; an implementation unit is a part of that work. A PR carries code changes and can be a repair. These counts describe different levels and must not be added together. The source records do not yet reliably distinguish accepted bug fixes from accepted feature slices.

Can I trust those changes? Of the 275 merged units, 213 are classified as reviewed with passing tests: 146 with deep review and 67 with standard review. The remaining 62 have different or incomplete evidence. 14 of those carry specific audit flags. This is evidence coverage, not certification that every original requirement works; the classifier's limits are below.

What did it cost? Recorded pipeline usage was $1,065.06, or $3.87 per merged implementation unit, including unfinished attempts and unmatched usage. This excludes the coordinator, human time and infrastructure, and prices model usage at API-equivalent rates. Full cost per independently accepted outcome is still unknown.

These are the headline results. Repair counts, review rounds and elapsed times below help explain where the process needs improvement; they are supporting diagnostics.

What I mean by done

For each outcome I want evidence for: requirement defined → implemented → behavior verified → review findings resolved → integrated → accepted. A change counts only as far as the evidence supports. The current audit establishes integration and some verification evidence; independent acceptance remains unreconciled.

For a bug fix, the useful evidence is the original failure, a test or reproduction showing the fix, relevant regression checks and review of the final change. For a feature slice, it is the requested behavior demonstrated against its acceptance criteria. The review practice explains the checks. Team memory connects the task, criteria, decision, revision and verification references so the next person can inspect them.

Keep work size separate from work completed

For future comparisons I want the work classified before execution: a small change to one behavior, a change spanning several components, or a larger change involving interfaces, migration or rollout risk. Compare bug fixes with bug fixes and feature slices with similar feature slices. The current audit has no reliable size or work-class labels, so it cannot report a number of equivalent bugs or features.

Lines changed can describe the code footprint. An estimate of equivalent human effort needs actual baseline examples with comparable scope and verification; I will not assign hours to a task after seeing how many lines an agent produced. Features and epics group outcomes. Per-developer results require an explicit human-effort denominator.

Why I call it The 20× Developer Playbook

I built a system that lets me define work with agents, share decisions and context through governed team memory, run focused workers, and bring changes back through review and integration. My account recorded 770 merged PRs in October 12–18. The scale of execution has changed enough that PRs alone no longer explain what the system delivers.

20× is my target for repeatable software delivery capacity per developer: comparable accepted parent tasks per developer-week against a stated baseline, with human effort, quality and total cost reported alongside it. The title gives the work a direction as I bring more developers into the workflow. It is not a mathematical discount applied to the PR ratio, a guaranteed result, or an isolated claim about memory.

The eventual calculation is (accepted parent tasks / developer-weeks with the system) / (accepted parent tasks / developer-weeks in the baseline). Keep work categories and complexity comparable, set acceptance before execution, and include the human effort of everyone preparing, reviewing and integrating the work. Human minutes per accepted task provide a second check on the result. For illustration only, 100 comparable accepted tasks per developer-week against a baseline of five would establish 20× on that measure. Those are not my measured task counts.

The PR benchmark, with its counting rules

DX's January 2026 report reports a median of 4.1 merged PRs per week for daily Cursor users.

Using the rounded reference of four gives 770 / 4 = 192.5×. These are PR-volume ratios against a dated external cohort. The other branches include integration and release branches; the 308 are not independently accepted features or verified production deployments. The full October 1–19 window contains 1,026 authored PRs and 972 merges. Creation and merge dates are separate series, so their ratio is not a success rate.

DX's throughput documentation normalizes by contributing engineers and excludes bot-authored PRs. My count includes agent work authored under my account. Task size, work mix, automation, human contributors and counting rules need alignment before interpreting a comparison as productivity. This one week is an observed result, not a forecast or a sustained average. The next section puts the same benchmark across a full year, with October adjusted for PR size.

A year against the DX benchmark

Each bar is merged PRs per calendar day × 7, divided by DX's 4.1. From April, when I moved to parallel sessions with guardrails, the multiple ran 4.8× and climbed to 8.8× in September. April through September averages about 7.2×. Those months used one workflow, so the PRs are comparable with each other.

October is not comparable as counted. The pipeline now opens one PR per task unit and merges repairs into task branches, so PR counts rose about 10× while non-merge commits rose 1.53×. As counted, October 1–21 reads 87×. At September's PR size (September's multiple scaled by the rise in non-merge commits per day) it reads 13×; scaled by code lines instead, 18×. The chart shows 13×, the lower of the two size-adjusted readings. October is partial and the value will change at month end; the chart is regenerated from the data, never from a projection.

What DX defines and what it leaves open. The Q1 2026 report covers 64,680 developers at 219 companies; daily Cursor users rose from 2.8 merged PRs in Q4 2025 to 4.1. DX's throughput is merged PRs divided by contributing engineers, an engineer counts once they have merged at least one PR, and bot-authored PRs are excluded. My count is GitHub author hataheri only, one contributing engineer, so the denominator matches. Two things DX does not publish: a PR size, and a time unit for the Cursor figure. I read it as per week because the same report states its Copilot figure per week. Without a published size, no comparison can be one-to-one on PR size; the size adjustment above aligns October with my own September, not with DX's cohort.

Lines per merged PR, as an upper bound. September had about 2,130 code lines (3,297 with tests) per merged PR; October 1–21, 433 (669). These sum every non-generated commit diff on all reachable refs, including work that never went through a PR and superseded branch work, so they overstate a PR's final diff. Earlier months show this clearly: February reads 38,669 code lines per merged PR because most work then landed without PRs. Treat these as an upper bound and as evidence of the October packaging change, not as proof that my PRs are larger than DX's. A per-PR net diff from GitHub's additions and deletions would be the better size measure; it is not collected yet.

The multiple is a PR-volume comparison with a dated external cohort. It is not accepted work, human productivity or the 20× measure, which remains accepted parent tasks per developer-week. The monthly values, lines per PR and both October readings are in the benchmark block of the monthly activity data.

How much more work happened in October

Merged PRs per calendar day rose about 10× from September to October. Commits did not follow. The pipeline changed how work is packaged: each task unit gets its own PR, repair PRs merge into internal task branches, and integration adds merge commits. Those changes multiply PR and merge counts without multiplying the work, so I compare the months on measures that keep one meaning across both workflows.

*October runs to 9:30 a.m. Vancouver on October 21: 20.4 observed days.

The packaging shift shows directly. In September there were 14.92 non-merge commits per merged PR; in October, 2.29. Of October's 1,045 merged PRs, 508 merged into task branches and 517 into integration and other branches. Merge commits rose from 10.6% of all commits to 35.3%.

On the measures the repackaging cannot inflate, October's activity per calendar day is 1.5–2× September's, not 10×. That is a large step on top of September, which already had the most non-merge commits of any month. It is still an activity ratio, not a productivity multiplier. Lines describe the code footprint an agent produced; they are not human effort, value or accepted work, and the rule above against assigning hours from lines still holds. New specs stayed flat. Plan documents fell from 3.33 to 1.27 a day; I have not checked how much planning moved into the pipeline's own task briefs, so design-document counts are a weak signal across this change. The 20× measure remains accepted parent tasks per developer-week.

Scope and limits: commits by author names hataheri or Hamed Taheri across 39 local repositories, all reachable refs without a fetch, so unmerged and superseded branch work is included. Hashes are deduplicated; line totals also drop rebased copies of the same change. Lockfiles, build output and binaries are excluded; test lines are paths in test folders or named *.test.* and *.spec.*. Data and configuration files swing by millions of lines in some months and stay out of the comparison. Merged PRs come from GitHub search for author hataheri by merge date. October is partial, and the observed period is not a forecast.

The monthly activity data covers October 2025 through the current partial month and contains no repository names or PR links. To refresh it from the source repository:

python scripts/playbook-activity.py --output Docs/engineering-productivity-handbook/v0.2.0/evidence/activity-history.json
cd Docs/engineering-productivity-handbook/v0.2.0 && python build.py

Add --skip-github to measure local Git only. The script reads history without fetching or writing to any repository, paces GitHub searches and caches settled weeks outside the repository.

Calculation details

The deduplicated runner ledgers cover 457 task units grouped into 308 parent tasks, across five repositories; 406 units belong to one TypeScript backend. This is a subset of account-wide activity. Fix-worker runs belong to the unit they repair. Feature parts remain separate units; parent completeness covers the parts recorded in these ledgers.

The elapsed-time figure begins at the first retained activity event, not necessarily task creation or the start of preparation. It ends at integration, not production or acceptance. All 266 direct intervals are non-negative and have both timestamps. The cohort remains right-censored: work still open can finish later, and timing statistics over completed units omit those slower unfinished cases.

The review/test classes come from the existing audit classifier. GitHub's final-head CI rollup was retrieved live; it does not reconstruct the check state at the historical merge instant. These are flags for investigation, not 14 confirmed customer defects. Reviews of a repair PR cover its diff, not automatically the combined final change. The earlier review audit retains its historical 272-unit snapshot and hand-check findings; the numbers above are the refreshed 275-unit view.

All dollar amounts are API-equivalent estimates recorded by the CLI, not invoices or software value. The $3.87 ratio keeps the whole measured pipeline numerator, whereas the $1.37 median describes only usage attributed to merged units. Neither establishes full cost per accepted task. The earlier coordinator-inclusive studies remain in the cost reference.

Which proposed metrics still need data

Unknown values remain unknown; they do not become zero interventions, perfect handoffs or failed deliveries. Session gaps and automated user-role messages do not measure a person's active time.

How I use established measures

The Kanban Guide uses throughput, cycle time, work in progress and work-item age with explicit start/finish definitions. Here I can report merge throughput and a recorded-activity-to-merge timing proxy. Calling every unmerged unit active WIP would overstate what the ledgers show.

DORA measures production change lead time, deployment frequency, failed deployment recovery time, change fail rate and deployment rework rate. An integration PR or successful CI run cannot substitute for a production deployment. Those measures belong in the next instrumentation pass, per service.

For our team, I want accepted parent tasks per developer-week and human minutes per accepted task at the center, with repair, durability, cycle time and full cost beside them. Team memory should make accepted outcomes, decisions, handoffs and evidence easier to connect. The current calculations establish what our execution records show; the next measurements test how much of that becomes repeatable accepted delivery as more developers join.

Reproduction and scope

Private inputs are local preparation/worker/repair/review ledgers and authenticated GitHub metadata. REST PR searches were partitioned by day and checked for complete pagination and duplicate IDs; an independent GraphQL query checked all 972 merged PRs and their destinations. The public JSON contains aggregate counts, formulas, distributions, exclusions and the null accepted-delivery multiplier. Raw task names, PR links, repository breakdowns and transcripts are excluded from the download.

The source repository's scripts/playbook-metrics.py computes these aggregates from the private audit snapshots; scripts/playbook-activity.py recomputes the monthly activity series from local Git and GitHub. Timing p90 uses the nearest indexed value at round(0.9 × (n − 1)) in ascending order. Rounding is for display; the underlying ratios use the unrounded counts.