# Evidence and limits

Version 0.1.0, prepared October 14, 2026. The human playbook is a practitioner's account. Observed case findings, pricing calculations, starter examples and proposed experiments are kept separate.

## Author's operating scale and ambition

On October 15, Hamed described the current working scale as tens of agents and the ambition as hundreds or thousands of autonomous agents working in parallel alongside developers. The former is the author's account of current practice, not a reconstructed peak-concurrency measurement. The latter is a vision, not an achieved capacity benchmark. The retained session counts below measure activity over a period and cannot establish simultaneous agents. No new activity or cost totals were collected for this editorial revision.

## Journey and benchmark comparisons

The opening statement describes personal practice evolving alongside recorded output. It does not attribute a causal multiplier to the playbook. Both external sources were rechecked on October 14, 2026.

| Comparison | Own monthly average, April–September | Published reference | Raw volume ratio |
|---|---:|---:|---:|
| Merged PRs vs daily Cursor cohort | 770 / 6 = 128.33 | Median 4.1/week, converted using 52/12 to 17.77/month | 7.22× |
| Authored PRs vs AI-forward company | 829 / 6 = 138.17 | Mean 44.3/month per active developer | 3.12× |

[DX's Q1 2026 report](https://newsletter.getdx.com/p/ai-tooling-benchmarks-pr-throughput) covers 64,680 developers across 219 companies. The relevant comparison is the median **merged PRs** for daily Cursor users; it is not an industry-wide average or all Cursor users.

[The enterprise study](https://arxiv.org/html/2607.01904v1) reports an April 2026 company mean of 44.3 **authored PRs** per active developer. Section IV-A, footnote 6, distinguishes it from the company's merged-PR target. It is one AI-forward company's mean, not the output of the world's top AI users or a percentile benchmark.

The 2.66× journey figure is the historical multi-repository commit count, 2,559 / 963. Merged PRs rose from 85 in April to 154 in September, 1.81×. October's partial 431 merges (through October 15) / September's complete 154 = 2.80×. Keep each metric named; do not relabel commit volume as the same productivity quantity as PR volume.

Work mix, PR size, review standards, quality, available time and observation periods are not matched. The 40-hour human week is an assumption rather than a time log. The evidence supports a substantial personal output trajectory, alongside evolving practices, not headcount replaced or proof that these practices caused 7× or 3× gains.

## Cost per task, October 5 to 14

Three cohorts with different scopes. Keep them separate.

| Cohort | Scope | Result | Source |
|---|---|---|---|
| October 5 baseline | About 20 tasks in one long coordinating session with 54 subagents; coordinator ran about 12 hours | $327.50 calculated ($326 shown in the app), about $16.38 per task | Private cost study; see the [cost section](https://hamedtaheri.com/playbook/v0.1.0/playbook.md#4-manage-usage-and-cost) |
| October 5 single-task comparison | The same task run three ways with `claude -p` (Claude Code CLI 2.1.215 for the first two, 2.1.290 for Opus) | Default worker $1.86, 21 calls, 109K peak context, one acceptance check missed. Lean Sonnet 5.5 $0.48, 8 calls, 45K peak, all checks met. Lean Opus 5.5 $0.73, 11 calls, 53K peak, all checks met | Private run ledger |
| October 6–14 pipeline runs | 206 runs over 201 tasks; 205 have recorded dollar values. Includes stops and incomplete stages; coordinator and unpriced Codex usage excluded | Median recorded $0.67 per run, 90th percentile $1.69, highest $2.72, total $167.78. Stage medians: prep $0.18 (176 records), worker $0.43 (199), review $0.11 (151) | [pipeline-outcomes.json](https://hamedtaheri.com/playbook/v0.1.0/evidence/pipeline-outcomes.json) |
| October 10–13 whole workflow | 444 accounting rows, 106 integration PRs, coordinator included | About $339 at October 14 prices; $3.20 per PR; $5.30 per completed leaf task | [october.json](https://hamedtaheri.com/playbook/v0.1.0/evidence/october.json), [cost-per-task.json](https://hamedtaheri.com/playbook/v0.1.0/evidence/cost-per-task.json) |

All figures are Claude Code cost estimates at API prices, not invoices; most of the work ran on a subscription. The October 6–14 and October 10–13 cohorts overlap in time but are counted differently: one row per pipeline run versus every recorded model call, including the coordinator. Do not add them together.

**Pipeline outcomes, October 6–14.** 31 runs ready for the landing gate, not yet merged; 120 approved with a reviewer `MERGE AFTER` condition that holds them for a person; 37 stopped by the worker and returned to preparation; 10 finished without a PR; 4 stopped at preparation for an open question; 3 errors; 1 failed review. A run is not an accepted task: review conditions, integration and acceptance still need to be established. The recorded dollar totals omit unpriced Codex stages; a recorded zero or partial amount does not prove zero total usage cost. Post-merge defects, reverts and human minutes are not measured yet.

**Other single incidents cited in the playbook,** from the private notes kept at the time: a PR with nine fix rounds and two follow-ups (about $6.68, about 2.5 hours, October 6); a spec review run as an in-session subagent (about $17.30, 124 model calls, context 63K to 362K tokens, October 11); a CDN invalidation of 8,336 single paths ($36.70, October 6); a startup document of 301K characters in a 1,051-document organization, reduced to 3.9K by capping its catalog (October 5).

Earlier versions of this page also showed a $10,000-a-month API-equivalent usage target and a projected October PR count. Both were removed from the main page because they measure usage and pace, not delivered value; the comparisons below are kept for reference.

## Historical output

[Monthly data](https://hamedtaheri.com/playbook/v0.1.0/evidence/output-history.json) contains 829 authored and 770 merged PRs for April-September 2026. Authored counts use creation date; merged counts use merge date and can include older PRs. Counts cover the author's accessible repositories and account, using the earlier audited snapshot. They are not a cohort acceptance rate. The associated 40-hour week is a user-chosen assumption; comparable human time was not measured.

This release does not convert those counts into labor saved, headcount replaced or an AI-caused multiplier. Tests, documents and split deliverables are valuable work but need distinct categories for comparison.

## Broader activity and October refresh

The historical Git audit adds 9,527 commits across April–September: 963, 1,200, 1,319, 1,591, 1,895 and 2,559 by month. September is 2.66 times April. These counts combine local and remote reachable history, deduplicate identical hashes and use provisional author names. Superseded rebased hashes can inflate activity. Commit volume is not accepted work or the same scope as the GitHub PR series.

The October refresh uses authenticated GitHub search for PR author `hataheri`, October 1 at 00:00 Vancouver through October 14 at 11:57:12 Vancouver (18:57:12 UTC). Four pages for each series returned 333 creations and 312 merges, with no incomplete-result flags; totals, unique IDs and every event timestamp passed validation. The public data includes aggregates and pagination evidence; PR bodies, repository names and raw identities remain outside the release. Author-scoped searches exclude PRs authored by other people or bots, even in the author's repositories.

| Vancouver period | Authored | Merged |
|---|---:|---:|
| October 1–7 | 93 | 83 |
| October 8–14, as of 11:57 a.m. | 240 | 229 |
| October so far | 333 | 312 |

A second refresh with the same rules runs to October 15 at 11:30 p.m. Vancouver (`refresh_prs.py`, which re-checks pagination, totals and unique IDs). It reproduced the October 14 counts exactly, then added the rest of that day and October 15:

| Vancouver period | Authored | Merged |
|---|---:|---:|
| October 1–7 | 93 | 83 |
| October 8–14 | 282 | 270 |
| October 15, to 11:30 p.m. | 80 | 78 |
| October so far | 455 | 431 |

October's 431 merges already exceed September's complete-month 154 by 2.80 times. This is a raw volume comparison across different periods and task mix, not an hourly productivity estimate. The 106-PR priced case is a subset; its price must not be multiplied by the wider October PR count to estimate total October cost.

## Session and usage history

The original October 1 report and its calculation script were recovered from the local Claude Code session history. [Selected aggregates and definitions](https://hamedtaheri.com/playbook/v0.1.0/evidence/ai-activity.json) preserve that historical snapshot; the older source session files may since have been deleted. A new October-only scan uses the same first-record token method with an explicitly bounded Vancouver interval.

| Period | Main sessions | User-role messages | Active session-hours proxy | Replies with usage | Helper reply share | Weighted token index, millions |
|---|---:|---:|---:|---:|---:|---:|
| June | 1 | 110 | 20 | 2,572 | 22.0% | 166 |
| July | 9 | 362 | 79 | 9,225 | 47.6% | 502 |
| August | 46 | 1,660 | 249 | 51,740 | 66.2% | 2,358 |
| September | 144 | 3,326 | 434 | 109,976 | 69.3% | 4,108 |
| October 1–14, through 12:09 p.m. Vancouver | 956 | 3,261 | 155.8 | 44,424 | 57.0% | 1,388.2 |

![Retained Claude Code activity and partial October](https://hamedtaheri.com/playbook/v0.1.0/figures/ai-activity.svg)

Session count uses distinct main-file session IDs with an event in the period, including resumed and headless worker/reviewer sessions. The time proxy sums adjacent main-session timestamp gaps **strictly under ten minutes**; longer gaps are excluded. Concurrent sessions sum, so this is neither elapsed calendar time, actual compute time nor measured human labor. Helper transcript durations do not enter the hour count. User-role strings can be automated prompts; they are not necessarily personal interventions.

The historical weighted activity formula is `fresh input + 0.1 × cache reads + 2 × cache writes + 5 × output`. It applies fixed weights across models and is **not current API pricing**. June–September's retained index grows 24.75 times; July–August grows 4.70 times and August–September 1.74 times. The old phrase “nearly doubled each month from July” is inaccurate. April–May have no retained records in the original report, and June–July were incomplete. Do not convert that observed index growth into a complete-account utilization multiplier.

The historical script grouped timestamps by UTC month. Git and PR tables use Vancouver; these are neighboring descriptive series, not precisely joined task cohorts. The old “text Claude wrote” column is output **tokens**, not words or characters. Helper share measures reply counts, not the share of delivered work. First-record deduplication can undercount later partial output; it is preserved to explain the old index rather than used for invoice-grade accounting.

The fresh October scan read 4,090 retained JSONL files with no unreadable files, invalid selected rows or missing assistant IDs. It recorded 7.829 billion cache-read tokens and 17.281 million output tokens. No message bodies, private session IDs, prompts or transcripts are exported. It excludes Claude web/phone, other computers and Codex usage.

## October Git detail

The matching multi-repository refresh now records **1,722 commits** in October 1–14: 824 in the first week and 898 in the second week so far. This uses the same provisional Hamed author-name rule, original repository inventory, remote-branch plus local-ref scope, and global commit-hash deduplication as the historical chart. All 56 remote repositories were refreshed in isolated bare snapshots, and all 43 common local Git directories were read successfully. Active checkouts and their refs were not changed.

The same scope records +300,840/-28,680 code lines and 19,043 estimated replacement lines. The code filter includes tests/comments/blank lines; the replacement estimate overlaps additions/deletions. The exact cutoff and aggregate weekly totals are in `matchedHistoricalScope` in [the Git dataset](https://hamedtaheri.com/playbook/v0.1.0/evidence/october-git.json). October is incomplete, and the earlier monthly counts retain their original snapshot dates.

[Git aggregates](https://hamedtaheri.com/playbook/v0.1.0/evidence/october-git.json) use local reachable history through October 14 at 12:08 p.m. Vancouver. The `origin/dev` snapshot contains 572 non-merge commits from all repository authors, with 104,310 code lines added, 11,089 removed and 6,666 estimated replacement lines. Claude co-author trailers appear in 476/572, or 83.2%. The local `origin/master` snapshot has the same non-merge totals; branch reachability does not verify production deployment.

The Hamed-attributed subset on `origin/dev` is 442 non-merge commits, 75,038 code lines added, 8,298 removed and 4,083 estimated replacements. Across all local refs, all-author non-merge activity is 670 commits, +123,625/-12,762 code lines. These broader refs can include unmerged or superseded work and should not replace the integrated-branch denominator.

Code filtering matches the earlier audit: conventional code extensions, with generated/vendor/build paths excluded; tests, comments and blank lines remain included. The update estimate sums `min(added, removed)` per file per non-merge commit. It overlaps additions/deletions and is not an independent exact revised-line count.

The old 780 September commits and 87% Claude attribution describe `ai-fargate`'s all-author, non-merge, all-ref snapshot. The 2,559 September commits describe the multi-repository Hamed-name-attributed series, including merges. Both repository scope and counting rules differ. They must not be spliced into one growth series.

## October workflow and pricing

[October summary](https://hamedtaheri.com/playbook/v0.1.0/evidence/october.json) reports the complete recorded cohort's visible model usage. It excludes the parallel Retrieve cohort and counts preparation, stopped and finished workers, review/fix work, coordinator, attached subagents and direct runs. There are 444 accounting rows, not 444 tasks.

Current standard first-party API rates produce approximately $339 for the 51.7-hour elapsed case and $3.20 per integration PR across 106 PRs. Active human time was not measured. Integration is not production deployment. Ancillary non-persisted classifier usage may be absent; paid CI, infrastructure and human labor were not established. This is counterfactual API valuation of subscription activity, not an invoice.

![Repricing the same recorded activity](https://hamedtaheri.com/playbook/v0.1.0/figures/pricing-correction.svg)

The original $535.50 valuation and current $339 valuation price the same activity differently. The difference is not an additional workflow improvement. Do not infer a subscription's monthly dollar ceiling from a single session's reported quota reading.

## Practical observations used

The source field notes document repeated review rounds, specifications losing important constraints, workers stopping on genuine gaps, tests mocking the changed layer, shared-checkout Git lock collisions, and a migration-runner defect caught by rehearsing the actual apply tool. These are selected operational cases, not randomized comparisons or proof that the practices generalize to every team.

The memory adapter and compact report/cost tools shipped here are new synthetic examples. Their local tests qualify their stated behavior. They are not the original production runner, a hosted memory backend or a live-qualified Personize integration.

## Research context

- [Three software-developer field experiments](https://www.microsoft.com/en-us/research/publication/the-effects-of-generative-ai-on-high-skilled-work-evidence-from-three-field-experiments-with-software-developers/): assistant access increased completed tasks in the pooled trial; that intervention differs from this operating method.
- [METR's early-2025 trial](https://arxiv.org/abs/2507.09089) and [February 2026 update](https://metr.org/blog/2026-02-24-uplift-update/): findings depend on setting; later selection effects and concurrent-agent timing complicate measurement.
- [Chen and Stratton's August 2026 working paper](https://fion.ac/jellyfish.pdf): coding gains did not fully translate into software output, with a review bottleneck.
- [DORA metrics](https://dora.dev/guides/dora-metrics/): use service-specific delivery throughput and instability as additional outcomes.
- [SemiAnalysis's October methodology](https://newsletter.semianalysis.com/p/anthropic-subscriptions-offer-5x): API-equivalent subscription value depends on plan, model and workload. The exact $11,726 figure located in secondary reporting has not been independently extracted from its primary chart and is not used as a measured result here.
- [Claude API pricing](https://platform.claude.com/docs/en/about-claude/pricing) and [Codex usage guidance](https://learn.chatgpt.com/docs/pricing): tool/model details and limits are dated; recheck before selecting a plan or projecting capacity.

## What the next data should establish

Measure active human time across preparation, questions, review, debugging and integration. Group PRs into independent accepted deliverables, classify work mix, retain failed attempts and shared costs, and observe attributable repairs over a stated follow-up window. Compare matched cohorts at the same pricing basis and acceptance standard, then seek replication by another team.

The desired x-times utilization, y-percent cost reduction and z-percent output increase remain measurement objectives. This release does not publish them as achieved universal results.

## Cost per task bridge

The [measurement reference](https://hamedtaheri.com/playbook/v0.1.0/reference/cost-per-task.html) defines attempt, solved-trial, accepted-task and integration-PR costs. The [derived ratios](https://hamedtaheri.com/playbook/v0.1.0/evidence/cost-per-task.json) preserve unrounded numerators and explicit denominators from the October cohort. The [task reconciliation](https://hamedtaheri.com/playbook/v0.1.0/evidence/october-task-audit.json) now establishes $5.30 of recorded-window usage per 64 board-completed leaf task records, after removing matched parents and retaining unsuccessful work. It does not establish lifetime costs or independently verified acceptance; accepted-task cost remains unknown. Stage averages exclude shared overhead; the whole-workflow per-PR ratio includes all recorded components.