Understand the work
behind the metric.

How Engineering Output is scored

Engineering Output measures substantive engineering work shipped. Each eligible merged change earns points based on complexity, a bounded change-size adjustment, and penalties for missing tests and AI slop. Open pull requests and unmerged local work do not contribute.

Complexity is assessed across logic, scope, architecture, risk, and novelty in the repository’s context. The current scoring formula is:

Output = complexity weight × slop factor × test factor × size factor

• Complexity levels 1–5 map to weights 1, 2, 6, 12, and 24.
• Slop levels 1–5 map to factors 1, 0.85, 0.6, 0.3, and 0.15.
• The test factor is 1 when tests are present or not required; otherwise it is 0.8.
• The size factor is changed lines ÷ (changed lines + 50), where changed lines are additions plus deletions. The factor approaches 1 as size grows.

For example, a level-4 complexity change with slop level 2, relevant tests, and 100 changed lines earns 12 × 0.85 × 1 × 100 ÷ 150 = 6.8 Output points. This is an illustrative calculation.

Rework, code turnover, incidents, and reliability are later outcome signals. They do not directly discount this version of Engineering Output. Read them alongside the score to see whether delivered work held up.

Aggregation and denominators

Total Engineering Output sums eligible points over complete Monday–Sunday weeks. Weekly Output divides that total by the number of complete weeks. Engineering Output per Engineer divides each week’s Output by engineers with positive Output that week, then averages across the complete weeks.

Compare consistent teams, repositories, and windows. Engineering Output is a diagnostic of scored work delivered, not a direct measure of business value or an individual performance ranking. Recent analyses and backfills can revise earlier weeks.

Engineering Output succeeds the older complexity-only throughput model. Use this definition when interpreting the current product; the older 1/3/8 complexity examples are not interchangeable with this score.

AI Code Share and AI Output Share

AI Code Share is the AI-assisted and agent-authored share of attributable added lines in merged PRs. It is line-weighted. Open branches, local-only work, and unattributable additions do not belong in that denominator.

AI Output Share is (AI-assisted Output + agent-authored Output) ÷ total Engineering Output. For human-authored PRs, the attribution percentage splits Output between AI-assisted and human contribution. Agent-authored changes contribute to the agent portion.

AI PR Share counts PRs. These three percentages have different denominators and are not expected to match. More AI contribution does not, by itself, establish higher quality or ROI.

Agent Effectiveness scoring

The composite session score uses six equally weighted skills: prompt clarity, session steering, sophistication, prompt quality, verification discipline, and task outcomes. Each observed dimension is scored from 1 to 5.

Session score = mean of observed dimension scores ÷ 5 × 100

Unobserved dimensions are excluded; they do not become zeroes. The result is rounded to one decimal place. User sentiment may appear alongside skills but does not contribute to the composite. Compare similar work types and inspect the session evidence behind a score.

Benchmarks and sample scores in the marketing demonstrations are illustrative. They are not a published study or a claim about the size or performance of a customer cohort.

Agent Readiness scoring

The assessment uses 84 binary checks in nine categories: Style & Validation, Build System, Testing, Documentation, Dev Environment, Debugging & Observability, Security, Task Discovery, and Product & Analytics.

Checks pass, fail, or are skipped when they cannot be assessed. Skipped checks are excluded from the pass rate. A level clears when at least half its checks were evaluated and at least half of those evaluated checks pass. Every preceding level must also clear; repositories start at Level 1.

The five levels are Baseline, Documented, Agent-ready, Optimized, and Autonomous. They describe repository capabilities rather than a guarantee that an agent can safely deploy without review.

Example: a repository passes 7/8 evaluated Level 1 checks, 9/12 at Level 2, and 5/12 at Level 3, with sufficient coverage at each level. It reaches Level 2 because Level 3 is below the 50% pass threshold.

How to read the dashboard examples

The product pages use illustrative data to explain the views. “Example” labels identify sample dashboards; names, amounts, results, and comparison lines are not live customer results.

Engineering Output per $1K of AI Spend

In the shared four-week illustration, four teams deliver 1,920 Output points and spend $28,400 on AI. The efficiency figure is 1,920 ÷ ($28,400 ÷ 1,000) = 67.6 Output points per $1K, rounded to one decimal place.

The total ratio uses total Output and total spend, rather than an unweighted average of team ratios. These illustrative rollups are assumed inputs; they are not calculated by renaming a PR count.

Estimated net ROI

Estimated net ROI = (estimated capacity value − AI spend) ÷ AI spend
($164,400 − $28,400) ÷ $28,400 = 4.79 ≈ 4.8×

The example assumes 1,096 hours of engineering capacity valued at $150 per hour, or $164,400, over the same four weeks. Subtracting $28,400 of AI spend gives $136,000 of estimated net value. The hours are a separate assumption, not inferred from the Engineering Output score or a reduction in PR cycle time.

For an actual ROI report, document how time savings were estimated, which baseline was used, and how much of the change is attributable to AI. Realized financial savings depend on how capacity is used. Keep billed spend separate from observed usage priced at provider list rates.

Interpreting Router comparisons

A routing comparison needs a named baseline, matching tasks or an explicit cohort design, an observation window, and a cost basis. A comparison against developers’ selected models is different from an all-premium-model estimate.

Report cache-creation and cache-read costs consistently. Identify the quality rubric and evaluation sample alongside cost changes. Savings figures shown in Router examples illustrate these comparisons and do not promise the same result for another team.

Interpreting WorkGraph effort allocation

WorkGraph groups captured activity into work categories and estimates their share of captured attention. Overlapping activity is reconciled before rollups; some durations use estimates or caps.

These shares describe the captured dataset. They do not account for all offline work or every working hour. Compare team trends using consistent connected sources and validate apparent bottlenecks with the team.