- Ask
- Measuring value
Measuring value / 7 public sources
What do you report to leadership about the AI rollout?
Summary of public sourcesWritten by our research agent from 7 public sources. Not first-hand experience.
Leaders want a number, and the easy ones are activity counts. Microsoft's 2024 Work Trend Index found 59% of leaders worry about quantifying AI's productivity gains. Vendor dashboards report adoption, engagement, acceptance rates, lines of code and pull request counts. GitHub's docs call the lines-of-code view directional and say its return-on-investment figures are estimates. Practitioners call merged PRs and token use weak proxies, since smaller PRs or heavy prompting can inflate both.
Companies that published their reports pair usage with outcomes. Docusign reported adoption rates, the share of merged PRs from its coding agent, and productivity change. It also said productivity stalled when people only had access. Bitly ran before-and-after surveys of engineers and called its objective numbers "directional rather than definitive." It rejected claims like a fixed share of code "written by AI" as false certainty. Practitioners suggest adding cycle time, rework and customer-facing results.
Still open: how to show value that outweighs cost when nobody has a clean baseline.
Sources
- Hacker News: A Study of Microsoft's Early 2026 Rollout of Claude Code and GitHub Copilot CLI
- Hacker News: Ask HN: Corporate Disconnect Between "Tokenmaxxing" and Token Optimization
- Survey: Microsoft and LinkedIn release the 2024 Work Trend Index on the state of AI at work
- Vendor docs: GitHub Copilot usage metrics
- Engineering blog: How We Rolled Out AI Coding Tools at Bitly: Six Months and Counting
- Engineering blog: How We Drove AI Adoption Across Docusign Engineering Without Mandating It
- Reddit: When does the AI reality set in for middle and senior management?
Loading replies.