Measuring Adoption Without Making It a Compliance Exercise
Usage counts measure activity, not impact, and create perverse incentives (open Copilot, ignore output, metric goes up).
Lesson 5
Stories beat usage counts.
Track what matters: behavior change, work quality, time on high-judgment work, capability to explain and fix outputs.
Measuring adoption
Core principles
- Story bank: specific accounts in the voice of the person, "15 minutes on summary plus 20 verifying vs 90 reading" beats a percentage.
- Qualitative signals: reaching for AI earlier, sharing bad outputs openly, iterating prompts, personal workflows.
- Avoid making metrics the goal, report usage as baseline if required, pair with stories and observed behavior.
- One agent, one colleague, one feedback round, or for managers: one experiment, one debrief, one story in the bank.
Check yourself
Why does the lesson argue that usage counts are a poor measure of AI adoption?
A person can open Copilot and ignore the output, metric goes up, value stays zero. Usage counts also create the wrong incentive: use it more, not use it better. Behavior change, quality, and time on high-judgment work actually signal adoption.
Do this in Copilot
Commit to one team experiment debrief and one story captured this month.
Paste this into Copilot Chat and work through it before moving on.
Capture a adoption story
Help me write a 3–4 sentence story for our team story bank: specific task, what I used Copilot for, time or quality difference, and one honest limitation. Tone: factual, not promotional.
- Story capture
Did you run this in Copilot? Mark complete when you have tried it.
RecordedEdition complete
AI for Managers
Navigate: press j for next lesson, k for previous.