Why PR counts feel like progress but often aren’t
Teams reach for pull request metrics, commit counts and ticket throughput for a simple reason: they’re easy to count. “ That makes the numbers feel clean, tidy, and reassuring. If the graph’s going up, surely the team’s doing well, right?
Well, maybe. Or maybe the team is just generating more visible motion.
That’s the trap with a lot of engineering metrics. They describe activity, not quality. A busy repo can belong to a team shipping useful work, but it can just as easily belong to a team splitting the same change into smaller chunks, reopening PRs after review churn, or churning out small edits that don’t move the product much at all. The dashboard doesn’t know the difference. It just counts bodies in motion.
AI assistants make this problem easier to miss. A developer can ask a tool to draft a function, rewrite a test, or scaffold a patch, then spend the rest of the afternoon cleaning it up. The result might be a higher commit count or more PRs, but the underlying change could still be modest. In some cases, the assistant generates code that’s duplicated, thinly adapted, or only partially correct, which means the human work shifts from implementation to verification and repair. That’s still work, of course, but it doesn’t map neatly to “more PRs equals better engineering.”
A dashboard full of activity can still describe a team that moved a lot and changed very little.
That distinction matters because a lively-looking dashboard can hide awkward truths. A team might be shipping many small changes while spending just as much time on rework. It might merge more frequently while also creating more follow-up fixes. It might close more tickets because the tickets were sliced thinner, not because the system improved. None of that shows up if the only question is, “Did the count go up?”
This is why treating pull request metrics as a proxy for engineering quality gets people into trouble. They’re convenient, sure, and they’re also seductive. Numbers that are easy to measure tend to get promoted into status, even when they only describe motion. People start improving the scoreboard instead of the outcome and the scoreboard obliges by rewarding whatever is easiest to inflate, once that happens.
The fix isn’t to throw away measurement. It’s to stop pretending every metric answers the same question. Activity tells you how work is flowing through the system. Outcome tells you whether that work actually helped. Keep those separate, and the dashboard gets a lot less smug. Mix them together, and you end up celebrating speed without knowing whether anything worthwhile happened.
Activity metrics: useful for flow, not for quality
Once you stop treating PR count like a score, the next question’s more useful: what do these numbers actually tell you? On their own, PR count, commit count and ticket throughput are flow signals. They show how work moves through a system. They can reveal a queue that’s backing up, a review process that drags, a handoff that keeps stalling, or a team that’s slicing work too finely because one stage keeps slowing everything down.
That makes them useful. Just not in the way a lot of dashboards pretend.
Google’s 2024 DORA report and the broader State of DevOps research are decent reminders of this split. The useful metrics there are about delivery flow and system health, not a trophy case for whoever shipped the most stuff. Even when the numbers look tidy, the real question is whether they help you understand where work is moving cleanly and where it gets stuck. That’s a developer productivity question. It is not, by itself, a software engineering quality verdict.
Activity metrics tell you where work is moving. They do not tell you whether that movement produced anything worth keeping.
A team with high PR volume might be moving quickly, or it might be burning time in review. Those are very different states, even if the dashboard looks lively in both cases. A surge in ticket throughput could mean fewer blockers. It could also mean the team split one meaningful task into six smaller tickets so the board looks healthier. A pile of commits might show disciplined progress, or it might show an overactive formatter, a bot rewriting generated files, or a developer fighting an awkward repo setup one tiny patch at a time.
AI makes this fuzzier. An assistant can produce boilerplate, test scaffolding, and repetitive refactors at a pace no human can match. That can be useful when the generated code’s reviewed, trimmed and folded into a real change. It can also inflate activity without adding much substance. A PR that once took a day to assemble may now arrive in an hour, but if three-quarters of it’s generated code and copied patterns from elsewhere, the count went up while the underlying engineering effort stayed roughly the same. The dashboard sees motion. It’s no clue whether that motion matters.
The same thing happens when teams split work to make the numbers look neat. A feature that should’ve been reviewed as one coherent change gets cut into five PRs, each with one tiny slice. Sure, the PR count rises. So does the review overhead. So does the chance that one slice breaks another. The team might even celebrate a faster merge rate while quietly increasing rework. Anyone who has spent an afternoon untangling “small” changes knows the joke: tiny PRs can still make a giant mess.
This is why activity metrics need context. A large PR volume with heavy rework does not say much about correctness. Many small commits do not prove maintainability. Fast ticket movement does not guarantee customer value. The numbers are still real, but the story they tell is narrow. They are good at answering questions like, “Where is work piling up?” and “Which stage is slowing us down?” They are bad at answering, “Was this the right work?” or “Did the change improve anything users care about?”
If your team uses these signals well, they can help with reviews, planning and incident follow-up. You can spot a branch that sits open for days and ask whether review capacity is the problem. And you can notice a spike in tiny tickets and ask whether the team’s hiding complexity instead of managing it. You can also catch tooling issues, like automated diffs that flood the repo with noise. That’s all useful. It just lives on the flow side of the fence.
The line is simple enough, even if dashboards love to blur it: activity metrics tell you how busy the system is. They don’t tell you whether the work improved the product, the service, or the codebase. That part comes next.
Outcome metrics: prove the work mattered
Once you stop treating PR counts and commit count as proof of quality, the next question is obvious: what should you measure instead? The answer lives on the other side of the deploy button. Outcome metrics tell you whether the work changed something users, systems, or the business can actually feel.
Because of this, that usually means looking for fewer defects, fewer incidents, faster recovery, lower latency, better conversion, or higher retention. Those are not vanity numbers. They connect engineering choices to observable change. That’s a very different story from a release that trims crash reports by 30 percent or cuts checkout latency enough to lift conversion, if a release ships five polished PRs and the bug rate stays flat. Same amount of coding, very different result.
If a metric never changes when the product gets better or worse, it’s probably measuring effort, not impact.
A decent dashboard needs more than one type of outcome, because teams rarely work on one dimension alone. Reliability work should care about incident rate, mean time to restore service, and perhaps service error rates. Product work might care about activation, conversion, or task completion. Internal platform work may track adoption by other teams, build times, or reduction in support tickets. The point is to choose the thing that actually changes when the work succeeds.
Google’s DORA research has pushed teams toward operational outcomes for years, especially metrics like change failure rate and time to restore service, because they say something about the health of delivery and operations rather than just the amount of activity (DORA). That framing matters. A high commit rate can mean the team is busy. A lower failure rate after a release tells you the change did what it was supposed to do.
The same idea shows up in SRE practice. The 2022 State of DevOps work tied reliability to service-level objectives and error budgets, which gives teams a concrete way to judge whether shipping more code is helping or hurting the service (SRE in the 2022 State of DevOps report). That’s a much cleaner signal than staring at the PR queue and hoping it has opinions.
Lagging indicators matter here, even if they’re a little annoying. Engineers love immediate feedback. Point taken. Dashboards love immediate feedback. Customers, sadly, don’t care about our preference for instant gratification. The real effect of a change often shows up later, after traffic ramps up, after edge cases hit production, or after users have had time to either adopt the feature or ignore it. You can miss the actual result, if you only measure what happened during the sprint.
That lag is why outcome metrics should be chosen with the team’s mission in mind. An systems team should probably care more about uptime, recovery time and latency than about how many tickets got closed. A growth team should care about conversion, signup completion, or retention on the flow they changed. A payments team might watch authorization success and failed checkout recovery. If the metric doesn’t move when the system improves, it’s a vanity stat wearing a fake mustache.
The practical test is simple. Ask, “What would get better for the user or the business if this work worked?” If you can answer that cleanly, you’ve found an outcome metric. If you can’t, you may still have a useful activity metric, but you do not yet have proof that the work mattered.
Build a dashboard that pairs motion with impact
By the time a team reaches this point, the pattern’s usually obvious: the activity numbers are easy to collect, easy to praise and easy to game. A dashboard full of PR counts, commit counts, and ticket throughput can look tidy while saying very little about whether the work changed anything users care about. That’s the problem to fix.
The cleanest way to do it is to split the dashboard into two buckets. The first bucket tracks activity. It answers, “How is work moving?” The second bucket tracks outcome. It answers, “Did that work actually improve something?” Once those buckets sit side by side, the dashboard starts acting like a decision tool instead of a scoreboard for whoever opened the most pull requests before lunch.
If a metric can’t be tied to a real change in the product or system, it’s describing motion, not progress.
The pairing doesn’t need to be fancy. In fact, the simpler it is, the more likely people will use it. A team that tracks PR count should also track at least one result that the PRs are supposed to influence. If PR volume rises, did escaped defects fall? Did review cycles get shorter without piling up rework? Did release frequency improve without sending incident rates through the roof? Those are the questions that matter. The raw count alone can’t answer them.
The same logic applies to ticket throughput. A board that shows more tickets closed might look healthy, but it can also hide a mess of broken work items, duplicate tickets, and tiny slices created only to make the board look busy. Pair that number with a result metric that reflects actual system change. For product teams, that might be conversion rate, activation, or support volume. For platform teams, it might be change failure rate, recovery time, or latency. For a reliability team, the Four Keys model from Google Cloud gives a useful pattern: deployment frequency and lead time belong next to failure rate and mean time to recovery, because output without stability is a half story.
That pairing also helps when AI coding assistants are in the mix. A team can ship more PRs with less human typing and still produce the same amount of useful change. Sometimes the code is solid. Sometimes it’s a pile of repetitive edits with a cheerful commit message. The point is not to moralize about the tool. It’s to stop confusing easier production of code with better engineering. If AI assistants double PR volume but defect rate stays flat, or worse, climbs, the dashboard should make that obvious. And if ticket throughput rises because people are splitting work into smaller and smaller pieces just to show movement, the outcome column should expose the lack of progress.
Microsoft Research’s work on developer productivity is useful here because it treats productivity as more than raw output. Their paper, The Space of Developer Productivity: There’s More to It Than You Think, is a reminder that local activity and real impact are different things. A team can be busy and still be stuck. A dashboard that only counts visible work tends to miss that.
In practice, you want each activity metric to have a companion outcome metric and a question attached to it. A few examples make this easier to keep straight:
PR count plus escaped defects. Are we shipping cleaner changes, or just more changes?
Review time plus rework rate. Are reviews getting faster because the code is better structured, or because people are rubber-stamping?
Ticket throughput plus customer-reported issues. Are we closing work faster, or creating more cleanup later?
Commit count plus incident rate. Are smaller commits improving safety, or are they just the byproduct of a tool doing half the typing?
Next up, you don’t need a wall of charts. That usually becomes dashboard wallpaper, which is great for office decoration and terrible for decision-making. Three to five paired metrics are usually enough for a team. Keep them visible in retrospectives, planning and incident reviews. If a release looks fast, check whether quality held steady. Ask whether the team spent too much time on process churn to get there, if quality improved. And if a metric moves and nobody can explain the business or system change behind it, that metric probably belongs in the activity bucket, not the success bucket.
A good dashboard should create friction in the right places. It should make people pause before celebrating throughput, and it should make them ask whether the change mattered. The numbers stop flattering the team and start teaching it something useful, when those questions become normal.
Measure motion, manage impact
If your dashboard rewards whoever can generate the most pull requests, commits, or tickets, you’ve built a scoreboard for motion, not quality. That’s the trap. The numbers move, people feel productive and the graph goes up and to the right. With AI assistants in the mix, those numbers can climb even faster while the underlying engineering change stays modest, gets duplicated, or arrives with a fresh pile of cleanup work attached.
That’s why PR counts have to be treated with care. A large pile of merged branches might mean the team’s shipping well. It might also mean the team’s splitting work into tiny slices to make throughput look healthy, or that an assistant’s drafting code faster than anyone can review it properly. Commits can be even noisier. Ticket counts can be gamed by turning one real task into five smaller ones. None of that tells you whether the product got better.
A dashboard full of motion can still hide a team that didn’t move the product very far.
Activity metrics still earn their keep. They tell you whether work is flowing, where things back up, and whether a team’s stuck waiting on review, design, QA, or a decision nobody wants to make. Used that way, PR counts and commit volume are useful signals. They can help you spot friction. And they can also show when a process change reduced handoff pain or when a release train started to speed up.
What they can’t do is prove value by themselves.
That’s the line worth keeping in your head. If a metric can’t be tied to an outcome that matters, it shouldn’t be treated as a success signal. A high merge rate means very little unless it connects to fewer defects, faster recovery, lower latency, better conversion, happier users, or whatever result your team actually cares about. If the dashboard can’t make that connection, it’s measuring movement and calling it progress. That’s how teams end up celebrating busy calendars and ignoring stale product problems.
The cleanest rule is simple: use activity metrics to watch flow, and use outcome metrics to judge impact. When the two move together, great. You’ve learned something useful, when they drift apart. Maybe the process’s noisy. Maybe the team is shipping a lot of low-value changes. Maybe AI is making it easier to produce code than to produce results. Whatever the cause, the dashboard should point at it instead of dressing it up.
A better dashboard does a quiet job. It cuts down the theater, keeps the conversation grounded and gives teams fewer reasons to argue about vanity numbers. Quick aside. More to the point, it helps people ship with a clearer view of what changed, why it changed and whether anybody actually benefited. That’s a much better use of screen space than counting motion for sport.
