Metrics: When the Numbers become the Goal

In general, I’m in favor of development teams gathering metrics. Metrics about your codebase or test execution times can tell you a lot, especially if you track them over time for trending. Metrics about your development process can help you plan more realistically. So metrics, properly used, are a good thing.

But to ensure that metrics are “properly used” it is necessary that they not be disseminated outside the development team. Managers like numbers — they are specific, they are concrete, they look good in reports (especially when they are trending up). But managers just can’t seem to resist using them improperly.

The trouble with metrics is that numbers have no context. A number doesn’t mean anything except in relation to other numbers; metrics don’t mean anything unless you understand how they are derived. Even then, a single metric rarely has much informational value — you usually need several to characterize anything. And once you’ve got multiple metrics, you’ll now have disagreements about their collective interpretation. Which is cause and which is effect?

But even in the rare cases that the development team fully understands the meaning and nuances of a particular metric value, their managers most likely will not.

Inevitably, the numbers will take on a life of their own when they escape the context that created them. The numbers will become the goals of the team, rather than a tool for the team. They become benchmarks and lose all their informative and predictive power.

So use metrics to learn about your code and processes, but hide the numbers!


About jeffkotula

Software engineer, musician, and so on...
This entry was posted in Process, Rant, Software and tagged , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s