by Jan Scott
About ten years ago, I was the test manager for a large project where a custom-developed application was being written by a Big Six consulting firm. As the project progressed, I began to see that development of the new application was going well except for the interfaces to several important legacy systems. The consultants were reporting success, but no one was talking about the interface problems. I knew I couldnt just contradict the consultants reports without having some facts to back up my position. So I developed a metric to count the transactions that were successfully passed through the interfaces. Like transactions were grouped to fulfill a particular business objective.
I decided that the metric would show the percentage of objectives that had been met and measure the percentage of completion of the interfaces. I explained this metric and began to use it in my weekly reports. The consulting firms management did not accept my metric, saying it was too specialized to be a true representation of progress. However, the consulting firms developers began to work hard to improve the numbers I had established. They were quite excited when the percentage went up. Not only had my metric focused attention on the problem but it had altered the behavior of the developersfor the better!
A classic definition of a metric is a measurable indication of some quantitative aspect of a system or project. Or more simply put by Tom DeMarco, a metric is a number attached to an idea. We typically use metrics to predict what will happen in the future by comparing plans to actual results. For example, we may have a metric that tracks actual hours worked per task against planned hours per task. If we see that actual hours are greater than planned hours, we can predict that the project will finish late. Presumably, this knowledge will allow us to take some corrective action such as reducing project scope or adding people to the project team. Sometimes there is more than one way to measure an attribute. For example, how would we measure the ease-of-use of a newly designed user interface? By the time it takes to train a new user, the number of calls to the help desk, or a questionnaire? Any of these ways will work as long as everyone accepts the metric as a valid representation of the attribute you are trying to measure.
Project managers learn that good metrics should be objective and not influence what you are trying to measure. In DeMarcos book Controlling Software Projects, he says that metrics must be independent of any conscious influence of project personnel. He suggests that if project members know what you are measuring, they will take whatever actions necessary to improve the metric. The results of the metric then will be skewed and no longer valid. In his example, youve determined that the number of smiles on Monday morning is an indication of project morale. If the project team knows this, they will all smile on Monday mornings. On the other hand, DeMarco says that if you keep this metric a secret, then you will get a truer picture of actual morale.
for more on this story: http://www.stickyminds.com/BetterSoftware/magazine.asp?fn=cifea