Recently, I was involved in conducting a strategy session where we defined metrics for measuring progress of achieving the company's objectives based on a balanced scorecard. The responsibilities for main strategy areas were divided among the senior managers with each manager defining metrics for their respective sections. The initial metrics that were developed were quite vague, not specific, and the calculations were not clearly defined, leaving room for different people to interpret the metrics in different ways. However, a round of reviews and resultant improvements based on feedback allowed us to further refine the metrics and align them more toward company objectives.
The success of this exercise will depend on a lot of factors. It is said that companies will abandon 80% of initiated measurement programs within 2 years (2). The main factors for failure are misunderstandings of measurements, incorrect use of measurement data, and disregard for human factors (2). Therefore, measurement programs should be well planned before they're implemented. Goal-Question-Metric (GQM) recommends a goal driven method to implement a metrics program (1).
This introduces 3 levels:
- Conceptual Level (Goal)
- Defines what's required to achieve. This could be related to products, processes, or resources.
- Operational Level (Question)
- Defines questions that characterizes the goals. These questions are used to indicate whether the goals are met.
- Quantitative Level (Metric)
- Metrics are defined to answer the questions in a quantitative way.
References:
- Basili, Victor R.; Caldiera, Gianluigi; Rombach, H. Dieter. "The Goal Question Metric Paradigm,"; Encyclopedia of Software Engineering (Marciniak, J.J., editor), Volume 1, John Wiley & Sons, 1994, pp. 578-583.
- Carol A. Dekkers, Patricia A. McQuaid. "The Dangers of Using Software Metrics to (Mis) Manage," IT Professional. 4.2. Mar/Apr 2002: 24-30
- https://www.goldpractices.com/practices/gqm/
