We have all quoted metrics such as return on investment (ROI) or percent of labor as means to justify a capital investment for process improvement. The reason for this is metrics attempt to provide objective data that can be readily analyzed. Numeric data readily lends itself to benchmarking comparisons, trend analysis, and so on. Interestingly, what is often overlooked is that throughout the whole process, questions are being raised and communication between the various stakeholders takes place.
What’s Important
The first step is to decide what is important to the organization and this involves dialogue with the primary concerned stakeholders about both issues and goals. Note the use of the word “dialogue.” It is very important that these discussions are truly two-way. Having discussions of this nature with a preset agenda is only likely to foster mistrust and erroneous conclusions. In other words, don’t walk in the room and tell your colleagues how it’s going to be. Doing so is apt to increase the risk of failure because you may very well be unaware of issues that the stakeholders are already aware of.
At any rate, in the case of issues, they need to be identified along with root causes, potential corrective measures, and risks. These factors must be taken into consideration to identify what data elements to collect, how to analyze the data, target scores, etc.
For example, while measuring a symptom/metric such as “lost accounts” is useful as feedback, going after the root causes, such as Defective Parts Per Million (DPPM) and Percent On-time Delivery (POD) are even more important for the simple fact they the cause behind the symptom. Assuming the correct root cause elements are identified, there will be a very strong positive correlation of the relationship between the root causes and the symptom. This can be proven with statistical measures as well as discussions with the various stakeholders. It is always good to couple measures with real-world discussions. It is all too easy to perform analyses that accidentally lead you in the wrong direction. Use interviews and measures as a check and balance against one another.
Goals
Goals are very similar: where does management want to take the organization? These high-level outcomes should be strongly related to the value drivers of the organization. Metric systems need to be developed that track the value drivers relative to organizational performance. Once again, be sure to analyze the relationships and make sure that both the correct metrics are used as well as investigating those metrics really are the true value drivers of the organization. It is possible that what is perceived at one level is not, in fact, what is driving the organization’s success.
Now, let’s stop for a moment and think about this. People are meeting, communicating and identifying the goals and issues of the organization. These types of discussions are not held in a vacuum, or at least, they shouldn’t be. As a result, not only do managers and executives learn what concerns one another, but they also align under what is important and thus what is important enough to measure. These ideas are also communicated, or should be, between management and lower-levels in the organization. Hence, a correctly managed metrics process can send powerful messages throughout the organization. To highlight this further, let’s consider additional metric phases.
Collection – Once what is to be measured has been identified, the metrics project implementation team will need to talk to people at the appropriate levels to get the relevant data in place. It is very possible issues will surface about what is being collected and the collection methods.
There is an important consideration to be learned from activity based costing: do not spend more collecting the data than what the data is worth. When dealing with the groups about the metrics system, involve them early on to determine what is feasible and what is not. Furthermore, their early involvement will not only assist in implementation, but also likely generate stronger models.
Analysis – Collected data must be analyzed and in many cases, there are a variety of statistical methods to highlight results. As the saying goes, “There are lies, damned lies and then there are statistics.” Do not warp the analyses to the point that transparent fictions are being created. What is being suggested is the appropriate analyses methods should be selected for the case at hand. For example, a multiple regression analysis may serve very well to show the relationship between profits and a number of inputs if there is a good relationship amongst the variables.
Reporting – The subsequent reports and timing of their release can send powerful messages. Not only do the stakeholders see what is viewed as important, but they also see changes over time. When properly implemented systemically, this feedback serves to reinforce behaviors by showing cause and effect.
Feedback – The reaction of the organization is very important to gauge. Do they feel the metrics are proper and fair? Typically, metrics evolve over time. The challenges of today will likely not be the challenges of tomorrow. Thus, it is very important to monitor the evolution of the market and feedback from stakeholders to gauge if it is time to evolve the metrics.
Standards – If your organization is struggling with what to measure, then consult quality standards systems and associated support groups. ISO 9000 and Six Sigma have a tremendous body of knowledge you can leverage. Not only do standards systems help you get started, but also useful tools over time. It needs to also be noted that as a result of using standard metrics the organization can benchmark itself against other firms, the market, etc. For example, they can compare their DPPM to what is generally accepted for that type of product in a given market.
The entire process of deciding what is important, what metrics to use, and the metrics process in general all serve to facilitate communication. The process of establishing metrics is an excellent opportunity for an organization to align its various functional units and levels of the organization. Far from being a one time project, metrics and their evolution are a continual process that needs to be managed. Without a doubt, there are many benefits that make this process worthwhile.
George Spafford is a project management consultant and instructor who lives in Saint Joseph, Mich. He has more than 10 years of experience in the fields of information technology and project management. His areas of personal interest include project management, software engineering, and organizational learning. He can be reached at [email protected].