Lab managers believe that their labs operate at near top efficiency within available resources; if they believed otherwise, they would change the system to make it so. However, this belief is based mostly upon intuition, informal observation, or other qualitative, and often flawed, information—obviously, every lab can’t be a top quartile performer. Good labs employ a variety of quality measures to indicate the state of operations¹ but even these quantitative measures only hint at the true quality of the results, leaving the performance grade subject to interpretation.² This is where benchmarking metrics come in—to provide an external standard for comparison. In common usage, benchmarking and metrics surveys are often used interchangeably, but, strictly speaking, these are quite different processes with different goals. Benchmarking aims to identify and implement global best practices to improve operational performance while metrics surveys measure the operational characteristics of systems for evaluation purposes. Benchmarking typically involves selection of a partner recognized for exceptional excellence in an area of interest and then assembling a team for a site visit for in-depth documentation of the best practices of their model. The partner company may be in the same industry or may be in an unrelated industry that utilizes processes that are similar—the oft cited examples of the latter strategy are Southwest Airlines partnering with Indy pit crews to learn how to rapidly turnaround their planes and Remington Rifle Company partnering with Mabelline (a cosmetics company) to learn how to make its shell casings shiny. Metrics provide feedback on the performance of subsystems within an operational area and are a convenient way to compare models to identify best practices.

When metrics reveal areas that are underperforming, the manager must decide among competing objectives to select the ones with the highest probability of delivering workable solutions. Ideally, the selections will take the lab along the optimum path directly to the best practices, but rarely does this occur. Most labs opt to search for their own solutions which can waste scarce resources, and, in the end, still leave the system sub-optimized. This outcome might be avoided by selecting a willing partner recognized as having the best practice in the area of interest and organizing a benchmarking team to visit, observe, and collect ideas. This requires a considerable time commitment but is more likely to yield a better result than the first approach. A third option is to retain expert consultants who presumably have visited the top performing labs and have assembled an inventory of best practices. This is often the most cost effective and efficient option for focusing resources on the problem but the quality of the outcome is highly dependent on the quality of the consultants. Regardless of which strategy is pursued, benchmark metrics provide the basis for selection of objectives, setting realistic targets, and monitoring progress to insure that the lab advances in concert with comparable labs. Value is derived from integrating these efforts into annual performance contracts or other management systems favored by the lab.

The metrics survey

Ideally, a metrics survey should probe into every aspect of operations to provide feedback that reflects performance of each system in a manner that facilitates comparison between operational models to allow identification of best practices. These measures must be monitored over an appropriate time interval to observe trends and relative rates of change in performance of the different models prior to selecting the superior performers. The systems typically examined by a comprehensive metrics survey instrument are listed in Table 1. Each of these systems will have multiple measures to examine every facet of its operational characteristics.

The second issue in using a benchmarked metrics survey approach is obtaining a statistically significant population of respondents for meaningful segmentation and analysis. The entire population of laboratories in the U.S. is estimated at between 25,000 to over 100,000. The survey organizer must have sufficient resources to reach a large portion of this population in order to segment into statistically significant peer groups. This hurdle can be lowered by applying the concept of differential segmentation—rather than segmenting laboratories based upon total operations, segment based upon similarities of individual systems. For example, safety metrics for a particular lab may be compared to one set of labs with similar safety issues while sample administration metrics for the same lab may be compared to a different set of labs. The key to using this approach is the ability to identify the specific system characteristics that define a particular segment to properly classify each lab.

The other element affecting the quality of a benchmarking metrics study is the report. It should present results in a concise manner that clearly indicates the relative position of the lab with respect to composite results of similar labs. Quantitative results are typically reported as the lab value along with the average, best quartile, and worse quartile composite scores. Charts or graphs that visually depict historical trends over several years are also essential for proper interpretation of the results. In addition, good reports will include comparisons of significant ratios and other combinations of metrics that reveal information on the interactions of laboratory systems as well as non-quantitative comparative data, reported without interpretation, to facilitate subjective evaluation of the maturity of cultural or human relations programs.

Using benchmarking metrics

In referring to metrics, Meyer says “…measures tell an organization where it stands in its effort to achieve goals but not how it got there or, even more important, what it should do differently.” ³ Measures indicate a need for change but offer no clue as to how to adapt current practices or even if a completely different model should be adopted—these answers lie in the benchmarking process. By systematically studying the operations of the laboratories that achieve superior metrics, it is possible to accumulate an inventory of those elements that contribute to their success, (i.e., best practices.) Organizations that are able to identify and implement these practices achieve near best-in-class performance without the inefficiencies inherent in the usual trial-and-error approach. They may then elect to pursue one of two strategies— maintain a follower position by continually monitoring and copying the leaders within their peer segment, or adopt a leadership position by extending the current best practice. Either of these strategies requires some ingenuity since simply copying best practices is not likely to yield the same results in every lab due to organic and cultural differences in operational environment— best-in-class systems must be adapted to accommodate the individuality of the organization without losing the essence of what makes them successful.

Driving improvement and productivity

Most organizations formalize their annual objectives during the last quarter of the current year or the first month of the new year. The actual process has several variations depending upon current management fashion but the gist remains the same. Typically, broad, high level goals and objectives cascade down from executive management becoming more specific and detailed at each level so that individual personal objectives align with the organizational initiatives to insure that everyone works in concert — this is referred to as linkage. Laboratory objectives derived from this process generally fall into five categories:

• development of new services needed by the organization,
• cost reductions,
• improved system efficiency,
• cultural issues,
• and work safety initiatives. 

Benchmarking metrics can play an integral role in selecting objectives in each of these areas, defining reasonable targets, and pointing to potential roadblocks or issues requiring management attention.

New services

Benchmarking metrics reflect the state of lab operations, give an indication of organizational readiness to take on new work, and hint at the likelihood of success. For example, if a particular lab lags its peer group in productivity measures, it is an indication of unused capacity that might be reclaimed through an improvement initiative to provide the resources to take on new work. Likewise, if a lab is leading in productivity measures, it may be an indication that current resources are fully utilized so that additional resources will be needed to take on new work. Other measures give an indication of the state of readiness of the instruments, need for additional capacity, or need for capital replacement. In all cases, benchmarking metrics alert the lab manager to the preferred areas to focus attention to assist the staff in completing their objectives and provide a basis for evaluating the probable success of each objective.

Cost reductions and improved system efficiency

Benchmarking metrics are an especially fertile source for gleaning ideas for cost savings initiatives. Underperforming operations usually imply excess costs and are opportunities to eliminate waste or to adopt a new system—and even the best labs are almost certain to have a few of these opportunities. Comparison with the best-in-class standard provides an estimate of the savings potential associated with each low performing area. Balancing savings potential with the effort and resources required for improvement yields net benefit for each project which can be displayed in a Pareto chart to evaluate and prioritize according to economic viability. The evaluation process should also compare the cost of purchasing external solutions with the opportunity cost incurred in using internal resources, (i.e., the value of alternative work that could be accomplished with the resources).

Cultural issues

The effect of cultural issues on laboratory productivity cannot be overstated. The cooperation and support of the staff is the most critical element in meeting department goals and achieving excellent performance.⁴ Lab managers typically try to build a positive culture through various programs to improve employee satisfaction and increase their involvement in the business. Benchmarking metrics can examine the range of these programs compared to the peer group but cannot judge their effectiveness—this requires polling the opinions of the staff. However, labs may use benchmarking metrics to judge whether they have the right balance of programs to address cultural and satisfaction issues in comparison to their peer group.

Safety initiatives

The most important area to benchmark is laboratory safety. This responsibility is mandated by both regulatory requirements and moral imperative. The benchmarking survey examines legal compliance activities to identify deficiencies that might lead to enforcement actions and, more importantly, compares the scope of an organization’s efforts to fulfill its moral obligation to protect the workers from harm with current industry best practices. Both types of measures are important in managing risk—organizations are subject to fines or prosecution for non-compliance and face substantial civil liability for failure to maintain a safe workplace. More importantly, striving for a world class safety program is simply the right thing to do.

Conclusion

Trends drawn from annual metrics studies tell laboratories whether they are lagging, matching, or surpassing the improvement rates of their peers which directly measures performance. This information is invaluable in celebrating successes or rousing the staff from complacency. Trending over multiple years is the best way to get an accurate assessment of performance and to insure that the laboratory maintains progress comparable to its peer group. Externally benchmarked metrics finally provide a reliable way for lab managers to grade their laboratory’s performance and are an invaluable data feed into the performance management system.

References

1. L.W. Collins, “Laboratory Quality Measures Revisited.” Managing the Modern Laboratory, 1(3), 1995, 89.
2. L.W. Collins, “Grading Laboratory Performance.” Managing the Modern Laboratory, 5(4), 2001, 77-80.
3. Meyer, C., “How the Right Measures Help Teams Excel.” Harvard Business Review, 1994, May-June, 95-103.
4. L.W. Collins, “Managing Change.” Managing the Modern Laboratory, 7(3), 2005, 54A-56A.

Dr. Wayne Collins, Informatics & Services Division, Thermo Electron Corporation, 1410 Gillingham Lane, Sugar Land, TX 77478-2890; (713) 272-2282; (713) 272-2272; Wayne.Collins@thermo.com.