How the Effective Use of Metrics Can Maximize Return on R&D Investments

By using metrics effectively, laboratory managers can better focus their R&D efforts and be more effective in improving their firms’ sales and profitability. This is essential, now more than ever, given the slow recovery from the “Great Recession.” An American Productivity & Quality Center (APQC) study indicates that the opportunity for improved laboratory performance to increase sales and profits is substantial at many companies. The results summarized in Table 1 indicate that the top 20 percent of the companies surveyed had achieved substantially higher revenues and profits than the overall average for all 105 companies surveyed. These metrics can be used to benchmark and improve the effectiveness of your own firm’s R&D efforts.

The results of this survey indicate that many firms can substantially improve the effectiveness of their new product development programs. Only 44 percent of firms’ product development projects meet their financial objectives, according to the APQC study. Meanwhile, 32 percent of the businesses surveyed rated their new product development (NPD) speed and efficiency as “very poor,” and only 27 percent rated their NPD profitabilityrelative- to-spending as “high.” In addition, 28 percent of businesses do not even measure their NPD results. Dr. Scott J. Edgett, CEO of the Product Development Institute, has gone so far as to say, “New product management is in trouble.”

Effectively defining and using laboratory metrics can improve this situation. Some metrics, and the percentage of companies (surveyed by the Goldense Group) that use them, are presented in Table 1 of Reference 1. They include:

These metrics enable companies to benchmark and judge the overall effectiveness of their product development programs. More detailed metrics can be applied to individual laboratory projects.

Defining R&D metrics

The use of metrics is more suitable for applied research and technical service projects (see below). It is difficult to apply metrics to basic research, in which answers are uncertain and goals evolve during the course of a project.

Simple metrics are useful even if they do not measure all of the laboratory activities on a given project. However, too many or overly complex metrics can become an excessively bureaucratic exercise that reduces productivity rather than increasing it. In addition, spending an excessive amount of time on metrics can be counterproductive. Without a convincing explanation of the need for metrics, laboratory staff members can become cynical about the entire metrics process.

Timely updates of metrics data is essential if the metrics are to drive project planning and execution. If they are not measured in a timely fashion, the use of metrics can become a retrospective process only. However, timely data can be used to take immediate, corrective action rather than just to determine what went wrong after excessive amounts of time, money and effort have been expended on a project. The most commonly used metrics for individual product or manufacturing process development projects involve setting project milestones and completion date targets. The number of staff members working on a project, their skills and the project budget supporting them are the three primary determinants of whether project and performance target milestones will be met. It is essential when designing projects to include milestones that are both important and occur early in the project. These allow prompt remedial action to be taken to ensure that the project proceeds as desired. Remedial action can include terminating projects when failure to achieve milestones indicates that large amounts of additional time, increased staff levels or much additional funding will be required to succeed.

Metrics for outsourcing R&D

For many firms, outsourcing means capitalizing on the capabilities of other organizations in order to achieve their R&D goals. Processes once performed by the “core firm”— concept development, all or part of the R&D and commercialization— can be contracted to other firms if it increases cost effectiveness and speed of commercialization.

Wayne Mackey, principal, Product Development Consulting, Inc., offers a hierarchy of partnership types. The most basic type is what he terms the “directive” partnership, in which a contractor builds or develops to the contracting organization’s specifications. All decisions at all levels reside with the contracting organization. The next level up the hierarchy is the “interactive” partnership, in which the contracting organization specifies the desired outcome of the partnership but not how the contractor achieves this outcome. All decisions reside with the contracting organization. Mackey terms these two categories “outsourcing.”

Outsourcing relationships are easier to manage than the two types of co-development relationships that are further up in the hierarchy. The first is “collaborative,” in which the contractor and contracting organization jointly design the project and split its ownership. One party still receives a fee from the other. At the top of the hierarchy is “risk-reward sharing,” which incorporates the collaborative relationship but both parties also share the financial risks and rewards. Co-development partnerships often evolve into the contracting organization taking an equity stake in the contractor or even acquiring it. This has often been the case in co-development projects between large pharmaceutical firms and relatively small biotechnology contractors.

Mackey recommends that a “partnership coordination council” be established at the beginning of a co-development project, and that it be defined in the contract between the two organizations. For example, the partnership coordination council would include one technical manager and one project representative from each partner. The technical managers are concerned with resource allocation and strategy. This includes intellectual property concerns, information flow and other technical issues. The project representatives are concerned with the cost and schedule of the project. The council defines the metrics to be used in the project.

As noted in the second paragraph of this article, Dr. Edgett notes that an effective and profitable R&D process achieves a good balance between the number of projects and the available resources. This can be considered a ratio, with the number of projects in the numerator and, in the denominator, weighted factors representing the number of R&D staff members, their skills and the facilities available (e.g., number of laboratories, number of hoods and amount of specialized instruments and equipment).

To improve this ratio and add innovative ideas and technology, lab managers increasingly rely on outsourcing and co-development. These programs require the preparation and management of R&D contracts between the organizations. Lab managers should be involved in setting the terms of these contracts, particularly the metrics used to set performance standards and define success.

Factors to consider in outsourcing and co-development contracts include the obvious one—stating what your organization needs. This can be more difficult than it sounds because your organization may have competing demands. As part of the negotiations, both parties must clearly define the innovation strategy and how goals will be achieved. The level and detail of information required from the contractor as the project proceeds, and how this information is to be delivered, should also be specified. Determining how well an outsourcing or codevelopment partner fulfills a contract requires metrics.

The primary metrics are milestones and the dates of their achievement. Performance factors such as process yield and product performance can be made a part of these milestones, as well as economic factors such as project spending for each milestone. Incentive payments may be written into contracts as rewards for achieving milestones in a timely fashion. Milestones, and the metrics for their successful achievement, must be clearly defined and communicated to project participants.

Edgett notes that companies are now doing more than writing milestones and timing into their outsourcing R&D contracts. They are defining specific metrics to be used in determining project progress and milestone achievement. These need to be defined very early in the co-development project. However, one can “kill innovation by putting too much rigor into metrics at the front end,” Edgett warns.

Conserving laboratory resources

Outsourcing and judicious selection of new product development projects are important ways to make better use of your laboratory’s resources. Lab managers should guard against pursuing too many projects, resulting in consumption of too much lab time and staff being spread too thin. Another concern is undertaking projects of low value to the company. Managers must use metrics to winnow out these marginal projects; metrics that could be used include anticipated revenues, profits and return on investment should the project be successful. The estimated likelihood of a project being successful is another factor to consider. One can design an equation using weighting factors to assign values to projects and employ spreadsheets to rank projects.

Another way to conserve resources for the most important projects is to design projects so that key decisions (for example, to proceed with the project or terminate it) are made as early as possible in the life of the project, before large sums have been expended. A good time to make these decisions is when the project hits early, major milestones. Answers to the following questions can help determine whether a project meets the company’s metrics and will be allowed to proceed:

• Does reevaluation indicate whether project success will have a substantial impact on the company’s bottom line?
• Does the manner in which the project has proceeded indicate it still has a chance of success that is consistent with the company’s requirement?
• Is the project achieving milestones on schedule and on budget?

It is essential to have the right senior-level staff conduct the reviews that will result in making these key decisions. People working directly on projects often have an emotional stake in them that can affect decision making.

One also must manage the department or laboratory portfolio of projects in its entirety to be sure it has the appropriate mix of projects to satisfy the company’s metrics while meeting strategic priorities and goals.

Technical service work

Technical service work is an important component of the workload at many laboratories. It is easy for lab managers to agree to do technical service work, even though it may not justify itself in terms of customer retention or additional sales to new or existing customers. A parameter must be used to define appropriate technical service spending for each customer. When spending reaches this level, lab managers have to consider carefully whether additional work is justified. If it is not, one runs the risk of alienating a customer. An alternative is to request that the customer pay for technical service work beyond the limiting metric set by the lab manager.

This metric can be established by setting technical service spending as a percentage of sales or profits derived from each customer. Reviewing lab records from previous years can provide a historical record of technical service spending for each customer and whether business retention or a sales increase resulted. A sharp increase in technical service work can provide justification for asking the customer to contribute to technical service costs. On the other hand, a major customer’s low level of technical service requests in previous years may serve as justification for not using a standard metric to limit spending on a technical service project.

Another strategy to justify spending beyond a set metric on a technical service project is obtaining the customer’s agreement to let your firm use the data in a technical service bulletin or conference presentation. Customers will often require that their names be kept confidential. These bulletins and presentations can be useful sales tools.

Technical bulletins and conference presentations

Technical bulletins, conference presentations and trade magazine articles can serve as other metrics to measure the lab’s creativity and effectiveness in promoting additional product sales. In many cases, laboratory managers review their firm’s needs and make plans to do the laboratory work needed to prepare specific deliverables in these areas in the upcoming fiscal year.

The dark side of metrics

Time is one of the lab manager’s most important resources. However, an overemphasis on time-based metrics can result in an excessive focus on short-term projects, steering managers and staff members away from longterm projects that drive the major innovations essential to the long-term health and profitability of the business. In particular, time-to-market metrics, which should measure time from new product or process conception to commercialization, can result in an excessive focus on short-term projects that, while improving revenues and increasing profits, do not have a major effect on the company’s business.

An overemphasis on inappropriate metrics in other parts of the firm also can have a negative effect on the laboratory manager’s ability to be effective in conducting the R&D and technical service work the company needs. For example, too much focus on quarterly results and "making the numbers" can result in laboratory budget cuts at some firms. This can starve laboratories of the resources—people, instrumentation, other equipment and outsourcing capability—that lab managers need to innovate successfully. Overemphasis on reducing staffing levels, and their associated costs, can reduce the ability of laboratories to carry out the thoughtful, often time-consuming work that is required for major innovation and effective new product development.

Wrap-up

Simple metrics are useful even if they do not measure all of the project activities. One should be careful of having too many metrics and of turning metrics into an overly bureaucratic exercise. Finally, timely updates of metric data are essential if the metrics are to drive project planning and progress.

References

1. J.K. Borchardt,“Using Research Metrics Helps Get More Bang for Your R&D Buck,” Lab Manager Magazine (April 2007) www.labmanager.com/articles.asp?ID=67.

Dr. John K. Borchardt is a consultant and technical writer. The author of the book Career Management for Scientists and Engineers, he often writes on career-related subjects. He can be reached at jkborchardt@hotmail.com.