In 1999, the Centers for Disease Control and Prevention (CDC) entered into the Public Health Preparedness and Response for Bioterrorism Cooperative Agreement (BCA) with 62 jurisdictions in the United States. Since the early days of the BCA, CDC has been working with the jurisdictions’ public health labs (PHLs) to improve the public health response to a large-scale chemical exposure incident. Partly by plan and partly by trial and error, a network has been created in which different members contribute to the program based upon their interests and abilities, which benefits everyone in the network. Over the past five years, the CDC and its partners have not only created a functional chemical incident response network but have learned a number of important lessons.

In new undertakings, processes rarely proceed as initially planned. Identifying those aspects of a new program that are essentially correct and those that need to be modified is critical to the evaluation of that undertaking. Successful programs discover how processes can be improved and adapt. During the creation and implementation of the Laboratory Response Network–Chemical (LRN-C), an offshoot of the larger LRN and an initiative between the CDC and state PHLs to expand the PHLs’ chemical terrorism response capability, many lessons were learned. The most important lessons will be explored below and include the following:

1. Money can’t buy happiness.
2. If you can’t do it, you can’t teach it.
3. Use critical resources for critical tasks.
4. 24/7 emergency contacts don’t work 24/7.
5. Being dogmatic is for the dogs.
6. Partners are necessary for networks.

Money can't buy happiness

Everything takes longer and is more difficult than expected
Adequate funding is critical for creating laboratory facilities and for stocking them with instrumentation, equipment, and supplies. Funding is also vital for paying laboratory employees’ salaries. However, adequate funding is not always sufficient for obtaining qualified, motivated staff.

Hiring laboratory staff presents many challenges, including policy restrictions on the number of staff, availability of qualified personnel, and local pay scales. For the LRN-C, the funding available through the BCA is “soft money,” dependent on ongoing funding for the continuation of the specific project. This is in contrast with permanent civil service positions in government agencies. To staff a special program, a government agency must often make the critical decision to either allocate existing staff to the program or to hire additional staff to meet the terms of the agreement and the needs of the program. An alternative to increasing the number of staff is to use contractors; this practice does not have the long-term commitment associated with hiring government staff positions but may cost significantly more.

Hiring either civil service staff or contractors requires people who are available and have the requisite education and technical expertise. Low government pay scales add to the difficulty in finding qualified and interested people. When the LRN-C expanded from five to 62 labs, approximately 40 Ph.D.-level, or equivalent, analytical chemists with the necessary skills and experience — and the willingness to work for salaries well below the local private sector pay scale — were needed immediately. In many cases, these staff positions were filled by people graduating from undergraduate or graduate schools who could not find private sector positions because they lacked practical experience. Often, these staff members would gain experience by working for the LRN-C for a year or two and then would leave for more lucrative employment in the private sector. This created a significant turnover in PHL staff. Ironically, the LRN-C, a program with adequate funding to have completely staffed all labs in the network on day 1, has, years later, the equivalent of almost one opening per laboratory.

Hiring laboratory staff, even if it is authorized, is not often easy to do. Especially for civil service positions, the length of time between obtaining the authorization to hire and the actual arrival of the person onsite can be very long. As frustrating as this delay is for the local jurisdiction, it also adds a layer of complexity to the central coordinating entity’s work. For example, the transfer of the first Level 2 analysis method, cyanide in blood, from the CDC to the PHLs took almost two years as labs slowly staffed to the required levels. This delay required that the CDC maintain a training schedule flexible enough to train PHLs on both old and new analysis methods at any time.

In addition, calibration materials used by the first labs that were trained had reached and exceeded their expiration dates before the last lab was trained. As a result, new batches of materials and the overhead costs associated with validating and characterizing these materials had to be assumed. On a regular basis, two to three batches of materials were needed for each method transferred. Coordinating the various aspects of the program that are highly dependent upon each other has been particularly challenging.

If you can't do it, you can't teach it

But just because you can do it doesn’t mean you can teach it
Effectively transferring analysis methods from the CDC to the public health labs was identified as the CDC’s Technology Transfer Laboratory’s key activity. CDC technical subject matter experts (SMEs) would train the PHL staffs. However, because an excellent scientist is not always a skilled developer of training materials or an effective instructor, people with adult education experience and finely honed technical skills were hired as the program’s primary instructors. All the primary instructors had earned post-graduate degrees in chemistry and had taught in high school, college, and/or industrial settings. These instructors had a combined 100 years of professional experience.

The training program’s desired outcome was that students could perform a specific analysis with adequate accuracy and precision to produce interpretable results. Ideally, after training, the students should also understand the science underlying the analysis, but, for the short term, the student being able to perform the specific analysis was sufficient for training purposes.

The sample preparation, instrumental analysis, and data reduction techniques used for analyzing chemical warfare agents (CWAs) were commonly employed by a large segment of the chemical analysis community, but the integration of the parts and the operational aspects of the specific instrument system and controlling software required hands-on training. Because the students needed hands-on experience, the training program provided a low student-to-instrument ratio so that students had maximum direct experience with the analysis. Similarly, the program employed a 2:1 student-to-faculty ratio for the hands-on parts of the course, so that each of the two teams of two students would have one instrument and one instructor at its disposal.

The training team leader was one of the experienced post-graduate-trained staff members and was assisted by a cadre of lab instructors, who were bachelors- and masters-level staff members with about five years of hands-on experience. The lead instructor was responsible for the theoretical aspects of the training, and the lab instructors took primary responsibility for the practical aspects. With time and additional experience the lab instructors, like graduate students in a university setting, developed their skills and expanded their teaching beyond the practical aspects of the analysis methods.

Because the course developers and deliverers needed to be qualified and experienced scientists, the instructors could not engage in training activities 100% of their time and still retain their technical skills and credibility. After some trial and error, the program targeted the distribution of the instructors’ time, after accounting for overhead activities, as 35% for course delivery, 35% for course development, and 30% for technical/professional development. Instead of having high-level skills in the broad areas of clinical chemical analysis and adult education, the instructors had to be subject matter experts for each analysis that they would be transferring. Thus, the 35% of their time allocated for course development included developing proficiency with a specific analysis and developing course materials. The 30% of their time targeted for technical/professional development was for growth in areas that expanded the network’s capabilities, such as analytical method development; instructors were also encouraged to present their work at national professional meetings and in peer-reviewed journals. Allowing the instructors to continue their personal development has both maintained their credibility with the student population and created a stable instructor corps.

Use critical resources for critical tasks

Don’t do what others can do
With most programs, there are tasks that only the program can perform, tasks that the program would like to perform but other entities can perform almost as well, and tasks that the program does not want to perform and other entities already perform well. A program has to determine how best to use other entities to perform those tasks that, because of staffing or other limitations, it cannot perform and that would jeopardize its mission if not completed.

In the LRN-C’s case, the network was able to leverage existing external resources in instrument operation training and in the production of calibration and proficiency testing (PT) materials. Most instrument vendors provide instrument operation training to support their customer base. If a standard instrument operation course met the needs of the LRN-C training program, such as elemental analysis using the PerkinElmer ELAN DRC, that course and its delivery at the PHLs were included in the instrument purchase contract.

However, not all standard operation courses transferred so easily to the PHLs. For example, the gas chromatography– mass spectrometry (GC-MS) and liquid chromatography–tandem MS (LC-MS/MS) programs were more complicated and included multiple layers of training. Standard instrument operation training would not be adequate to develop proficiency in GC-MS and LC-MS/MS due to the added complexity of the analysis techniques. The hardware/software training provided by instrument manufacturers did not, and could not, adequately address the science associated with these techniques. The collaboration among CDC SMEs, who were developing training classes designed to reinforce operational skills and also to explore the full capabilities of the instruments, and the delivery of these classes by selected technical support staff of Agilent Technologies or Applied Biosystems provided the PHLs with instrument understanding beyond that gained in vendors’ operator training classes.

The LRN-C’s proficiency testing program initially included the production of calibration, quality control (QC), and PT materials in sufficient quantities to last at least one year and hopefully longer. To accommodate methods that had a throughput of approximately 25 samples per day, an ideal materials stockpile that would address a 10,000-sample incident, would require 400 vials for each level of a seven-level calibration for each validated method. In addition, generating the thousands of vials that would be required for each lab to characterize QC materials and PT challenge materials would easily exceed the capacity of the PT staff. The solution was to identify vendors who were already preparing standard solutions, commonly for environmental testing, and who were willing to expand their programs into clinical matrices. As the network expanded to more than 40 labs, each performing multiple methods, the use of external resources for materials became the only viable option and an LRN-C materials program was created. Vendors could prepare the samples in flame-sealed vials and also stock, sell, and ship the materials to PHLs that had completed their analysis method training and were ready for validation.

24/7 emergency contacts don't work 24/7

Or in emergencies
The LRN-C was chartered as an emergency response network. Therefore, when an emergency arises, CWA samples must be directed to the right people and those people must perform the analysis in time to meet response needs. To evaluate the responsiveness of the network, the LRN-C created Emergency Response Exercises. At the start of these exercises, an LRN-C quality assurance (QA) staff member would call the PHL 24/7 emergency contact listed on the LRN-C website and advise that contact that samples would be arriving at his or her laboratory by priority, next-day delivery. These instructions provided the PHL with approximately 24-hours notice. When the samples arrived, the PHL had to analyze them for one specific analyte and report the results as soon as possible through the LRN-C website reporting mechanism.

Although PHLs were given repeated warnings that a training exercise was starting on a specified date, the results of the first Emergency Response Exercise were more enlightening than satisfying. The LRN-C placed calls to ten PHL emergency response numbers and requested to be connected with the person responsible for chemical terrorism response at each PHL; in seven instances, the LRN-C found no connection between the emergency response number and the person the network was trying to reach. In contrast, one laboratory emergency contact mechanism worked flawlessly, and that laboratory reported its accurate results less than four hours after the arrival of the samples. When the results of this first exercise were reported to the network by teleconference, performance was dramatically improved for all subsequent emergency response exercises.

Being dogmatic is for the dogs

If you don’t bend you break
If it is to be successful, a lab network must be allowed flexibility in evolving its processes and activities. Flexibility was critical to the LRN-C’s performance in supporting method development activities at Level 1 labs, in supporting Level 1 analysis methods for Level 2 labs with the relevant instrumentation, and in evolving the QA program with PT materials and exercises.

The staffs at Level 1 labs are highly qualified analytical chemists with access to all program instrumentation. In the early days of the program, these chemists had many weeks where their time was not directly allocated to network activities. Because of the plethora of toxic industrial chemicals that might be used as terrorist weapons or that might be involved in large-scale industrial accidents, the LRN-C offered the Level 1 labs the opportunity to help develop analysis methods needed by the network for these chemicals. Each of the five Level 1 labs selected five of the approximately 25 high-priority analytes and began work. The CDC provided technical support to help with any analytical issues and financial support to finance the custom synthesis of metabolites and stable isotope-labeled internal standards. As a result of this mini-program, additional GC/MS and LC-MS/MS methods that were developed in Level 1 labs have been scheduled for transfer, which has increased the network’s potential capability without using additional CDC staff and with only a minimal cost increase.

Although many of the Level 2 PHLs had no chemical analysis capability at the start of the program, a limited number have already established highly successful programs with advanced instrumental capability. These labs requested the ability to train on the same advanced methods as Level 1 labs. Because the well developed Level 2 labs were allowed to train on more advanced methods, the network was able to rapidly expand the number of surge capacity labs, because three of the five new Level 1 labs were already performing most of the transferred methods as Level 2 labs. This program is ongoing, and the expansion of instrumental capability at the Level 2 labs — currently, more than half have purchased LC-MS/MS capability using state funds — has required engaging the Level 1 labs to train the Level 2 analysts for LC-MS/MS methods.

No part of the LRN-C has been more flexible than its QA program. The QA program started as a proficiency testing program and has developed and changed as program needs were better defined. In the beginning, the program’s scope was limited to assessing the network labs’ proficiency through regularly scheduled PT challenges. This assessment required consistent, high quality materials for calibration and quality control, which prompted the creation of the LRN-C materials program. Although the PT challenges could assess technical proficiency, they did not assess how well the emergency response network responded to emergencies. As a result, Emergency Response Exercises were created, in which the labs’ ability to respond to emergency samples and to produce reliable results were combined into unscheduled PT challenges that would better reflect the PHLs’ ability to perform their mission. All PHLs are required to demonstrate their ability to collect, package, and ship clinical samples to qualified laboratories; this requirement led to the creation of a mandatory exercise that tests this ability annually.

Partners are a necessity

What would a network be without them? The key component of any successful network is the partners who comprise the network. The LRN-C has built its success upon the participation of partners who are both external and internal to the CDC. External partners include the PHLs, who are the majority members of the network; the instrument vendors; the Federal Bureau of Investigation; and the Association of Public Health Laboratories (APHL); the latter two entities helped to create the LRN in conjunction with the CDC. Internal partners are both the technical and administrative components within the CDC’s National Center for Environmental Health’s Division of Laboratory Sciences; certain components of the CDC’s Coordinating Office for Terrorism Preparedness and Emergency Response; the components of the CDC’s National Center for Infectious Diseases that created the LRN website and secure communication capability; the former CDC Public Health Program and Practice Office (PHPPO), the enterprise learning office at the CDC; and the National Laboratory Training Network (NLTN), co-sponsored by both the CDC and the APHL.

The program’s instrument vendors have been vital in making the network’s hardware and software resources run smoothly. Because the program purchased multiple packages of identically configured instruments, the vendors provided excellent discounts and collaboration on training activities. The PHLs used some of the money saved from discounted instrumentation to purchase extended service contracts, which have assured support when needed and have been independent of a jurisdiction’s year-to-year variation in funds allocation. The willingness of two vendors, Applied Biosystems and Agilent Technologies, to collaborate on a joint training class has allowed the PHL staffs to bridge the multi-vendor gap in instrument operation and integration. In addition, Advion Bioscience’s collaboration with the Level 1 lab at the Wadsworth Laboratory in Albany, NY, provided the network with an analysis method that can produce a multi-analyte result in less than one minute.

After expanding from five to 62 members, the LRN-C realized that its training had to expand beyond analytical method technology transfer. The network’s first computer-based training (CBT) program was a product designed to lead students through the calibration of the Agilent 5973 mass spectrometer. Inexperienced with this type of training, the network turned to PHPPO’s Division of Laboratory Systems (DLS) for assistance. With PHPPO/DLS’s help and guidance, the LRN-C produced its first CBT product that familiarized new lab employees with instrument operation before the vendor delivered onsite training. By collaborating with the NLTN, PHPPO/DLS also removed the network’s burden of training the network labs in sample collection, packaging, and shipping. The NLTN already had subject matter expertise in the packaging and shipping of clinical samples and had access to appropriate facilities across the country. The NLTN scheduled and delivered hands-on packaging and shipping classes from Boston to the Pacific Basin for the PHLs, which allowed the LRN-C’s analytical subject matter experts to spend their time training in their areas of expertise.

Conclusion

The contributions of the PHLs to the LRN-C are numerous and ongoing. At their start, many of these labs faced instruments being delivered to a loading dock but having nowhere else to go because the labs’ health departments did not have chemical laboratories. Lab employees had to work within their local system to create a laboratory, including acquiring physical facilities, power, high-purity gases, safety cabinets, and sample storage, while trying to meet program requirements for training, validation, and proficiency testing. At the other extreme, established PHLs in the original Level 1 labs struggled along with the network as the program grew and changed and also were active contributors to method development and method improvement activities. Now, these original Level 1 labs are shouldering some of the training burden to make the network stronger and more capable. On a day-to-day basis, the Level 1 and Level 2 labs are not only engaged in CDC lead activities but are reaching out to other responders in their jurisdictions, to hospitals, to civilian support teams, to poison control centers, and to other jurisdictions to improve the United States’ response to a chemical exposure emergency.

Bob Kobelski, (long “e” please) Lead Chemist, National Center for Environmental Health, Centers for Disease Control and Prevention, has held bench and management positions for more than 30 years in organizations that range in size from less than 50 employees to more than 100,000 people. He can be reached at RKobelski@cdc.gov.