WASHINGTON — A new National Research Council report requested by Congress finds the U.S. Department of Homeland Security's updated site-specific risk assessment for the proposed National Bio- and Agro-Defense Facility (NBAF) in Manhattan, Kan., a "substantial improvement" over the original 2010 version, but it has a number of deficiencies and inadequately characterizes the risks associated with operating the facility.

The NBAF would be the world's fourth Biosafety Level 4 laboratory capable of large animal research, replacing the aging Plum Island Animal Disease Center located off Long Island. It would study dangerous foreign animal diseases -- including highly contagious foot-and-mouth disease (FMD), which affects cattle, pigs, deer, and other cloven-hoofed animals -- as well as emerging and new infectious diseases that can be transmitted between animals and people. In 2010 the Research Council reviewed the original site-specific risk assessment by DHS and found that it was inadequate due to flawed methods and assumptions. In response, Congress mandated that DHS revise its assessment to address shortcomings and directed the Research Council to evaluate the updated assessment, which is the focus of this report. Another Research Council committee, whose work is separate from this report, is examining the needs and possible scientific alternatives for a foreign animal and zoonotic disease research laboratory in the U.S.

The committee that reviewed the updated risk assessment found that many of the shortcomings identified in the 2010 report have been addressed and that the new version uses more conventional risk assessment methods and conceptual models, presents clearer descriptions of the approaches, and complies better with standard practices than the previous version. However, some of the risk analysis methods were misinterpreted and misapplied when executed. The assessment also contains questionable and inappropriate assumptions that led to artificially lower estimates of the probability of an accidental pathogen release from the facility, the committee said.

Information from the 2010 risk assessment implies that for the two greatest release scenarios there is nearly a 70 percent chance that a release of FMD could result in an infection outside the laboratory over the 50-year lifetime of the facility. In contrast, the updated assessment concludes that for 142 possible release events, the cumulative probability of a release leading to an infection is 0.11 percent, or a 1 in 46,000 chance per year.

The committee noted that some of the risk reduction may be explained by improvements to the latest design plans for the facility, but despite improvements, the updated assessment underestimates the risk of an accidental pathogen release and inadequately characterizes the uncertainties in those risks. Moreover, the committee found that the updated probabilities of releases are based on overly optimistic and unsupported estimates of human-error rates; low estimates of infectious material available for release; and inappropriate treatment of dependencies, uncertainties, and sensitivities in calculating release probabilities. For instance, the 2010 assessment concludes that human error would be the most likely cause of release, and the previous Research Council report agrees. However, the updated assessment concludes that natural hazards, specifically earthquakes and tornadoes, would be 20 times more likely to cause a pathogen release than operational activities at the facility. The committee questioned whether natural hazards posed the greatest risk for accidental release of FMD and believed the updated assessment overestimated the release probabilities due to both kinds of natural disasters versus other pathways.

In addition, the low estimates of risk found throughout the updated assessment are not in agreement with most modern, complex industrial systems, and in many instances the committee could not verify results because methods and data were unevenly or poorly presented. The updated assessment also contains inconsistent information, which made it difficult to determine the degree to which risks were underestimated.

"A risk assessment of this nature carries significant complexities, yet the practice of completing such assessments is mature enough that the committee's expectations are attainable," said Gregory Baecher, chair of the committee and Glenn L. Martin Institute Professor of Engineering at the University of Maryland, College Park. "Because a pathogen release from the NBAF could have devastating agricultural, economic, and public health consequences, a risk assessment that reaches inappropriate conclusions could have substantial repercussions."

The committee said the updated assessment addresses many of the issues outlined in the congressional mandate. However, it still does not adequately include overall risks associated with the most dangerous pathogens. Additionally, assumptions about surveillance, detection, response, and mitigation strategies were not adequately comprehensive or science-based.

The latest design plans for the NBAF appear to be sound, the committee said. It reviewed the design documents to understand the assumptions about the release probabilities for the risk assessment and to verify that design concerns raised in the previous Research Council report were addressed. The committee noted that inadequacies in the updated assessment do not imply inadequacies in the facility's design. It was beyond the committee's task to formally review or pass judgment on the actual engineering or safety of the facility.

The study was sponsored by the U.S. Department of Homeland Security. The National Academy of Sciences, National Academy of Engineering, Institute of Medicine, and National Research Council make up the National Academies. They are independent, nonprofit institutions that provide science, technology, and health policy advice under an 1863 congressional charter. Panel members, who serve pro bono as volunteers, are chosen by the Academies for each study based on their expertise and experience and must satisfy the Academies' conflict-of-interest standards. The resulting consensus reports undergo external peer review before completion. For more information, visit http://national-academies.org/studycommitteprocess.pdf.