A new customized extension has been added to the Information Technology Division’s (ITD) Building 515 at Brookhaven Lab — and it is green, within budget, and will be ready just in time. 

 

Once the first batch of new computers and storage systems are installed, the new data center will be ready to store and analyze hundreds of terabytes of data from the detectors at the Relativistic Heavy Ion Collider

 

A collaboration with the Lab’s RHIC/ATLAS Computing Facility (RACF), ITD, Facilities and Operations’ (F&O) Modernization Project Office (MPO), and Long Island-based architectural, engineering, and contracting/construction companies, BNL’s new building extension is a part of the computer data center expansion project and is the last major project to be funded by DOE’s General Plant Project Funds. Future similar projects will be supported by BNL Institutional Plant General Plant Project (IGPP) Funds. Data processed at the facility will assist researchers working with RHIC, the LHC’s ATLAS Detector, the Large Synoptic Survey Telescope in Chile, the Daya Bay Neutrino Experiment in China, and the Long Baseline Neutrino Experiment in South Dakota.

 

"This building represents a significant step forward in our Lab's computing infrastructure and will enable the installation of important new capabilities to everyone's benefit,” said Doon Gibbs, the Lab’s Deputy Director for Science and Technology. “I was particularly impressed at the way RACF, ITD and F&O worked together to bring this project to completion, and especially with F&O's ability to deliver such a major undertaking under extremely challenging constraints. They did it safely, on time and on budget and have produced a lot of happy customers. I congratulate F&O for a job very well done."

 

Totaling 11,100 square feet, the extension is made up of a 6600-square-foot main floor and a 4500-square-foot basement. The new data center will house servers and data archives capable of storing multiple petabytes of data. Just one petabyte contains approximately 1 million gigabytes — the equivalent of 6,250 hard drives in a new “stock” Apple laptop computer.

 

“We are essentially a library system that can store and crunch huge amounts of data,” explained Tony Chan, a sub-group leader within the RACF. “With our computers, a fast network, and a robotic tape archive system, researchers can request data and then analyze, store, and reanalyze it from anywhere.”

 

Built to Suit

 

“This building was not designed to be just a shell,” said Michael Ernst, RACF director. “Planning for the building began in December 2007 and the Lab’s Modernization Project Office held meetings with stakeholders and the construction companies each week, maintaining a lot of interaction to develop the building in the right direction. Those meetings were extremely important throughout the entire process.”

 

Unlike other on-site data centers that were installed in pre-existing buildings, this new data center will reside in a highly customized structure designed and built specifically for it.

“In addition to the security reasons, the main reason there aren’t any windows in this building is to keep it very well-insulated,” said Eric Blum, manager of the Brookhaven Computing Facility within ITD. “The computers in this room will produce a lot of heat so there are several very large cooling units inside the main room that will keep everything cool.”

 

Another custom feature of the new extension is the 30-inch raised floor with removable tiles, said Blum. “These large cooling units contain big fans that push a lot of air beneath the floor which then comes out through floor grates near the computers. This floor is 18 inches deeper than the floor in the older data centers but that big under-floor space is needed to keep all of the air circulating.”

 

Dennis Danseglio noted other advantages to the 30-inch raised floor. He is the MPO project manager who coordinated the entire construction of the new building extension.

 

According to Danseglio, electrical and networking cables below the floor’s surface are much more accessible for maintenance and upgrades because of the extra space. The subflooring has also been fitted with several drains and a leak detection system to contain any leakage and prevent flooding from affecting the computers and the facility.

 

Overridden Building Blocks

 

Under the management of Danseglio and F&O’s MPO, the building extension has reached “Substantial Completion” in only 11 months. Some work outside the structure on sidewalks, parking lots, and drainage is still ongoing as computing equipment is installed this month, but the facility is on track to be up and running this fall.

 

“All the snow and rain this year jeopardized the completion date. But we began pouring concrete two days before Christmas and crews worked through several weekends, including Memorial Day and the 4th of July, to make the deadline,” said Danseglio.

 

Not one injury has occurred to date in the time it has taken to reach Substantial Completion. In addition to standard safety equipment such as hardhats, safety glasses, and steel-toed boots, Danseglio had a permanent safety system installed so anyone working on the roof could “tie-off” to the structure, both during construction and in the future.

 

Keeping it Green

 

The extension was also built with the environment in mind, said Danseglio. The ceiling tiles were made from materials that are 71 percent recycled and “fly ash,” soot captured from coal-driven power plants, was added to recycled concrete. This recycled additive reduces the amount of water and concrete needed while also making the mixture stronger and more durable. Other building materials from regional sources were used too, as required by Leadership in Energy and Environmental Design (LEED), a standardized green-building certification system.

 

The computing facility will also rely on a more environmentally friendly back-up power system, or “uninterrupted power supply” (UPS). Rather than relying on a battery-based UPS that takes up large amounts of space and leads to dead, toxic batteries over time, this new facility is backed up by a flywheel system. These flywheels constantly spin using a small amount of power but in the event of a power outage, momentum from the rotating flywheels will provide enough temporary power for the electrical system to switch over to backup generators without shutting down any of the facility’s computers. This UPS equipment will be stored in the facility’s basement.

 

The Current Status

 

The first batch of computing equipment will be installed in Bldg. 515’s new extension during August. Only a portion of the facility will be used in the extension’s early years, but additional equipment will be added as more electricity is directed to the building between now and 2012.