New York literally came to a standstill following the destruction caused by hurricane Sandy. Sandy was an awakening call questioning the security of data in data centers. Sandy disrupted power supplies, and all other amenities required for the continuous operation of data centers. Unfortunately, in situations like these, there is hardly anything that can be done to help. The disaster preparedness of data centers located on the East Coast and facilities concentrated on the eastern seaboard were put to test. Reports have indicated that the data centers were not able to stand strong against the surge; they succumbed to the destruction. Several things have reportedly gone wrong, some as simple as proper generator location.
Manpower v/s machine power: Data centers are facilities that continuously operate on redundant equipments, and automation. Manpower usage has been reduced to a great extent. But the fact that there is no substitute for human beings was brought to light, when the supposedly reliable machines failed to help. Peer 1 data center was completely under a dire situation when Con Edison shut down the basic utility power supply. Backup power supply from the generators could not be restored because the basement where the generators were housed was flooded. Peer 1’s rooftop generator could not gain access to the 20,000 gallon fuel tank in the flooded basement. The pumping system remained disabled by the hurricane. The only solution that remained was creating a human chain with a bucket brigade stationed at every staircase landing. Fuel filled in buckets was passed from one person to another to refill the generators. Employees worked on shifts to ensure continuous operation of the facility. Daily wage laborers were also hired to help. Though the solution did not equal the level of redundancy that machines would have offered, it definitely created a certain amount of respect for human power.
Create backup: Datagram data center in Manhattan was forced to shut down when the building’s basement was flooded, creating hazardous conditions because of the electrical infrastructure therein. In this case, a shutdown of the facility was the only option. “Crews continued pumping water out of the basement on Wednesday afternoon. Power to the building could not be restored until all water was removed, the company said on a status page it put up to replace its website.” The hurricane also damaged communications lines to the facility, adding to the difficulties. The solution, however, was to switch to an alternate site: “The provider has been offering backup servers to New York customers out of its facility in Connecticut.” Redundancy in location—as well as in onsite systems—enabled Datagram to continue providing services despite a complete shutdown of its facility in Manhattan. Other companies did likewise, using failover sites to maintain operations despite difficulties in Manhattan or other areas.
A vibrant economy can handle a disaster and bounce back quickly; a struggling economy can be damaged severely by the same disaster. Whatever the case, however, we hope for a safe and speedy recovery in the areas harmed by Sandy.
Data Center Talk updates its resources everyday. Visit us to know of the latest technology and standards from the data center world.
Please leave your views and comments on DCT Forum