David C. Richardson 2016-03-02 17:37:54
Manning the Barricades Kyle Ellenberger of Power Systems & Controls has seen big changes in the data center industry since the earliest few data centers were launched in the United States. “The first data centers were crazy—they had soldiers guarding them and barbed wire. No, you can’t roll up in a tank and steal data,” he says. But things have changed a lot since then. Just about everyone recognizes that the most persistent threats to data are infinitely more likely to weasel their way down a thin strand of wire than rumble through the gates in a frontal assault. For Ellenberger, avoiding power interruptions represents one of the main lines of defense for critical facilities. “When people think about UPS systems they think about data centers, and that makes sense,” he says. “UPS is about a $10 billion industry, and data centers, data processing, and data storage are probably responsible for about $9 billion of that.” He adds that the buildup should not be entirely surprising. Referencing Moore’s Law and Kepler’s Law predicting the growth and importance of data and computing capacity, Ellenberger says, “Moore’s Law says that data processing doubles every 16 months. Kepler’s Law is that Data storage doubles every 12 months. From these laws it was clear that data centers from their inception were going to come into their own rather quickly.” With the primacy of data centers in economic life, a critical necessity for reliable power has also arrived. According to Ellenberger, losing power in a data center can mean that heads will roll. In fact, he says the threat of losing power is taken so seriously in the critical facility power industry that he’s heard people in the business use the expression: “Death before downtime.” Fortunately, there’s a way to tell who in particular might find themselves in the position to take such an oath. As he puts it: “If the power goes out and you can go home before it comes back on, then you’re not mission critical, and you can relax.” Today, the armed guards and perimeter defenses have largely faded from the data center picture. Ellenberger says that data centers can be found distributed almost anywhere in the world in discrete unmarked buildings, each one closely duplicating its peers both in its operations and the data contained within. The data center business philosophy has evolved to one of safety in numbers. They call the current strategy redundancy, and it is the core of protecting the bits and bytes that run much of today’s economic productivity. Data centers are backed up by still more data centers and ideally, the power supplying each data center is backed up by more power. A Squirrel and a Cable Michael Balles’s firm, Venyu Solutions, has data centers in Baton Rouge, LA. He explains that his company serves a number of critical functions. “We serve anyone who needs data: health care, insurance, sports ticketing companies, and computer colocation operations where they can bring in their own computers. Our customers are all over—wherever you can make a connection.” In some cases, the data he stewards can be critical to health and survival. “I have health care clients for which power loss is potentially life-threatening to patients in the hospital,” says Balles. That means that the servers need to be kept up-and-running under all circumstances. It is well known that two of the main threats to data storage, management, and transmission are excessive heat and power loss. However, to be precise, even temperature control in data centers is at the mercy of the availability of high-quality power, making power loss the top threat. Balles says this enemy of data centers’ critical functions can creep up with little warning. “Virtually anything can result in a power loss—a guy running into a utility pole, a squirrel chewing a wire, or a substation problem, almost all of which I have no control.” In other words, power loss from the utility cannot be prevented. What happens after a power loss is therefore critical to whether a data center enterprise rises or falls. As Balles says, “The heartbeat of the data industry is electricity—without power there is no business.” Venyu selected four Generac Gemini units to provide backup power to its newest facility, which it plans to install working closely with Generac. “Everything we do is critical and requires high redundancy,” says Balles. From an operations and maintenance standpoint, he’s found Generac to be superior. And for its capacity, it has the highest reliability. In the name of fail-safe redundancy, Balles also put a few extras into his Baton Rouge II facility. “I got four generators even though I only need three.” In addition to overall backup power, the site will feature a Generac 80-kW backup generator specifically to power fire pumps in case of an outage. “It’s the only data facility that will have an independently powered generator for the fire pump,” explains Balles. “If we lose power, we still need to have power for the fire pumps.” He adds that it was vital to have a source of backup with at least 20 hours of run time. It’s so important in fact, that he considers the onsite generation capability that the Generac Geminis provide to be the facility’s primary power source even though they need operate only during emergencies. Although the local utility supplies most of operational power needs under normal circumstances, he views the utility as just an “inexpensive alternative” that happens to be available most of the time. According to Balles, the Generac Gemini diesel generators are only part of the complex system designed to deliver continuous power to the data center’s critical circuitry and computers regardless of the condition of the grid. With Double Conversion UPS units powering all of the servers and isolating them electrically from any disturbances to grid power, Balles says, the UPS configuration gives the facility “10 minutes of run time at full load.” But the idea is to never need that 10 minutes of run time if the utility power fails. That’s where the Generac comes into play. At the first sign of trouble indicating a power outage, he says the BMS for Venyu’s Baton Rouge facility notifies the fast automatic transfer switches to send a signal to the Generac telling it to fire up. Within eight seconds, the generator registers stable power. Within 10 seconds, it latches on to take over the facility’s load, maintaining stable power until grid power can be restored. Curt Gibson says a unique feature of the Generac is its built-in paralleling capability. That means that any number of units can be easily configured to become synchronized with each other and operate as a single generator. According to him, Generac has codified the complex electronic control required to provide this functionality onto a circuit board installed in each unit. This is a first in the industry. “We put paralleling on one motherboard,” he says. “It used to be done with a complicated mix and match of various components that would enable generators to communicate with one another and coordinate their operation. But we’ve made that easy.” “The paralleling capability is really unique,” says Balles. “We saved a lot of money by not having to purchase additional paralleling gear. And when you’re speaking about millions of dollars, everything counts.” In addition, it provides an added level of redundancy. If one generator were to experience problems or fail, the other generators in the parallel generation model automatically coordinate among one another to pick up the slack, maintaining a constant output as a system and supporting the load. The system really needs “no human intervention,” says Balles. However, he cautions that reliable power is not just a matter of having the right technology in place. “We’re on a strict maintenance routine on all of the equipment with a partner from Baton Rouge that comes out monthly.” Balles says, “I’ve used just about every vendor and I was formerly a consultant in this industry. It’s one of the most important things. Anyone who buys a generator has got to provide maintenance. You can’t just let it sit. A well-maintained diesel generator will last, probably forever. You have to have a good preventive maintenance program in place with a qualified team. “The generators get a test run every week,” he adds. “We have a building management system and any anomaly will force a phone call to our maintenance provider.” He gives high marks to the company on its performance. “In all of our testing, Generac has never failed to meet the 10-second startup time.” A Stitch in Time “We’re getting closer to where bits and bytes are what fuel everything,” says David Dunn, COO of H5 Data Centers. “What fuels bits and bytes, however, is electricity—a lot of it. H5 Data Centers owns and operates of about 1 million square feet of data centers across the country in seven US cities. The company supports large deployments such as major enterprises and content carriers including big Internet companies. Dunn says, “For many of these enterprises, their entire business is online.” The firm’s facilities also support telecoms and dense wave division multiplexing systems that enable the Internet to run. Eaton works with H5 Data Centers and their UPS system, to ensure that power gets to the devices whenever it is needed. In the critical data business, oftentimes that means 24 hours a day, without a fraction of a millisecond of interruption. “In the data center business, there have been a couple of major utility outages,” notes Dunn. “In 2004, there was an outage that took down the whole eastern seaboard across many cities and states, for something like 24 hours. That’s a catastrophic outage. These catastrophic outages can happen, but day-to-day, it’s the transient outages that can potentially impact the reliability and credibility of your operations. “If you don’t have the right systems or people in place,” he continues, “you’re much more likely to have potential outages that don’t look like that big of a deal but can have a real effect on your uptime, the productivity of your workers, and how vendors and customers perceive you in terms of online interaction.” According to Dunn, a power outage of even an instant can amount to a no-win proposition. “When you’re doing animation or you’re rendering a new movie, a one-millisecond outage could take out a bunch of work that was never saved.” All Shapes and Sizes Paul Fischer, data center manager for Eaton, says that the data centers his company serves come in all shapes and sizes. “They could be as simple as single communications devices with wireless capability, all the way up to a data center comprising several hundred or thousands of square feet. What goes into building a data center depends on the scope. Data centers used to be a two-year planning, [and] now we often see this compressed to well under 60 days, or six months.” The core of the plan once again is multiple layers of redundancy. With Eaton’s assistance, the H5 Data Center began renovations on a 300,000-square-foot campus in Colorado it acquired at the beginning of 2014. The facility originally operated as the United Airlines Data Center facility, Dunn says. Over the years it has evolved into a full-fledged data center that he plans to expand to host additional clients. It currently supports Fortune 500 companies, as well as travel and hospitality companies, and a local Denver company that does colocation and data management service for Colorado-area companies. Picturesque Colorado is a great locale for setting up a data center, Dunn says. “It has a fairly dry climate and a fairly temperate climate from an outside air perspective, so you’re able to use the environment to help you cool the data center. It’s also one of the better places in the country for solar.” He says the company is doing a large solar installation at the campus—something in the area of 400–500 kW. Dunn says Eaton helped H5 Data Center configure a concurrently maintainable facility, which delivers the capability to do work or maintenance on any aspect of the data center and not take down a single customer in terms of their live load. Repeating an industry mantra, he says the key to this system is again redundancy. “We enable a 2N+1 configuration,” says Dunn. “Servers often have an A feed and a B feed going into the server. If you lose the A feed, you can rely on the B feed. On the A side we have an N + 1 UPS system, so if you lost a UPS unit on that side, there’s a plus one that would back it up. On the B side, we have a separate N + 1 system. So that approximates a tier 4 standard. It’s very reliable. And on the cooling side, we have a concurrently maintainable cooling design that is N + 1.” To carry the theme of backup power systems further, he adds, “We’ve got large fuel and water storage facilities at the site, so if we lost power for any extended period of time from Excel Energy, we’d be able to run the data center for days, if not months, with both our onsite diesel fuel storage, and our contracts to bring more fuel to the site in the event of an extended outage.” Dunn continues, “Being concurrently maintainable means we can take out any component and the data center still runs. We can replace any component without interrupting operations. That increases the longevity. It allows us to continually maintain the systems at a high level from a continuity perspective, and to continually install more efficient equipment as we go.” He says that the new switchgear that Eaton helped provide and install enables three major components of the power system to interact. Describing the power train Eaton designed, Dunn says, “The utility power comes into the switchgear as it would in any office building. The high-voltage input would be stepped down to a medium voltage. That’s where the emergency generators would be connected as well. So, in the event of a utility outage, the switchgear senses that and starts up the generators. The generators then feed power back into the switchgear, and the critical systems basically don’t know any difference.” Between the switchgear and the critical systems there is a layer of protection provided by Eaton’s UPS systems. Dunn says the switchgear feeds a stream of power into the UPS units that allows continuous operation of the systems at the facility. “In the event of a power glitch—let’s say the utility goes down for a couple of seconds—the generators kick on, which takes roughly seven seconds,” he explains. During that interim, while the generators are coming up to speed, the batteries of the Eaton UPS units carry the load of the critical systems, guaranteeing no loss of critical functionality. “We selected Eaton for their whole economic package, which includes their installation team,” says Dunn. The company’s products compared favorably on H5 Data Center’s total cost-of-ownership calculations, he adds. But the efficiency of Eaton’s innovative UPS units themselves was also a deciding factor. “Legacy UPS systems were fairly inefficient,” he says. “If your output was 100 units of kilowatt-hours, the actual input that the utility would have to deliver to meet that would need to be around 120 kilowatt-hours. So you’d lose 20% of the energy just going through the system. Now Eaton has made those much more efficient, so you’re able to get 93, 94, or 95% efficiency.” Saving on that extra energy expenditure, according to Dunn, can make a significant dent in the bottom line in terms of operations costs. “Colorado is nice in that we have pretty competitive pricing on utility power, but if you were in California, those dollars would add up pretty quickly,” he states. Nonetheless, he says that the UPS units are essential. “The local utility does a great job,” concedes Dunn, “but stuff happens. You get a power spike or maybe a substation blows. We have customers that every second of potential down time is hundreds or thousands of potential transactions that wouldn’t take place online. They have critical systems that, if they went down, basically the companies would stop. So these companies invest a lot of money in not only their servers, but where they locate their data centers. If they put their equipment in H5 Data Centers, we work with them in creating the right levels of redundancy that meet the performance expectations that they have.” He goes on, saying, “We have a track record of over a decade of 100% uptime with zero failures of power to the server. That means if you had a server at that site there would be no failure of power from an outlet or a lack of cooling such that you would have to shut down your server.” Widgets Plus People On a philosophical note, Balles says, “Backup power is critical in my industry, but there is a growing list of other industries in which having backup power is critical.” As an example, he shared the story of an ice storm in St Louis during which lots of customers lost power. “The biggest losers were grocery stores,” says Balles. “They lost a huge amount of food—all of the frozen food. Think of that investment. Now they all have generators. When they added up all they had lost, they saw that it was worth investing in onsite generators.” Reflecting on the emergence of the data industry Dunn says, “Data is productivity. Without the ability to interact and interconnect with others, you regress back to a pre-1980s work environment that I think is widely regarded as less collaborative, slower, and not as effective as what we have today. When you talk about data as power, in some businesses the data is the business. If you’re Amazon, without data you have no business. We’re getting closer to where ones and zeros—bits and bytes—is what fuels everything. It’s taking raw energy and converting it into information. The information is the economic power, and it requires pure energy to make that work.” Making it work, however, takes a lot more than the latest widgets. “You can compare it to the Titanic,” he states. “You can have a great ship that has all the redundancy, but if your captain runs you into an iceberg, you’re toast. Similarly, you can have an amazing system with all the redundancy in the world, but you need great people and processes to run that system. You need a great engine of either people or software capability to run the data center as well.” Today, reflecting a touch of irony in the world of critical data facility power, PSC UPS units have been deployed to guard the military at Holloman Air Force Base, rather than the other way around. The Air Force Base selected the Series MC rotary UPS to replace older static UPS units, which they determined obsolete. The new PS&C Rotary Uninterruptible Power Supply systems will be dedicated to protecting their critical communication and information processes. Holloman personnel decided on PS&C because the company’s rotary UPS systems offer complete electrical isolation and have a robust design. In addition, the RUPS when compared to static, provided twice the number of years of service, which was crucial for their ROI review and analysis. Ellenberger says the beauty of rotary UPS units is their reliability. “They are always online and always conditioning power. We’ve had rotary UPS online for 20 years at some facilities with less than one hour of downtime.” BE David C. Richardson is a frequent contributor to Forester Media publications.
Published by Forester Media. View All Articles.
This page can be found at http://digital.businessenergy.net/article/DATA+CENTERS%3A+Data+Center+Power%E2%80%94The+specific+power+requirements+of+data+centers+and+the+importance+of+redundancy/2414776/292162/article.html.