“If we lose our network,” begins Ed McKenzie, corporate facilities engineer for OSF Healthcare System, “we're going to continue to treat patients. So the amount of lost revenue is difficult to estimate. Within a hospital, if we lose a medical-imaging or a surgical suite, that can be as much as $15- to $20-thousand dollars per hour in lost revenue. And we're not even talking about other health care concerns or loss of data. If the network goes down, we can't retrieve medical films off of the systems, and we have to go to the paper records. So the loss of revenue and loss of production is immeasurable. It's astronomical.”

For OSF, UPS technology has been, and still is, the preferred choice of electronic protection. “Right now there is some other technology developing, such as fuel cells and microturbines, but they're very expensive and they just don't have the proven track record of reliability that the static UPSs currently have. Using these is the tried-and-true way of doing it. They have the best track record, and they're the most cost-efficient.”

The MGE Switchover
Although OSF has been using only MGE Systems Inc. products for years now, it has been using UPSs ever since its new corporate office was built in 1990. UPSs were installed then for reliable power in general, and specifically to provide extra protection for the company's data-automation equipment. Naturally, as the facilities grew, so did the number of UPSs. Now the technology protects all of OSF's heavy-end computer equipment, including centralized computer equipment, the network itself, and multi-gigabyte servers and high-capacity storage devices—all of its networking equipment. This kind of comprehensive, sure-fire protection is crucial for OSF, because all of its facilities are connected to one centralized computer system based in Peoria, IL, with each individual facility having only the peripheral equipment it needs to operate as a separate entity.

OSF was content with its original UPS manufacturer until about six years ago. “We had a small UPS, which served our insurance company, that failed one weekend,” recalls McKenzie. “It was an $8,000 unit. It basically shut down the entire 200-person insurance office. When we called for service, it took the service person eight hours to show up. They told us they couldn't fix it and that the only alternative was to buy a new one.” But afterwards, when no one from the supplier's service department followed up, McKenzie took matters into his own hands. He called a local electrical supply house and asked what equipment was available. The electrical supplier referred him to MGE, and a new unit was installed within 24 hours of the call. OSF has been with MGE ever since.

“Even though it was only an $8,000 sale, they treated us like we were a major data center with a major problem,” says McKenzie, who figured—correctly—that his experience with this one unit was indicative of the manufacturer's high standards for service in general. It is McKenzie's belief that the quality of available UPS technology doesn't differ that much from one manufacturer to another. So, for him, the main determinant in choosing a manufacturer was service. Another reason OSF went with MGE had to do with practicality: The company provided them with greater local access to sales, service, and technical review than their previous manufacturer.

And the Installations Go On and On
After the initial $8,000 unit, OSF began installing additional units almost immediately. As a healthy, expanding company it's always in the construction stage, so UPS installations have become a routine task. “After the first unit,” begins Denzil Merrill, vice president of standard three-phase products at MGE Systems Inc., “we put in a couple of 20- and 30-kilovolt-ampere units, three-phase, on some of their new construction. Then we designed and installed a redundant, parallel UPS system for their main data center. That was a 225-kilovolt-ampere system.”

Most of OSF's new construction calls for new UPSs, all of which are MGE upgrades. None of the replacements, with the exception of the first unit, has been done as the result of failure. Merrill currently counts 121 UPSs at OSF in his maintenance inventory, ranging from small 1,000-volt-ampere units to 225-kilovolt-ampere units. The largest UPSs protect critical systems containing all the information in the data rooms and data centers. These can take days to install. The smaller ones, the EX RTs, take minutes to install.

Over the years, in addition to switching manufacturers, OSF started using a different kind of UPS. At the beginning, obviously, it used older UPS technologies, namely line-interactive, or offline, UPSs. “That wasn't very productive for them, because when they had an outage the unit had to switch to batteries,” says Merrill. “So now they've gone to double-conversion products, also called true double conversion, so that there's no outage and the UPS supplies power 100% of the time under 100% of conditions with no switching. The only limiting factor is the amount of minutes on the batteries used by the UPS.”

So What Does It Protect, Exactly?
In the Midwest, the utility power can range from very stable to extremely unstable based on weather alone. Tornados, lightning, and high winds occur frequently, and many of OSF's facilities are in rural areas where they're particularly vulnerable to severe winds. And with so many different types of equipment, the company's power protection needs are quite diverse.

Most of OSF's new construction calls for MGE upgrades.

“OSF's needs range from the very simple single-phase unit—700 volt-amperes to protect work stations—all the way up through parallel redundant 225-kilovolt-ampere UPS systems,” says McKenzie. “If the computer is protected by a network, it's protected by a UPS. This includes routers, servers, mainframes, and storage devices. We're currently migrating to paperless medical-record systems, so all the data in our systems has to be protected.”

Currently, 95% of OSF's buildings are dedicated to direct patient care. The other 5% are for support service. The vast majority of the hospital's medical equipment is not protected by UPSs. Instead, most have their own internal batteries and backup systems. Pay records, paychecks, medical records, human resources, and centralized material-management systems are covered by UPS technology.

At OSF's facilities, total power outages typically occur two or three times a year. But as far as surges and spikes coming down the line, OSF doesn't monitor them. Reports from Edison Institute show that those can occur as often as 10,000 times a year. “And surges and spikes can actually do more damage,” offers McKenzie. “With the loss of power, you lose data. Then you lose data and you lose revenue because you can't operate. A surge or spike coming down the line, however, could cause a catastrophic failure of equipment in the computer room.”

Multiple Needs, Multiple Uses
There are three types of UPSs used today: 1.) offline, 2.) line interactive, and 3.) double-conversion technology, which is what the hospital uses. With double conversion, the AC source is taken from the raw utility power and changed back into a DC source. Then it's put into the batteries and immediately taken right back out of them, changed to AC again, and fed to the load. “We're always changing the power put into the battery and taking it right back out,” says Merrill. “So if there's an outage or voltage fluctuation or frequency difference, the load never ceases, because … we're always converting it. That is the method OSF has chosen, because it provides them with 100% of the voltage protection they need under any condition.”

Three series of double-conversion MGE UPSs are used at OSF: Comets, Galaxies and Pulsar EX RTs. The Comets and Galaxies are three-phase units, and the EX RTs are single-phase. The three-phase units are used for equipment that has high power needs, whereas the EX RTs are used for small power needs, such as closet protection. “Depending upon which products we use, some UPSs run at 480 V and others at 120 V. The Comet and the Galaxy are either 208 V or 480 V, and the small, single-phase products are 120 V or 208 V at single-phase,” says Merrill.

So how much power can each unit handle? The Galaxy can be sized from 40 kilovolt-amperes up to 75 kilovolt-amperes at 208 V. The Comet can be sized from 10 kilovolt-amperes up to 150 kilovolt-amperes at 480 V. EX RTs can be sized from 700 volt-amperes up to 11 kilovolt-amperes at single-phase. In general, the cost of an MGE UPS runs about $600 per kilovolt-ampere, and the price covers installation.

“The customer's equipment determines which type of UPS would be supplied,” says Merrill. “Larger equipment usually works at 480 V and typically requires hardwiring of the building to install it. At OSF, there are a lot of smaller UPSs at 120 V, because it was inconvenient to rewire the building.”

Everything Has Its Place
Whether there are a smaller number of larger UPS systems protecting more power at once, or a greater number of smaller systems protecting smaller amounts of power, how many UPSs are used at a facility is an economical decision dictated by the existing AC wire at the facility. “When you're designing a new building, it's more economical to design part of the wires for critical applications and the others for just raw utility power,” says Merrill. “In some of the buildings you can see orange outlets, then normal ones.” (Orange outlets are for critical power.) “When building a brand-new building, it would be very easy and cost-effective to protect the entire building. However, in an older building, you don't want to back up a coffee pot or a copier or something like that, because it would be a waste of money. So, to break that circuit off, it requires a lot of effort and sometimes it's almost impossible. In these cases, individual UPSs make more sense.” OSF has several hundred or so small UPSs because it was more economical to put in a UPS per application instead of powering the entire building.

Throughout OSF's grounds, the UPSs are stored away from where the employees work, predominantly in data rooms and data centers or in data closets. OSF has nine data centers and many more closets—over 100.

OSF uses its UPSs in three different ways. As mentioned before, in certain situations it was impossible to rewire the building for one large central UPS because of cost, so in these cases MGE supplied a small UPS that fits under a table or on top of a desk. These are used to back up PC-type equipment. “Those are our single-phase-type products, the EX RTs,” elaborates Merrill. “Then in another application, OSF has joined four to five areas together, and they use a small, three-phase-type product. For large applications, like CT-scan equipment, large computer rooms, or X-ray equipment—all of which require more power—they use single, larger UPSs, either the Comet and Galaxy series.”

The main data center has a very large footprint as well as cooling needs. So the UPSs in those spaces require a good deal of room, many taking up about half the size of a typical work desk. The data closets are small, however, and rack-mounted UPSs are usually placed in those, which tends to save a lot of space.

A Hero to the Hospital
Specifically, in what conditions might UPS technology come to the rescue for a large hospital such as OSF? For starters, in hospitals there's been a growing trend of conducting medical tests internally rather than sending them out to a lab. OSF needs UPSs to back up the time it takes to process these tests within its own facilities. “Some of these tests will take a few hours or so, and if there's an outage during that processing time, the hospital will have to start it again,” explains Merrill. “And in some cases they only have one sample and if they lose that test, they'd have to go back to the patient and do the test all over again.” It's also crucial for the hospital to back up its normal billing, computer processing, and scheduling. Without such backups, the facilities simply can't function.

Another situation the UPSs protect against if that of frequency changes from the utility. “Examples of this would be if, while generating power, the hospital switches to a different generator, or when they have a phenomenon called ‘power factor correction,' which causes a voltage spike or frequency shift for the customer that will affect the computer's power supply,” says Merrill. “A UPS prevents that from happening because it's changing not only voltage but cleaning up and changing frequency to make sure that it's always 60 Hz. UPSs not only correct the voltage, but they also correct the frequency.” In this last scenario, Merrill is referring to a double-conversion UPS. Single ones can't do what he described without going on batteries.

OSF has a generator that backs up its power, and UPS technology also plays a critical part in this. When OSF has an outage, it takes a little bit of time, a few seconds or so, for the generator to start up. “So the UPS bridges that power outage so that the patient or equipment doesn't see the outage until the generator starts up. The UPS acts as a bridge between when the power comes on and when it goes off and when the generator goes on and off,” says Merrill.

The Battery Factor
The double-conversion UPS is on all the time, constantly bridging either the utility or generator. There's never a switchover, never an outage. The key factor in how long a UPS can supply power at full load is the size of its battery.

In general, the UPS batteries are sized for a minimum of five minutes, typically in the area of 15 to 20 minutes. In certain cases it may be as long as an hour. “Anything beyond an hour, and it doesn't make a lot of sense from a dollar or size standpoint,” says Merrill. If a customer has a 20-minute battery, this means the UPS would supply the full power amount for 20 minutes if an outage occurs. “When the batteries have been exhausted, the UPS switches to its bypass to supply the load with raw power from the utility, if available.”

In OSF's case, it buys 10 minutes of battery to provide ride-through until the emergency generator takes over, which usually takes about 10 seconds. A UPS battery is composed of a number of individual jars. Those are composed of strings, and a number of strings compose a battery. “In the small UPS, there's generally two or three jars internally,” explains Merrill. “With some of the larger ones, there may be 180 jars that consist of one battery. So when I say battery, I'm not necessarily referring to just one unit. A battery can be composed of a number of jars.”

Maintaining the Goods
Included in MGE's service package is routine testing and maintenance of the UPSs. For the larger systems, or mainframes, it is recommended that preventative maintenance be done at a minimum of every six months. For the smaller systems, such as the EX RTs, very little maintenance is required—once a year, perhaps, to check the batteries. Maintenance on some UPS systems requires sophisticated equipment, whereas the smaller systems are generally checked by means of software only. At times testing is done during scheduled downtime on the system, and other times—when it will not interfere with employee functioning—it is done during working hours.

MGE UPSs have a built-in, automatic device called a bypass. A bypass allows the service field technician to repair or change out the batteries without interference to the customer. If a unit fails, it automatically transfers to the bypass line and goes around the UPS. “The customer will know the unit failed because of built-in alarms, but he will not have the UPS protection because it's on raw power,” explains Merrill. “Another purpose of the bypass is to perform preventive maintenance, such as changing out the battery without interference to the customer.”

Apart from coming out and testing a UPS, MGE has additional methods for quickly uncovering any problem a unit might have. It installs monitoring devices on its UPSs so that if a unit's self-diagnostic program triggers, an alarm will be sent by e-mail. They also have a simple network management protocol card.

At this point, the design life of the UPS can be anywhere from 10 to 20 years, while the battery design life is basically five years. Preventive maintenance on the batteries dictates when they should be replaced.

And the Growth Goes On
So what does OSF have planned, UPS-wise, for its near future? “Currently, on the drawing board for us is a 50-kilovolt-ampere system being planned for our disaster-recovery site,” says McKenzie. “We also have four data rooms for which we're looking to add a redundant system with an A-bus/B-bus arrangement and 100 Pulsar Ellipses for desktop protection through the systems on critical desktops.”

OSF has never had a problem with power since it started using MGE Systems. “The teamwork that's required between the owner and the supplier, sales, and service rep,” says McKenzie, “is what distinguishes one UPS manufacturer from another.” He likes the fact that MGE is a one-stop shop and that one contact person answers his question, whether it has to do with maintenance, sales, or repair. “It's truly a team there,” he adds.

From Today to Tomorrow
“With the advancement of technology, the use of computers in everyday appliances and equipment is becoming more common,” says Merrill. “The average car, for example, has somewhere around 17-20 different computers in it. Any computer needs to be protected against bad power. This means the application of the UPS is growing because the utility companies remain basically the same.” Utilities are good providers of power for the average business, but not for those with especially critical loads or very fine power needs, such as OSF or any other hospital. “It would be too expensive for a utility company to provide that kind of power or service protection for everyone on the grid. With critical applications such as ours, customers have to provide an extra level of protection for themselves.”

L.A.-based journalist AMY SORKIN KURLAND specializes in marketing communications.

DE - March/April 2006