UPS Shorts Out and Kills Nortel Phone System PSU

I had the strangest bit of luck a week ago regarding our 4 APC units (Smart UPS 1500 for reference) that supply backup power to our server closet. I walked in that morning for an unrelated event and see a bright red light shining at me telling me that a one of the UPS's (#4 in particular) needed a battery change. I figured sure, I will replace it tonight at 9pm and move on with my simple life.

That evening a simple battery change resulted in the same issue again when I fired back up the UPS unit, a glowing red change battery light. At this point our telephone and internet had been down for long enough that I decided to replace the UPS entirely so I could get those items back up again. Not thinking about anything else I replace the UPS, plug in the power cables for a 2nd time and let everything take it time to come back up. The good news was that due to redundancy in our ISCSI network backbone and dual PSU's in our servers and SAN storage no data was lost, damaged or stopped working internally during this time.

30 minutes later however, the telephone system had still failed to come up and multiple telephones gave off a creepy Silent Hill like static warning me of coming danger. I decided then something had happened and gave our phone vendor a call, he was not optimistic and prepared for a long night with me at work. In the end (1am at this point) we were able to keep our programming by replacing the 300$ power supply and get her back up and running, but we were still perplexed as to how 1 system behind a UPS, attached to a surge protector went bad out of 12 items plugged into the unit.

Here it is a week later and the "Bad" UPS has functioned fine during tests with both its original battery and the one I tried to replace it with. I surely won't place this into production, but I would love to know what happened last week and any way I could prevent it from happening again.

1. Make sure the UPS is unplugged and all items are also unplugged before removing the battery

2. Replace the battery and power on the unit before plugging in any items to the back

3. After 15 minute run time plug in items one at a time so as to not create a large power draw if items are booting up.


Hopefully changing my steps a bit will help me, and you if this happens in your IT world.