A shocking surprise after my UPS failed

So in my cellar I have a Linux server… and a NAS running FreeNAS… and a managed gigabit swich… and a router running pfSense… and maybe some PoE gear feeding power to an ADSL modem and a wifi access point elsewhere in the house. Yeah, I like complicated, complicated is cool.

Losing data because the power goes out isn’t cool. But having a UPS and the NUT UPS tools on my network is really cool – power goes out, the UPS starts screaming at me and all the hardware gets told and shuts down in a tidy manner. Very nice.

What isn’t nice is when the UPS appears to be faulty – randomly deciding the mains has gone off, even when it hasn’t, which is what it did at 1:55am this morning

Aug 4 01:51:57 cex upsmon[1996]: UPS ups@chi on battery
Aug 4 01:55:22 cex upsmon[1996]: UPS ups@chi battery is low
Aug 4 01:55:27 cex upsmon[1996]: UPS ups@chi: forced shutdown in progress

Nice to know the system works, but would have been nicer if it only shut down when the actual mains had actually gone off, for real (like it did the other day at about 3pm). Seems I also need a better UPS, 4 minutes of power is a bit … short.

So anyway, I woke up this morning and realised I had No Internet and after entering my underground data centre I was greeted with that eerie silence computers-that-are-supposed-to-be-on-but-aren’t make. Muttering at my crappy UPS I stuck my arm across the first machine to turn it on and received a bit of a zap.

It was like this, only not as bad.

Nothing too bad, kind of like licking a 9v battery, but on my bare arm (which is neither a tongue nor damp). And this is where things got weird. I got my multimeter, stuck it on AC voltage and put one of the probes onto the metal casing, the other probe wasn’t touching anything. The meter read 8VAC… Bit weird, so I took hold of the free probe and tapped it with my finger… the meter then read 115VAC.

Yes America, the voltage between my PC case and my finger read the same as your wall voltage. And here I was, stood barefoot on a slightly damp stone floor going “hmm this is indeed strange, seems there’s 115V going down my finger”. OK so the actual current must have been tiny since I wasn’t lying on the stone floor gently convulsing, but still, clearly we had a bit of a ground fault.

And yes, the sockets in this house are protected by an RCD. It didn’t trip, so I guess the current must have been really small and it was only the relatively high voltage that made me notice it. I have an 88 amp-hour 12V DC battery and can hold its terminals all day without feeling anything. Volts thrill, amps kill, but you need both together to grill.

After some probing with the continuity tester I realised there was no continuity between the wall socket’s earth pin and that PC I had been touching. Nor was there any continuity between the casing of the PC under it and this PC. A bit more probing with a multimeter showed me there was no continuity between the earth pins on each end of that PC’s power cable.

So yeah, I’d been drawing about 60W of 240V quite happily without an earth connection, gradually letting the casing charge up. No wonder the UPS had been going strange – the ground pin of the USB connector was probably sending this unwanted current into the UPS’s monitor and control electronics.

I’ve removed the offending power cable and cut it in half. The bit with the IEC plug has proper continuity, the bit with the plug on doesn’t. Later I’ll cut the plug open and see if there’s anything interesting in it.

This would also explain a few strange things I’ve noticed in the past. The UPS would only seem to go crazy if its USB management cable was attached. My old server would freak out whenever I plugged the UPS in – telling me there was ESD on the USB port. At one time I had a USB SATA enclosure that uses an IEC mains cable for its PSU. This SATA enclosure always seemed a bit odd, in that its metal casing always felt “strange” (if you’re one of those people who can touch an aluminium Macbook and feel a weird ‘bumpiness’, you’ll understand). I guess the common thing between them all was this mains cable.

Why did I keep using a clearly faulty mains cable? Well it was a convenient short length – it was only 75cm long, ideal for keeping wiring tidy. And it was a cable with moulded plugs that contain all the required UK safety logos and numbers.