After I got out of the Army I went to work as a tech support contractor for the Marines and learned a valuable lesson. Marines are great at breaking things, be it the enemies things or USMC IT resources. This story is about the intersection of the USMC Private First Class and some very poor engineering.
So, the guts of what I supported was essentially a network in a trailer. It had switches, routers, servers, a NAS device, and three rack mounted UPS systems. All of this stuff was covered by canvas and connected internally in a magnificent web of cabling. All of this cabling led to an I/O panel above where the tail gate dropped into a sort of desk. Now, central to the power panel was the Emergency Stop Switch which was wired to the serial ports on the UPS systems to kill power in an emergency.
Makes sense, right? Well, due to some craptastical engineering, not only did it turn the UPS systems off, it also managed to fry the controller boards so they would never turn on again.
Enter my service call, where I am called out to the field from my nice cushy office to diagnose and repair a power failure. I come out and sure enough, no power. I try turning the UPS systems on, no dice. I check to make sure the cabling is all connected right, yup. Then, on a whim I pull on the emergency stop button and it clicks back out.
Me: Why was the emergency stop pressed? Marine: I dunno. It was like that when we got here. Me: Was there an overnight watch? Marine: Yes. PFC Dingus was on watch last night. Me: Someone needs to go get him right now.
Wait a while
PFC Dingus arrives loudly exclaiming innocence, with his cover pulled suspiciously low.
Me: Please take of your cover. He slowly removes it to expose the big red circle on the side of his forehead.
Heads shake around the room.
Apparently PFC Dingus fell asleep on watch and fell forward effectively head-butting the emergency stop button (and costing the USMC about $12,000 in parts and labor)
'via Blog this'