3 ms·

Once upon a time when I was really really green we had this little lab datacenter, just a room full of machines, and the lab manager guy, crazy dude that I was

Once upon a time when I was really really green we had this little lab datacenter, just a room full of machines, and the lab manager guy, crazy dude that I was a bit afraid of, handed me his pager for the weekend, I guess he had to be gone. I didn't really get this whole on-call thing back then, and so I shoved it into backpack and forgot about it. Well, there was a windstorm, so Sunday evening I check it, and there is a sequence of 24 messages from the UPS "power lost, shutting down server X#", with a gap of a few hours, then 23 messages from UPS "power reappeared, booting up server X#". As I said, I didn't really get this whole on-call thing so I ignored it until Monday when I manually booted server X24. Nobody noticed. Glad it wasn't that important...

neilv 4y ago
Reminds me of one time I was working as a teen, for a workstation software development group, and my time was shared as an assistant to the site sysadmin. The head sysadmin was gone and there was an incident that required rebooting or power-cycling the critical Sun 4/390 server that was in a locked machine room, so people were asking me what to do, but I hadn't yet been allowed to touch that server. This machine handled email, special Internet gateway, Usenet, Sun YP maps, license servers, most of the critical NFS volumes (home directories, tools, SCM of our code), and maybe some weird thing for the non-engineering people who only had Windows boxes. At the time, I thought of power-cycling as risky (for one thing, we had enough SPARCstations with Quantum 105 drives, that a rare power outage meant that we'd usually lose a workstation, though the 4/390 used huge IPI drives). And I think I didn't have that root password, and maybe not even the door keypad code that the console was behind. So I mostly just speed-walked around the building, as if with purpose, while trying to figure out who I could contact, to get access and tell me what the safe procedure was to reboot/cycle the big loud machine that everything depended on.
- LilBytes 4y ago
  Don't leave us hanging! What happened next!?
- nielsole 4y ago
  On my very first admin job all the way back in high school the single server with all the schools data had hung. So I power-cycled it only for it to blow the UPS and the fuse. Probably too much dust had accumulated in the PSU and the current surge of booting shorted it out. The following conversation with the adult in charge was not pleasant. When you press the reset button and the lights above go out with a bang - a stressful day.

neilv 4y ago

Reminds me of one time I was working as a teen, for a workstation software development group, and my time was shared as an assistant to the site sysadmin. The head sysadmin was gone and there was an incident that required rebooting or power-cycling the critical Sun 4/390 server that was in a locked machine room, so people were asking me what to do, but I hadn't yet been allowed to touch that server. This machine handled email, special Internet gateway, Usenet, Sun YP maps, license servers, most of the critical NFS volumes (home directories, tools, SCM of our code), and maybe some weird thing for the non-engineering people who only had Windows boxes. At the time, I thought of power-cycling as risky (for one thing, we had enough SPARCstations with Quantum 105 drives, that a rare power outage meant that we'd usually lose a workstation, though the 4/390 used huge IPI drives). And I think I didn't have that root password, and maybe not even the door keypad code that the console was behind. So I mostly just speed-walked around the building, as if with purpose, while trying to figure out who I could contact, to get access and tell me what the safe procedure was to reboot/cycle the big loud machine that everything depended on.

LilBytes 4y ago
Don't leave us hanging! What happened next!?
nielsole 4y ago
On my very first admin job all the way back in high school the single server with all the schools data had hung. So I power-cycled it only for it to blow the UPS and the fuse. Probably too much dust had accumulated in the PSU and the current surge of booting shorted it out. The following conversation with the adult in charge was not pleasant. When you press the reset button and the lights above go out with a bang - a stressful day.