4 ms·
Well, it depends on the colo. I mean, it's not like a colo doesn't have issues. At one point our colo was running some cable and we discovered that their cable
by manv1 4y ago
Well, it depends on the colo.
I mean, it's not like a colo doesn't have issues. At one point our colo was running some cable and we discovered that their cables to our rack were wiggly...after our stuff went offline intermittently. That was hard to track down.
And you need to set up the LOMs, the management network (hey, you don't want that on the public internet), VPN, serial cables (backup to network LOM), maybe a load balancer. And you have to hope that when you reboot the box it doesn't fail POST. And you have to remember to turn off power saving in the BIOS (how many people forget to do this?). And you have to set up the RAID 5/10.
And you have to do this sitting on the floor, because not a lot of places provide chairs in the racks. Oh, do you have a rack-mounted KVM/monitor/serial console or are you using your laptop with a serial cable to do all this?
I mean, it's not hard, it's just a lot of detail work. And I suspect that these skills are going away.
- warent 4y agoThese are great points, thank you for the insight. A small note I will add is that these days there are remote BIOS options available for servers like iDRAC
- manv1 4y agoDo they need a reboot to take effect? I haven't touched a LOM in more than 8 years!
- hotpotamus 4y agoDRACs are a Dell specific term for what is generically known as IPMI or out of band management. They were quite well developed when I worked on servers well over a decade ago now. The good ones are basically as independent of the system they're attached to as possible and essentially give you the equivalent of physical access to a server.
- manv1 4y agoI know what DRACs are (it's Dell's LOM). I asked whether the DRAC bios stuff required a reboot.
- manv1 4y agoJust looked at the docs, it's apply and reboot which makes sense. Oh, I forgot that you also have to manage the certs on the LOMs. Luckily browsers still allow you to bypass expired certs by default. I wonder if you can still get into machines with the old SHA-1 stuff? That's why serial consoles are better IMO.
- hotpotamus 4y agoYeah, I'd consider the DRAC completely independent of the BIOS question. There are also OS tools - I believe Dell's is called OMSA - that can flip some of the switches in the BIOS (I remember this from having to mess with NUMA settings at one time), but they still needed to reboot. As far as the crypto stuff, it's an issue. I fairly recently had to tweak some old 11th generation Dell Servers and tried out the DRACs, but Java really doesn't want you to use RC4 - I remember that being the dealbreaker. Nicely, someone put a conatiner up on Docker Hub that dealt with it https://hub.docker.com/r/domistyle/idrac6/ https://hub.docker.com/r/domistyle/idrac6/
- Symbiote 4y agoI have rarely needed to make changes, but if I recall correctly there were some settings that would take effect immediately, and others that required a reboot. I'd need to refer to the documentation.
- Symbiote 4y agoYou write with authority, but then admit you haven't used this stuff for 8 years. 8 years ago you could already set up RAID or change BIOS settings using LOM, and KVMs were obsolete. The slightly janky Java remote access tools have been replaced by HTML 5 ones, and at least HP and Dell provide tools to script setting up a fleet of similar machines. When I set up new servers, I photograph the sticker with the serial number, MAC address and LOM password, mostly so I can be sure exactly where each machine has been racked. Connect power, ethernet, LOM. Check the DHCP server log on the LOM network to see there are the expected number of new MAC addresses present, then go back to the office to finish the setup.