5 ms·

We did something similar at Optiver. * boot a custom live cd (a la Knoppix) over PXE * Live CD places node into database if it doesn't exist yet,

We did something similar at Optiver. * boot a custom live cd (a la Knoppix) over PXE * Live CD places node into database if it doesn't exist yet, using dmidecode to find serialnumbers and such * Live Cd keeps querying database for instructions * Engineer adds a profile to the node in the database * Live Cd slices up disk to match the profile * Live Cd fetches a tarball of the base OS from an URL and throws it on the metal. Runs grub setup. Reboots. Tumblr did something like this with http://tumblr.github.io/collins/ http://tumblr.github.io/collins/.

defect 11y ago
Also checkout https://github.com/Tumblr/genesis https://github.com/Tumblr/genesis which does sort of the same thing as the PXE image GitHub writes about.
voltagex_ 11y ago
Hey cool so what I'm fiddling around with in iPXE isn't completely obsolete in the age of Docker and disposable VMs.
- mverwijs 11y ago
  Well, no. But IMHO, iron is only do-able if you have enough of it. This I learned the hard way at my previous contract, where we only had a handful of servers and all of them production. When you want to automate your complete infra, including rolling out hardware, you need hardware to develop and test on. Entropy of life will ensure that exactly that moment you need to reinstall that PostgreSQL slave from scratch, the PXE server is unreachable, or the server has a different diskcontroller, or an iLO certificate expired. Or something stupid. Test your code. And for Ops this means: machines that are solely for the testing pleasure of Ops. No other function.
  - voltagex_ 11y ago
    In a perfect world you'd have a Dev, Staging and Prod PXE server, but reality says you're not going to be able to get signoff to run that many.
    - mverwijs 11y ago
      But that is perfectly fine! That only means that you cannot have an automated install procedure that the company can rely on. There is really nothing wrong with a little manual labour at this stage. Do not spend weeks and weeks on automating this without having the environment to test your code.
- vidarh 11y ago
  Even those machines running Docker and disposable VMs still need to run on something. I've got systems PXE booting to CoreOS where everything else runs in Docker containers (even the odd KVM VM).
mpdehaan2 11y ago
I did something sort of similar with Live CD images for booting hardware not on a PXE network in Cobbler. It could also register machines seen then first time by making system records for them. https://github.com/cobbler/cobbler https://github.com/cobbler/cobbler What is interesting in this article in particular are the auto-firmware upgrade state transitions, which seem pretty neat. The chat bot is I guess neat, but if a lot of people are using the cloud that could be a hard way to get status.
invertedohm 11y ago
Kind of amazing that the state of the art in this area hasn't changed in nearly 10 years. The marketing angle is funny as well - everything has to be a cloud now. I think I used to call what we built at Optiver a "private cloud" but "Metal Cloud" is nicely buzzwordy as well. PS - Hi Marty, unknown nick here but I'm sure you can figure out who I am. :D
jsjohnst 11y ago
Actually there's heavy overlap in functionality if you include other Tumblr things with Collins like Phil, Genesis, and Configrr. Collins - asset management Phil - ipxe booting based on Collins state Genesis - base hardware configuration (firmware, bios, raid, etc), burnin, kickstart Configrr - state / config management

defect 11y ago

Also checkout https://github.com/Tumblr/genesis https://github.com/Tumblr/genesis which does sort of the same thing as the PXE image GitHub writes about.

voltagex_ 11y ago

Hey cool so what I'm fiddling around with in iPXE isn't completely obsolete in the age of Docker and disposable VMs.

mverwijs 11y ago
Well, no. But IMHO, iron is only do-able if you have enough of it. This I learned the hard way at my previous contract, where we only had a handful of servers and all of them production. When you want to automate your complete infra, including rolling out hardware, you need hardware to develop and test on. Entropy of life will ensure that exactly that moment you need to reinstall that PostgreSQL slave from scratch, the PXE server is unreachable, or the server has a different diskcontroller, or an iLO certificate expired. Or something stupid. Test your code. And for Ops this means: machines that are solely for the testing pleasure of Ops. No other function.
- voltagex_ 11y ago
  In a perfect world you'd have a Dev, Staging and Prod PXE server, but reality says you're not going to be able to get signoff to run that many.
  - mverwijs 11y ago
    But that is perfectly fine! That only means that you cannot have an automated install procedure that the company can rely on. There is really nothing wrong with a little manual labour at this stage. Do not spend weeks and weeks on automating this without having the environment to test your code.
vidarh 11y ago
Even those machines running Docker and disposable VMs still need to run on something. I've got systems PXE booting to CoreOS where everything else runs in Docker containers (even the odd KVM VM).

mpdehaan2 11y ago

I did something sort of similar with Live CD images for booting hardware not on a PXE network in Cobbler. It could also register machines seen then first time by making system records for them. https://github.com/cobbler/cobbler https://github.com/cobbler/cobbler What is interesting in this article in particular are the auto-firmware upgrade state transitions, which seem pretty neat. The chat bot is I guess neat, but if a lot of people are using the cloud that could be a hard way to get status.

invertedohm 11y ago

Kind of amazing that the state of the art in this area hasn't changed in nearly 10 years. The marketing angle is funny as well - everything has to be a cloud now. I think I used to call what we built at Optiver a "private cloud" but "Metal Cloud" is nicely buzzwordy as well. PS - Hi Marty, unknown nick here but I'm sure you can figure out who I am. :D

jsjohnst 11y ago

Actually there's heavy overlap in functionality if you include other Tumblr things with Collins like Phil, Genesis, and Configrr. Collins - asset management Phil - ipxe booting based on Collins state Genesis - base hardware configuration (firmware, bios, raid, etc), burnin, kickstart Configrr - state / config management