4 ms·

I'm surprised everyone is still installing to disk. In 2008/2009, we had a POC where we ran the OS from memory after booting over PXE. * boot a live image i

I'm surprised everyone is still installing to disk. In 2008/2009, we had a POC where we ran the OS from memory after booting over PXE. * boot a live image into memory * point LXD/RKT/Docker to /containers * ... * profit!

iofj 11y ago
The big issue I'm having with that is that it involves trusting vendors to get network boot right. Especially when it comes to the looping part of "loop until DHCP gets a response" it becomes a problem. One of the cheap vendor tries 30 times and then goes to a boot failed screen after trying the disk. Also, 1 time out of a 4-5000 or so network boot fails. Not sure why.
- vidarh 11y ago
  If you have IPMI on the server this doesn't become such a big problem - you can reasonably trigger resets/reboots if it's not up after a given amount of time.
  - iofj 11y ago
    We buy the cheapest server that meets our needs, and buy it in somewhat larger quantities (often double what was originally envisioned for less than was originally budgeted). Much more efficient. But it does mean no IPMI. However I built a small circuit that sits on a power cable that can interrupt said power cable with a relais that sits on a bus plugged into our server, so we can do the reboot thing. I've been meaning to redo that power cable circuit using wifi as the linking technology, now that we have esp8266 available.
- mverwijs 11y ago
  That's where iLO comes in. iLO is horrible, but you can ssh to it and set all manner of stuff. When we didn't have PXE, we had a script that told iLO to boot from CD, and that the CD was located at http://something/bootme.iso http://something/bootme.iso. iLO would always have network, and would pass the .iso magically to the server as device to boot from.
bcantrill 11y ago
For whatever it's worth, SmartDataCenter, Joyent's open source SmartOS-based system for operating a cloud[1], does exactly this[2] -- and (as explained in an old but still technically valid video[3]) we have found it to be a huge, huge win. And we even made good on the Docker piece when we used SDC as the engine behind Triton[4] -- and have found it all to be every bit as potent as you suggest![5] [1] https://github.com/joyent/sdc https://github.com/joyent/sdc [2] https://github.com/joyent/sdc/blob/master/docs/developer-guide/architecture.md https://github.com/joyent/sdc/blob/master/docs/developer-gui... [3] https://www.youtube.com/watch?v=ieGWbo94geE https://www.youtube.com/watch?v=ieGWbo94geE [4] https://www.joyent.com/blog/triton-docker-and-the-best-of-all-worlds https://www.joyent.com/blog/triton-docker-and-the-best-of-al... [5] https://www.joyent.com/blog/spin-up-a-docker-dev-test-environment-in-60-minutes-or-less https://www.joyent.com/blog/spin-up-a-docker-dev-test-enviro...
vidarh 11y ago
CoreOS lets you do this if you PXE boot. It will then by default run entirely out of RAM.

iofj 11y ago

The big issue I'm having with that is that it involves trusting vendors to get network boot right. Especially when it comes to the looping part of "loop until DHCP gets a response" it becomes a problem. One of the cheap vendor tries 30 times and then goes to a boot failed screen after trying the disk. Also, 1 time out of a 4-5000 or so network boot fails. Not sure why.

vidarh 11y ago
If you have IPMI on the server this doesn't become such a big problem - you can reasonably trigger resets/reboots if it's not up after a given amount of time.
- iofj 11y ago
  We buy the cheapest server that meets our needs, and buy it in somewhat larger quantities (often double what was originally envisioned for less than was originally budgeted). Much more efficient. But it does mean no IPMI. However I built a small circuit that sits on a power cable that can interrupt said power cable with a relais that sits on a bus plugged into our server, so we can do the reboot thing. I've been meaning to redo that power cable circuit using wifi as the linking technology, now that we have esp8266 available.
mverwijs 11y ago
That's where iLO comes in. iLO is horrible, but you can ssh to it and set all manner of stuff. When we didn't have PXE, we had a script that told iLO to boot from CD, and that the CD was located at http://something/bootme.iso http://something/bootme.iso. iLO would always have network, and would pass the .iso magically to the server as device to boot from.

bcantrill 11y ago

For whatever it's worth, SmartDataCenter, Joyent's open source SmartOS-based system for operating a cloud[1], does exactly this[2] -- and (as explained in an old but still technically valid video[3]) we have found it to be a huge, huge win. And we even made good on the Docker piece when we used SDC as the engine behind Triton[4] -- and have found it all to be every bit as potent as you suggest![5] [1] https://github.com/joyent/sdc https://github.com/joyent/sdc [2] https://github.com/joyent/sdc/blob/master/docs/developer-guide/architecture.md https://github.com/joyent/sdc/blob/master/docs/developer-gui... [3] https://www.youtube.com/watch?v=ieGWbo94geE https://www.youtube.com/watch?v=ieGWbo94geE [4] https://www.joyent.com/blog/triton-docker-and-the-best-of-all-worlds https://www.joyent.com/blog/triton-docker-and-the-best-of-al... [5] https://www.joyent.com/blog/spin-up-a-docker-dev-test-environment-in-60-minutes-or-less https://www.joyent.com/blog/spin-up-a-docker-dev-test-enviro...

vidarh 11y ago

CoreOS lets you do this if you PXE boot. It will then by default run entirely out of RAM.