8 ms·
State of netbooting Raspberry Pi in 2021
- krisoft 5y agoThis article rambles about everything under the sun and barelly scratches the topic promised in its title.
- gurchik 5y agoThe article links to a previous article which in turn links to the official RPi documentation on netbooting. Posting here for people who are curious: https://www.raspberrypi.org/documentation/computers/remote-access.html#network-boot-your-raspberry-pi https://www.raspberrypi.org/documentation/computers/remote-a...
- luma 5y agoIt appears to be an advertisement for the author's $40 workshop video which maybe has the actual details of how this is done.
- m1keil 5y agoFortunately Darren Shepherd came up with a novel idea of taking way the things he didn't want in Kubernetes and created a "distro" called K3s (3 is less than 8) It's not that it is less, "3" is what you get if you take the character "8" and cut it in half vertically.
- c0wb0yc0d3r 5y agoFrom the k3s readme, it states kubernetes is 10 letters stylized as k8s. Half of 10 is 5, and then the project "stylized" it as k3s.
- m1keil 5y agoIndeed, you are right! The explanation above was given by one of the Rancher staff members in one tech conference. It may be an alternative explanation that popped up with time. In both the point is that k3s is "half of" k8s.
- mlyle 5y agoThe BitScope racks are neat, but the thing that makes me sad about Pi is that there's no way to network from the 40 pin connector that would be reasonable to tie this all together. So you need a separate network infrastructure and a network cable to each pi.
- dividuum 5y agoThe Compute Module 4 has an Ethernet chip on-board and is quite a bit smaller than a normal Pi4. It would be perfect to build exactly that.
- mlyle 5y agoYup, a sled for CM4 might be a decent way to do this.
- AndrewSChapman 5y agoGenuinely curious, how would you network from the 40 pin connector in theory? Is it still ethernet?
- gsich 5y agoProbably? With SPI or much slower the UART and pppd.
- aylux2121 5y agoIt seems that many options are available, but SPI and NAND flash boot isn't supported (yet?): https://www.raspberrypi.org/documentation/computers/raspberry-pi.html#:~:text=Pi%20to%20boot.-,Pin%20Assignments,-Raspberry%20Pi%203B https://www.raspberrypi.org/documentation/computers/raspberr...
- themulticaster 5y agoThe 40 pins on the Pi are intended for GPIO and low-speed connectivity (SPI, I2C and friends). Unless I'm missing something, it is simply not feasible to use those GPIO ports for any high-speed interfaces for three reasons: 1. The pins are likely connected to a memory-mapped GPIO controller in the Broadcom SoC (you write a value to a special memory location, and the GPIO controller will physically change the voltage on the associated pin). While you can implement certain slow interfaces - UART is the typical example - purely in software using bit-banging, this approach only works as long as your desired interface speed is a lot slower than the CPU clock. For example: To implement a UART controller at 9600 Bd running on a 1 GHz CPU, you have over 100000 cycles for each symbol you send or receive. For 1000BASE-T, you are looking at a symbol rate of 125000000 Bd, so you'd have just 8 cycles for each symbol (and this is completely ignoring the line code used for 1000BASE-T, which the GPIO controller very likely doesn't support on a physical level). 2. If you want to run an interface faster than a few MHz, you have to be more careful on the electrical level to avoid reflections/impedance mismatch and other signal integrity issues. Essentially, you can't just hook up 2.54mm pin headers at gigabit speeds and expect everything to work fine. 3. Related to the two previous points: Almost all high-speed interfaces (including GbE, USB, HDMI etc.) use line codes at the physical level and/or use differential voltage signaling (LVDS/TMDS etc.) for transmission. The Pi's GPIO controller is very likely unable to send/receive data using differential standards. By the way, if you're interested in this topic: Most FPGAs (even entry-level FPGAs) support differential voltage standards in their IO blocks and even include special encoder/decoder blocks (for 8b/10b encoding).
- AndrewSChapman 5y agoSomething I'd like to know, if you're moving off AWS and using your own raspberry pi cluster, what would you do to provide a performant MySQL DB for example? I can't imagine that trying to run a high performance mysql instance on a PI makes sense. So would you still want an old-school rack server with RAID and all the rest to be your DB server?
- gizdan 5y agoAsk yourself, why do you need a MySQL DB in the first place? If the service can get away with sqlite, use that. Alternatively, there is also rqlite (and I think another similar project) that allows for sqlite to be HA.
- otoolep 5y agorqlite author here, happy to answer any questions about it. https://github.com/rqlite/rqlite https://github.com/rqlite/rqlite
- moondev 5y ago> "Where kubeadm created a headache for Raspberry Pi, K3s made things simple with an easy boostrap, low resource requirements and a single binary for distribution and updates." I'm curious what the headache was. K8S has shipped arm kubeadm and kubelet bins for quite a while. I also don't understand how k3s, microkube and others claim to be "lightweight". Lightweight compared to what? Certainly not upstream kubeadm which is as lightweight a bootstrap as you can get. > "Given how much I'd liked the Docker experience of "docker swarm init" and "docker swarm join", I decided to re-create a similar experience for K3s in the k3sup tool" "kubeadm init" and "kubeadm join"
- catdog 5y ago> I also don't understand how k3s, microkube and others claim to be "lightweight". Lightweight compared to what? For k3s, they explain what they do in their README: https://github.com/k3s-io/k3s#how-is-this-lightweight-or-smaller-than-upstream-kubernetes https://github.com/k3s-io/k3s#how-is-this-lightweight-or-sma...
- lifty 5y agoK3s for example is more lightweight than upstream kubeadm because it compiles all the required components into a single binary. That also leads to considerably less operational complexity.
- moondev 5y agokubeadm dosen't include ANY components such as kube-apiserver, kube-controller-manager, kube-scheduler, kube-proxy or etcd. These components are all containerized already from upstream. Pulled and cached when you run "kubeadm init" via static pod manifests. It seems trying to bundle these inside a non-upstream bootstrapper and running them out-of-band from kubernetes is what would add complexity and bloat.
- gizdan 5y agoI think you might be missing the point. You are correct, kubeadm doesn't include any components in that sense and runs them in containers. The issue with this is that there's a lot of overheads for each service that Kubernetes runs, causing Kubernetes as a whole to have a bigger memory footprint. K3s on the other hand combines all the necessary services and runs them in a single binary, which allows for a reduced memory footprint with very little lost in functionality. I don't know what all was involved when this was done but that is the gist of it.
- rtpg 5y agoAre RasPi's actually performant for this kind of use case? Like relative to just getting a random desktop or server rack... thingy? I can't help but feel like at one point you're just generating a bunch of garbage/wasting energy compared to more performant offerings that might be more efficient all around. Is it price? (Just to be clear, I think "having fun" is a fine excuse to do most stuff, but they're talking about "replacing usage of AWS" which sounds a hell of a lot like a serious business thingy)
- ekianjo 5y agoHardly performant enough. You are way better working with a Xeon server, even used ones.
- AndrewSChapman 5y agoWould certainly be interesting to see the comparison in terms of computing power and power usage between a small pi cluster and a single XEON server. I imagine such a thing would draw at least 200W? And pi's maybe 5 watts? So in theory, 30 - 40 pis vs a single server? The pis also have an advantage in terms of redundancy. A single pi going down shouldn't be a problem and very easily replaceable. Would be a really interesting article.
- tpxl 5y agoVS desktop class hardware, the power draw is a giant plus also leading to no or quiet fans. An old laptop, however, has about the same power/performance consumption while including an USP, keyboard, mouse and screen, albeit it is bigger and has no GPIO if you need those. That being said, I am running a NAS and PiHole on a RPi4B right now and am happy. I am planning on turning an old laptop with busted hinges into a server with a nice housing though. I'm very interested how it's going to pan out :) Edit: I had a 1U server with 2x intel CPUs (16 threads all in all I think?) with 32G RAM in 2012 or so and it didn't see much use. It was LOUD (small screaming fans that pushed air through the chasis) so it didn't see much use.
- 5y ago
- amelius 5y agoHow fast can a Pi boot? Are they suitable for embedded environments where "instant" booting is required? Are there any alternatives which run Linux with graphics which can boot faster?
- dividuum 5y agoThere is bare metal code that almost immediately starts after powering the Pi on. A Pi3 and below with an absolute minimal Linux based OS could probably boot in 3-4 seconds or so. With the Pi4 that has changed and the EEPROM can take a few seconds on its own before the Pi even gets to the kernel loading phase. TL;DR: no.
- wiml 5y agoThe Pi has a weird boot sequence because of the Broadcom chip. SBCs with uncomplicated boot loaders can get you to a shell prompt within one second of applying power; I imagine getting to a lightweight graphical environment in under 2 seconds is not hard. Doubtless you can reduce that a little with tweaking. That's a good deal faster than my car stereo, say.
- villgax 5y agoHey Alex, that was quite a writeup up it didn't really touch upon what exactly did you flash & where did it get flashed?
- rkangel 5y agoThat was the interesting detail that I was looking for too. I'm guessing that it's because at the bottom of the article you can see that he wants you to pay $30 for his ebook and video that will tell you how.
- paxswill 5y agoFrom my own experience, they're probably flashing the RPi4's EEPROM with an updated bootloader and config to enable netboot. Here's the official documentation on it: https://www.raspberrypi.org/documentation/computers/raspberry-pi.html#updating-the-eeprom-configuration https://www.raspberrypi.org/documentation/computers/raspberr...
- aylux2121 5y agoNo, he says that he flashed SD cards. > I flashed each of the Raspberry Pis using the same SD card, then I could close the case and take a note of each MAC address and serial number for later on. But if flashing SD cards, then why do we need network boot? Or, if we have network boot, why do we need to use SD cards?
- paxswill 5y agoThe easiest way to perform the flash is by booting into Raspbian and performing the process interactively. Without this one-time process, you can't netboot, as the default setting is the SD card, then USB: https://www.raspberrypi.org/documentation/computers/raspberry-pi.html#boot_order-fields https://www.raspberrypi.org/documentation/computers/raspberr...
- crispyambulance 5y agoOne thing I've always been curious about is hardware for mass-writing of sdcards. I've setup manufacturing stations for this and I've never been able to find truly robust gang-programmers for sdcards. Luckily my stuff was all low volume so it wasn't a serious problem. But it seems like there's no "manufacturing grade" gang programmers for sdcards. The stuff that is out there is just infinitely re-branded chinese "white-label" hardware-- just a lot of different "vendors" for a few different models of the basically the same hardware. My suspicion for there not being good options for gang-programming of sdcards is that savvy consumer manufacturers have realized what a headache it is to support sdcards in their products from the factory and thus opt to make the sdcard the customer's responsibility. In other words, they ship their products without sdcards and just require the customer to purchase the sdcard and program it if needed. I am not sure if my suspicions are justified, can anyone confirm if this is the case?
- rkangel 5y agoIf you buy them in volume, you can buy them with the data already on from the manufacturer. I'm pretty sure it's programmed into the flash before the flash is assembled into the sdcard packaging. You can do a similar thing with microcontrollers - if you're buying sufficient volume the manufacturer will supply them with your program already on, saving you a manufacturing step.
- crispyambulance 5y agoYes, I've done that before for flash IC's. It works out nicely for "large enough" order quantities. I would expect that doing that for sdcards would require even larger quantities to be economically viable.
- moondev 5y agoThere is no reason at all to boot a raspberry pi 4 from an sdcard in 2021. It's the wrong tool for the job and the worst option you can choose. https://github.com/pftf/RPi4 https://github.com/pftf/RPi4 enables several alternatives, all faster and more reliable * usb3 -> sata * usb3 -> nvme * iscsi * ipxe * netboot.xyz via pxe or usb * usb thumbdrive
- ac42 5y agoThe article has the wrong caption I would think ("running K3s on a bunch of PIs" would be more fitting). Netbooting a bare metal environment is quite straightforward for the PI 3 (dnsmasq has it all) and takes around 15-30 secs.
- josteink 5y agoOdd to see using UEFI to NetBoot not considered an option at all. It should eliminate whatever timing-bugs you have and leave you in a more reliable (and scriptable!) pre-boot environment. What more do you need?!? Both the RPi3 and 4 can be UEFI booted, so this is definitely a real-world option. Links: - https://github.com/pftf/RPi3 https://github.com/pftf/RPi3 - https://github.com/pftf/RPi4 https://github.com/pftf/RPi4
- manuel_w 5y agoSDcards are said to die often/easily in Raspberry Pis. I heard someone say they die when there's a write access going while the Raspi is taken from power. Is that true? That would mean running a Raspi from SDcard is fine as long as there is no write access happening, which can be done by enabling readonly (root) filesystem. Can anyone comment on that?
- uuidgen 5y agoFrom what happened to me - cards die, especially older ones when there is a lot of data written to them even without power failures (UPS ensured that). And when you have a journaled filesystem there is a lot of writes. I made hybrid systems where either the card is only used to boot the system and everything (root included) is loaded from a USB drive or where card is mounted R/O, there is small ramdisk for volatile data and only when upgrade/reconfiguration is needed you remount it r/w, do what you need and then remount it back r/o. In both those setups cards didn't fail for the last 3-4 years. With a standard SD card root-fs they lasted about 4-6 months.
- hipitihop 5y agoI have had a few cards go but I never isolated the cause. Certainly unceremoniously cutting power occurred a few times. I got tired of the inconvenience of having to rebuild the Pi installation and flashing SD cards, so now I boot my Pi4 directly from a USB attached SSD, no card on-board at all. I haven't looked back. Others seem to also be getting good result like: https://www.jeffgeerling.com/blog/2020/im-booting-my-raspberry-pi-4-usb-ssd https://www.jeffgeerling.com/blog/2020/im-booting-my-raspber...