7 ms·
Extreme Pi Boot Optimization
- opengears 2y agoIt is such a shame the RPi Zero2 does not support "traditional" sleep modes like the ESP32 for example - which is why we have to optimize the Linux boot process. https://forums.raspberrypi.com/viewtopic.php?t=243719 https://forums.raspberrypi.com/viewtopic.php?t=243719
- beachy 2y agoVery impressive. I've toyed with using the Pi for an intelligent trail camera. Startup time is critical - a PIR sensor detects an animal passing and you want to be taking photos ASAP so every second counts. Lowering the power usage is awesome too.
- tuatoru 2y agoJust use a purpose-built trail cam. Sub-second response, optics optimised for trail conditions, weatherproof, robust, durable, long battery life, designed to be secured to trees. There's a bit of engineering in a reliable trail cam. (Don't buy no-name Chinese.)
- beachy 2y agoI already have a couple of Bushnells. My desire is to be able to attach a small network of PIR sensors to each camera get better information.
- gigel82 2y ago3.5s is cool, but if the entire scenario was really connecting to WiFi and uploading an image every couple of minutes, an ESP32 would've been a much better choice for power consumption (unless the camera module you need for Pi has some specific features that none of the esp32-cam compatible cameras does)
- teamonkey 2y agoESP32 only supports up to 4MB of PSRAM while a single RPi HQ Camera still is 18MB.
- pitaj 2y agoDoesn't the camera have it's own framebuffer that the MCU can stream? I don't see why the MCU would have to hold the whole frame in memory.
- teamonkey 2y agoAt least with ESP32-CAM api, the instruction to capture an image returns a pointer to image data in psram. I would imagine a Pi Zero is more efficient at converting that raw image data to some compressed file format too.
- m00x 2y agoESP23-P4 will support up to 32mb in PSRAM, MIPI, and hardware h.264 encoding. It'll be a great chip for video.
- j16sdiz 2y agoThe library comes with ESP32 use DMA for image stream. Don't think you can workaround that, unless you write your own driver
- Neywiny 2y agoI might recommend a slightly higher end micro with a mipi csi interface but otherwise agree. This is so much work to do what microcontrollers can do almost effortlessly.
- nyanpasu64 2y agoI was thinking that Circle (https://github.com/rsta2/circle https://github.com/rsta2/circle) might be faster to boot than a kernel, but it doesn't seem to support MIPI cameras.
- ocean_moist 2y agoI wonder if booting the OpenBSD kernel would be faster. Although, the OpenBSD init system is notoriously slow. Also I feel (but don't know for sure) most of the time before executing the user space program would be spent by systemd.
- jmclnx 2y ago>Although, the OpenBSD init system is notoriously slow. Is this on the PI where it is slow. On my T420 it seems fine, but the re-linking of various daemons does add time. But that is done for security so I am fine to live with it. Me, I want fast power down time so I can get out the door fast. And so far NetBSD, OpenBSD and Linux seems to meets that need :)
- hamburglar 2y agoDon’t these types of systems generally just nuke init entirely and provide a custom PID 1 ?
- Avamander 2y agoIMHO boot times of Linux distros in general are rather sad, which is then significantly amplified on weak(er) hardware such as this. I've gone through similar efforts with the MQ-Pro SBC. One can also really feel this on laptops (except Macbooks I guess). Annoying.
- lionkor 2y agoIt very much depends what you define as "boot time". For example, Windows optimizes for time to fist UI, meanwhile everything else continues to load and the PC stays unusable for multiple seconds after "boot".
- pandemic_region 2y agosorry man but that typo had me lol, early mornings here I do apologize.
- layoric 2y agoExactly my experience as well. I moved away from Windows (10) earlier this year, and login times (waiting for the user desktop to come up after putting in credentials) took ~10 seconds. Sometimes it was faster after a fresh install, but it never lasted, even when regularly managing what services and apps were loaded on startup.
- gerdesj 2y ago"Windows optimizes for time to fist UI," I'm going to apologise right now for giggling at your typo. Microsoft have done some horrible things to UI. The start menu - yes they created the fucking thing, yes it is now called start by everyone - own it (think Biro and co and stop being dicks) and it belongs at the left hand side. "We" know better than you, lets put it in the middle and surround it with weird shit and lets make it odd and put fucking games controllers on a corporate laptop and other wankery that we can't be bothered to curate because we are so poor but if you love our weather forecasts and shitty ... whomever will pay us .. whatever thing. I think that Microsoft have lost interest in humanity as anything than a pool of subscription slaves to contribute to their bottom line. That is some pretty aggressive fisting.
- jtrueb 2y agoJust stay booted and use a lower power microcontroller … 105mA … that’s not the right order of magnitude
- award_ 2y agoI don't understand what this means exactly, could you please elaborate?
- declan_roberts 2y agoEvery single person is reading this with a voice in their head asking "why not use an esp32 et al?" But of course the article is good enough because it's interesting, even if it's not the right tool for the job.
- nextlevelwizard 2y agoAnswer is simple: speed of development. I am (sort of) in a same boat as the author. I have a silly little project that takes pictures that I PoC/mock-ed with RPi Zero. It does use battery, but I am not overly concerned with battery life - however boot time is killer. It takes loooong time to boot and be ready. However I wrote all the code in an afternoon and it just worked. I could use a higher level language (Python) to clue in all the C libraries that are used to talk with my camera, screen, and wifi stack. I know that I could achieve way better battery life and boot time by switching to ESP32, which is why I ordered one the day of after I got my PoC working. The parts have sat 9 months in my projects bin. I did write the basics of getting the image bytes off of the camera, but after that I couldn't find the motivation to continue. I already have a working version of the thing I wanted to make. It does everything I want already. Why spend bunch of time figuring out the way more esoteric libraries and actually try to implement everything (again) in C just to get a bit better battery life and boot time? Of course in a "production" system it would be very weird and suboptimal to continue using RPi after this point, but for a small one off or even low run project any extra development effort feels like a waste. I should dig up my project and see if I could flip some config bits and get the boot time lower, I'd be extremely happy with anything sub 10 seconds and then I could completely burry the EPS32 idea as redundant.
- phoronixrly 2y agoYou really should look into using the right hardware for the purpose instead. (Disclamer - I despise Raspberry and their overpriced closed devices, and also HN maniacally trying to use them for stuff they the wrong choice for)
- nbf_1995 2y agoWhat sort of hardware would you recommend for this use case?
- nickelpro 2y agoLiterally any ESP32 would be better suited. There is zero reason to be booting an entire OS to take a picture and blip it over WiFi.
- moefh 2y agoESP32 is great, but it simply can't work with the IMX477 camera used in this project. This camera has resolution of 4072x3176, or about 12M pixels, which is way above what any ESP32 can handle.
- alfanick 2y agoI can imagine following should be doable (with assumption that IMX477 has it's own buffer and doesn't DMA directly): 1) take a picture 2) read some lines 3) stream them via WiFi to some server 4) repeat 2-3 until whole picture is read 5) reconstruct the picture from slices on the server side
- teamonkey 2y agoThe sensor doesn’t have a framebuffer (because it’s just a sensor) and the RPi HQ cam is basically just a sensor on a board with some mipi connectors. You might be able to buy a package with an IMX477 sensor and some microcontroller/FPGA and frame buffer ram, but that would cost a lot more.
- Sarkie 2y agoWas expecting to see different governors tested. Is that not a thing on a pi?
- kiwijamo 2y agoI'm fairly sure this is, I recall playing with this when I first got my Raspi.
- dgacmu 2y agoPower is really one of the weaknesses of the rpi family (I'm quite excited for the new pico 2 for exactly this reason - it seems like they're finally making it easy to enter a relatively deep sleep without external hardware). I built some cameras for an application like this using a Google Coral mini, whose camera is not nearly as good as the HQ cam, unfortunately, but it supports a built in suspend + wake from onboard RTC that is very easy to use and perfect for a periodic camera app - while still having enough oomph and 2GB of memory to handle a high resolution image. (You can physically hook an HQ camera up but the software pipeline doesn't exist to manage it on the coral AFAIK.) The Rpi ecosystem is a lot more mature and (sorry, friends) I trust the future availability of rpi more than I trust Google to keep delivering the coral line, but it really underscored how helpful good power support in the hw was. (Ironically, we ended up outsourcing the next version of these cameras to a firm that built them using an rpi and we just threw in a much larger battery to compensate. Which means I have a stack of 100 unopened coral dev minis + cameras looking for either good ideas or to sell to someone. Oops.)
- cgearhart 2y agoInteresting, I’ve always run pis from wall power. Is the pi hardware incapable of similar power optimizations to coral, or is this a problem of a lack of software support for power management on pi? (I assume from your mention of external hardware to manage power that it’s not just a software issue.)
- dgacmu 2y agoI'm really mixing two things and wasn't very clear about it. The pico can kinda deep sleep but it requires an external wakeup trigger. It can't deep sleep from its own clock. Even so its deep sleep is pretty high power compared to most embedded chips. The zero (w) and zero 2 w don't have the equivalent of suspend-to-ram with a low sleep current. I'm not sure if that's a limitation of the SOC or the driver or both, but rpi was fairly clear it wasn't in the cards: https://github.com/raspberrypi/linux/issues/1281 https://github.com/raspberrypi/linux/issues/1281
- rcxdude 2y ago
- geerlingguy 2y agoSupposedly on the Pi 5, the SoC could be put to sleep while RP1 remains active, and the RP1 has enough compute to handle like 4 or 8 pixels of data from an attached camera... I think RPi might be able to get much better suspend support with their new PMIC and RP1. But so far still waiting to see something handy like Wake on LAN support native in Pi OS.
- dividuum 2y ago> Supposedly on the Pi 5, the SoC could be put to sleep while RP1 remains active [..] You mean "echo +60 > /sys/class/rtc/rtc0/wakealarm && halt"?
- markus_zhang 2y agoMy first instinct: can we use some other core? Do we really need Linux to take a photo and transfer it to the cloud? I'm not a hw person so curious how to complete the task with minimum budget. Interesting read. Thank you!
- bigiain 2y agoMy same first thought. For no reason other than I have a pair of them sitting on my kitchen table right now, I wondered how the ESP32-CAM setup would compare. I think it's only good for 2megapixel images, But I'd bet both its startup time and its power consumption would be close to an order of magnitude lower. (Here's some details if you're curious: https://components101.com/modules/esp32-cam-camera-module https://components101.com/modules/esp32-cam-camera-module )
- tetris11 2y agoI'm always a little perplexed by the world of microcontrollers. How would you program this without having some kind of embedded linux? And where does the OS live in this modules? Or does this sit on a Pi?
- deleted 2y ago[deleted]
- markus_zhang 2y agoIn many cases there is no OS, just bare metal. I have dabbed into embedded programming (but never really into hardware) briefly and the process looks like this: you manipulate some pins and they make things work. You read manuals to figure out which pins to use and how to manipulate them to make certain things happen. For example, to make a peripheral work, you first need to connect certain pins (following the manual), then you need to send some black magic signals to these pins to make it work in certain ways (think ROM reading/writing, LCD screen display, etc.). Reading the manual and the data sheets, IMO, is where the real complexity comes from -- and you can always use "Standard" components and use a library. Here is a good textbook: https://web.eece.maine.edu/~zhu/book/ https://web.eece.maine.edu/~zhu/book/ If you need an OS sometimes a RTOS is considered instead of Linux. Embedded Linux is pretty "heavy" in the embedded world AFAIK.
- hysan 2y agoTwo other good articles on decreasing Pi boot times are: - https://www.furkantokac.com/rpi3-fast-boot-less-than-2-seconds/ https://www.furkantokac.com/rpi3-fast-boot-less-than-2-secon... - http://himeshp.blogspot.com/2018/08/fast-boot-with-raspberry-pi.html http://himeshp.blogspot.com/2018/08/fast-boot-with-raspberry... I used these two to make a digital photo frame with a Pi that boots very quickly to a browser in kiosk mode. If you have very minimal requirements, you can get some very impressive boot times.
- rvdca 2y agoThabks for the link ! Got the code accessible for this kiosk picture frame by any chance ?
- LeonM 2y agoReading the first article it seems like OP could benefit from using start_cd.elf (3rd stage bootloader, but with the graphic subsystem removed), they report a 0.5s improvement in loading time
- deleted 2y ago[deleted]
- raggi 2y agoAssuming it stays up for about 10-15s this is a saving over staying idle of around 85%, based on the idle burn rate from toms hardware. Not bad at all!
- throw10920 2y agoI like the article as a whole, but I'm unsure about this point: > For example: Disabling CPU turbo just to save some current consumption is a bad choice, because the resulting extra time will use more energy than just getting the job done quickly and shutting off. In one of my computer engineering classes, I learned that power consumption rises as the square of clock frequency - so doubling the clock will quadruple the power. That seems like it'd imply that you'd actually have to measure the power difference to determine if the quadratic increase from the clock boost will outweigh the product of the constant power consumption with the additional time spent on the task. Related - it'd be nice if the Pi's CPUs included granular power consumption information, either derivable from the datasheet, or as real-time values exposed in registers.
- naming_the_user 2y agoThis is very interesting, thanks for sharing! So if it takes 1J to do some computation in 1 second (say 1GHz at 1W), you're saying that in the perfectly spherical cow case, it takes 2J to do that same computation in 0.5 seconds (2GHz at 4W). However, that's just CPU consumption, if the overall system has a static rate of 4W, then it takes 5J (1J CPU, 4J system) at 1Ghz to do the task in a second, or 4J (2J CPU, 2J system) at 2GHz to do the task in 0.5 seconds. Am I understanding you correctly? Basically, if the overall system's power consumption is similar to the CPU's power consumption at turbo, then it makes sense to turbo, if not, it doesn't?
- throw10920 2y agoYes, this is all correct, as long as you're implicitly assuming that the CPU itself has some static power dissipation as well (which it does) in addition to the rest of the system. Unfortunately I missed the actual benchmarks in the article that empirically measured the power difference.
- bjackman 2y agoI think you have it right but also my experience from optimising Android power usage was: your your intuitions are helpful for knowing what to try, but you have to test and measure everything as there are always complications. Luckily you are well equipped to benchmark it already :)
- kfarr 2y agoIf you like Rasp Pi ecosystem you might want to try the Pi Pico W, it's similar in spirit to microcontrollers like ESP32 but allows you to use micropython and has a neat set of peripherals that work "out of the box": https://shop.pimoroni.com/products/raspberry-pi-pico-w?variant=40059369619539 https://shop.pimoroni.com/products/raspberry-pi-pico-w?varia...
- zelo 2y agomicropython supports esp32 too
- arendtio 2y agoI wondered why a custom kernel came so late. If you want to optimize, wouldn't you start with LFS or some source-based distribution? Autonomous software updates don't seem to be a necessity anyway on such a device. In addition, I wonder if it would be possible to optimize the EFI/BIOS on such a device. At least on my standard Arch Linux desktop, it takes a significant amount of boot time: $ systemd-analyze Startup finished in 10.076s (firmware) + 1.339s (loader) + 1.569s (kernel) + 2.974s (initrd) + 3.894s (userspace) = 19.854s
- daalex 2y ago> I wondered why a custom kernel came so late. If you want to optimize, wouldn't you start with LFS or some source-based distribution? Autonomous software updates don't seem to be a necessity anyway on such a device. Buildroot (which they used) is made exactly for this. With buildroot, you configure your own "Distribution" and generate a single bootable image from it. > In addition, I wonder if it would be possible to optimize the EFI/BIOS on such a device. At least on my standard Arch Linux desktop, it takes a significant amount of boot time: Not exactly sure about raspberry pi hardware, but a lot of other embedded SoCs have a pretty minimal bootloader that runs with u-boot, which is typically very fast (at least if you set the delay it waits for user input to 0)
- creatonez 2y ago> wouldn't you start with LFS or some source-based distribution You don't ever want to actually use LFS (the manual from the LFS project) in the real world as compiling GNU is far too much work. A minimalistic kernel + busybox system is much less pain. But Gentoo would not be a bad option too.
- tetris11 2y agochrist, you've just shown me that I need to optimize my boot loader (systemd-boot), and how great apparently my firmware is. > systemd-analyze Startup finished in 3.259s (firmware) + 35.127s (loader) + 1.823s (kernel) + 2.927s (userspace) = 43.138s
- hcfman 2y agoLovely project
- stereo 2y agoIs the Pi connected to the network with a static IP? Getting a fresh one from DHCP can, in this context, take quite a bit of time and energy.
- merpkz 2y agoWhy did I always had impression that decompressing data is much faster than reading inflated data off disk? Like, if you need to read just 5MB and decompress it would take less time than just to read 10MB off a disk, for example, but this article kinda states the otherwise.
- olex 2y agoThe article states a "net-positive energy result", not necessarily a faster time for this specific optimization. They say GZIP decompression is energy-intensive, so while the combination of read + decompress may be faster, the CPU load during decompression and memory relocation of the decompressed data ultimately consumes more energy than reading an uncompressed kernel and running it directly.
- erinaceousjones 2y agoMight've been the case with "spinning rust" (hard drives) but solid state storage can have lower access and read times -- no need to wait for a disk to spin up or move read heads to the right position on a platter etc
- saagarjha 2y agoThis is actually something that flip-flops across hardware generations and platforms. Hard drives used to be really slow and consumer machines would often reduce data bandwidth by compressing things because processors were “faster” than disk. But today’s SSDs are actually really fast, sometimes so fast that CPUs can barely keep up just processing the data coming off of them, so the balance can also shift in the opposite direction. And in embedded your storage might be slow but you may not have the processing power to spare decompressing. Or maybe you do and it saves on flash write cycles. This is a complicated topic!
- rcarmo 2y agoImpressive. But every time I read one of these pieces I remember when I recorded Plan 9 booting on a Pi Zero: https://taoofmac.com/space/blog/2020/09/02/1900#resurrecting https://taoofmac.com/space/blog/2020/09/02/1900#resurrecting (GIF is real time output).
- fransje26 2y agoNice in its own way. But will that allow OP to load the camera and wifi drivers he needs for his project?
- turblety 2y agoThe real tragedy is the proprietary bootcode.bin gpu code that is a blackbox and we don't have the source code for. How horrible that a tinkering/hobbies project has to have these hidden secret blackboxes that can't be modified.
- pandemic_region 2y agoI'm guessing that having the source of the bootcode available would allow for extreme tinkering to the point that RPI can no longer guarantee it functioning properly? Or maybe it has something to do with proprietary drivers loading? Curious as well what's in there that they need to keep it closed source.
- pta2002 2y agoWell, you can already do extreme tinkering to the point that RPi can no longer guarantee it functioning properly :) IIRC it's just that the bootcode.bin file is provided by Broadcom, and not the RPi foundation, so they can't open source it because they don't have the license to do so (this isn't the only proprietary blob in the default Pi distribution, but most of the other ones have open source alternatives/aren't that necessary/are open source now that they use the RP1 chip instead of broadcom peripherals). There's similar, slightly more open arrangements, with the Pi Pico W, where they can't provide the firmware for the Wifi chip, but they can provide a library to interface with it, with the caveat that the license _only_ allows for that library to be used with the RP* family of microcontrollers [1] [1]: https://github.com/georgerobotics/cyw43-driver/blob/faf36381bad1f668a30172b6336c9a970966ef4c/LICENSE.RP https://github.com/georgerobotics/cyw43-driver/blob/faf36381...
- sydd 2y agoEhm instead of spending like weeks on this why not use a hardware that is meant for such applications like an ESP32?
- willyt 2y agoAccording to the back of my jar of mayonnaise there is 8400kJ stored in it, enough energy to power this rpi for ~62 days. This is probably a stupid question but just out of curiosity, why do people express electrical energy in Watts per second or Watts per hour instead of Joules. Unless school physics has deserted me completely 1 Ws = 1J no?
- omikun 2y agoBecause no one talks about units of Ws. It's all Wh or kWh. Or they talk about power instead of energy.
- willyt 2y agoWell you didn't read the article because the comparisons of the energy consumed are listed in Ws (Watt Seconds) e.g. > We can now boot into a Linux user space program in less than 3.5s! >~400ms is spent in the Linux kernel (difference between pin 0 and pin 1) >Total energy consumption: 0.364 As * 5.0 V = 1.82 Ws
- jakogut 2y agoYou can still save quite a bit by bundling your application in an initramfs linked into the kernel, which obviates the needs for any filesystem mounts in simple cases. In some cases, you can even replace something like BusyBox init with a simple bash script that does the bare minimum to boot your application. Mounting devtmpfs, proc, sysfs, etc. Dumping glibc is also worth exploring, if feasible. Chroot is a good tool to test your initramfs and see if all the necessary application dependencies are present before bundling it into the kernel. If you can run it in a chroot, the kernel can run it during boot, and the development loop is much tighter. Disabling kernel modules and enabling only the features needed linked into the kernel will save further space and boot time. It would also be helpful to test zstd compression instead of gzip.
- rubenbe 2y agoYou actually don't need a shell script to mount the different pseudo filesystems. You can do that in your application. So all that remains is an initramfs with a statically linked binary.
- jakogut 2y agoVery true, you can call into the C library or use system calls directly and have your application do all the init itself.
- amluto 2y agoOn the flip side of this, if your kernel + initramfs is being loaded slowly (either the previous boot stage and is not using the hardware at its full capacity or the image is large enough that it would be better to do something else in parallel with loading the image), then having the smallest practical image that can load the remaining software after userspace starts can be faster.
- jakogut 2y agoAbsolutely, there's a point of size and complexity where trying to bundle everything into an initramfs image is counter-productive. One example of loading the kernel and initramfs slowly is netbooting over TFTP. You're better off with a smaller kernel that can load the rest of the rootfs over a faster protocol, i.e. NFS, NBD, iSCSI, etc. Alternatively, you can load a bootloader that supports a faster protocol, such as GRUB, which can load the kernel binary over HTTP.
- tuatoru 2y agoThree seconds? A purpose built trail cam is considered slow if it takes 0.7 seconds to boot up and take a picture. 0.15 s is the going rate these days.
- exabrial 2y agoOk decreasing the regulator voltage was a real surprise! I thought switching regulators would be far more efficient at higher voltages! (Less current = less heat)
- manawyrm 2y agoSame. I really didn't expect that (and thus didn't even test it at first).
- 91010109 2y agoWhy not just boot baremetal, something like rsta2/circle?