4 ms·
FreeBSD/EC2 boot performance over time
- wadefletch 2y agoWhat are the y-axis units?
- NekkoDroid 2y agoI would guess hamburgers eaten :) On a more serious note, the only thing that really makes sense would be seconds.
- defrost 2y agoThat's an impressive drop from 30 minutes in 2019 to under 10 minutes today. No, wait .. maybe that's seconds? milliseconds?
- spease 2y agoMicrots!
- cperciva 2y agoSeconds, yes. Sorry I figured that was obvious, I'll add it to the graphs I generate next week.
- pushupentry1219 2y agoThank you! Was about to ask for axis labels as well. I assumed seconds but I had some doubts because there were no labels!
- defrost 2y agoGood to hear. Seconds was my best guess .. but that doesn't make it certain for the viewer. No real drama, but FWiW I've looked at a lot of data across engineering, math, geophysics, mining, energy, etc and unlabeled data and graphs are a major annoyance.
- amiga386 2y agoIf you're taking requests, then can the Y axis start at zero, please? https://handsondataviz.org/images/14-detect/gdp-baseline-merged-annotated.png https://handsondataviz.org/images/14-detect/gdp-baseline-mer... And what is going on with that Y axis? There is no consistent spacing between the marks! https://imgur.com/HaJmTSU https://imgur.com/HaJmTSU
- cperciva 2y agoIt's a bit hard for a log axis to start at zero.
- amiga386 2y agoWell start it at 0.00001 then. There's no need for a log axis, as all the values are within 2 orders of magnitude of each other - between 1 and 100. They will fit better, and will no longer be distorted, on a linear Y axis starting from zero.
- wright08 2y agoI wouldn't bother. It is super obvious.
- nijave 2y agoPresumably seconds since I'm not sure what else would make sense here. It'd also be helpful if the y axis had consistent scale between each graph and horizontal lines are set y axis intervals Seems zfs is quite a bit faster than ufs
- pluto_modadic 2y agowow they make it impressively hard to contact them.
- cperciva 2y agome @FreeBSD.org. I figured most people reading this would know who I am.
- tpxl 2y agoI have no idea who you are, nor do I check who submits what (the merit of the argument, not who said it and all that), nor do I have a reason to know (I have no reason to follow FreeBSD at this point). I understand you do important work, but some people, me included, do other stuff. And there's a lot of people here.
- Terretta 2y agocperciva is the HN comeback of all time: https://news.ycombinator.com/item?id=35079 https://news.ycombinator.com/item?id=35079 (And folks have had since 2007 to top it.) Funny enough, that started with a "I don't know you" as well: https://news.ycombinator.com/item?id=35071 https://news.ycombinator.com/item?id=35071
- nostrebored 2y agoFixing instance types was probably wrong. You’re getting progressively legacy (and more likely to be degraded) hardware. This impacts how tightly packed the instance type as a whole is, which impacts launch instance performance
- cperciva 2y agoFixing instance types was probably wrong. Depends what you're trying to measure, I think? My goal as a FreeBSD developer is to look at FreeBSD performance -- this is both to show the improvements which have been made over the years and to alert me to any performance regressions (I generate these graphs automatically when I build the weekly snapshots). If you want to compare EC2 to other clouds you would definitely want to use the latest instance generation, of course.
- silisili 2y agoWould be nice to see how this compares to Linux, I think, for perspective.
- cperciva 2y agoI did that comparison a few years ago: https://www.daemonology.net/blog/2021-08-12-EC2-boot-time-benchmarking.html https://www.daemonology.net/blog/2021-08-12-EC2-boot-time-be... Re-running the comparison with Linux AMIs from 2024 is on my to-do list.
- andai 2y agoDang, Clear Linux boots in one second? How is it ten times faster?
- transpute 2y agoOptimized for automotive use case (max 2s to enable rear camera), https://www.phoronix.com/news/Clear-Linux-Kernel-3s-to-300ms https://www.phoronix.com/news/Clear-Linux-Kernel-3s-to-300ms > They started out with around a three second kernel boot time but cut it down to just 300 ms. Among the optimizations carried out to really speed-up their boot time were ensuring more asynchronous driver probing, only initializing a small amount of RAM at start and then after booted hot-plug the rest of it in parallel via systemd, optimized root file-system mounting, disabling unnecessary kernel modules, and similar approaches. Moving forward they are still looking at optimizations for the boot process around in-kernel deferred memory initialization, SMP initialization changes, ACPI tweaking, and user-space/systemd optimizations.
- homebrewer 2y agoIt's been possible for a long time: https://lwn.net/Articles/299483 https://lwn.net/Articles/299483
- AdieuToLogic 2y agoIs there an architectural change and/or approach to which the boot performance increase(s) can be attributed? If not, which is understandable, is there something specific to stable/14 for interested parties to familiarize themselves with?
- cbhl 2y agoMy impression is that cperciva has been working on FreeBSD/EC2 boot times for the last six years, not just specifically some changes in 14. The most accessible summary of his work is probably on his Patreon: https://www.patreon.com/c/cperciva/posts https://www.patreon.com/c/cperciva/posts. If you go digging there are also some anecdotes on his Twitter too; most recent I can find is https://x.com/cperciva/status/1833735559614988526 https://x.com/cperciva/status/1833735559614988526
- cperciva 2y agoI gave a talk at BSDCan 2022 about this ("speeding up the FreeBSD boot") -- I'm on my phone right now so I can't provide a link but it's on YouTube. There's also a somewhat out of date summary of most of the work on the FreeBSD wiki: https://wiki.freebsd.org/BootTime https://wiki.freebsd.org/BootTime
- homebrewer 2y agohttps://www.youtube.com/watch?v=10XRCiBtyhA https://www.youtube.com/watch?v=10XRCiBtyhA
- cperciva 2y agoThe best summary I can provide is "profiling and lots of small wins".
- bas 2y agoYour FreeBSD on AWS work is appreciated, @cperciva.
- Temporary_31337 2y agoThat suits our deployments where for HA we simply terminate and replace unresponsive EC2 nodes with new ones. I’ll have to talk to other developers to see how much work would it be to compile to freebsd instead of Linux
- wilted-iris 2y agoLinux probably boots faster fwiw. As a starting point, see this bench: https://www.daemonology.net/blog/2021-08-12-EC2-boot-time-benchmarking.html https://www.daemonology.net/blog/2021-08-12-EC2-boot-time-be...
- alexellisuk 2y agoNice improvements in boot speed. Perhaps a little blurb / intro / summary would be helpful to the post to help with understanding the achievements made. Did the patches ever make it into Firecracker for booting FreeBSD as a guest? I looked back at the paper trail and it seemed like it may have stalled. Does anyone know?
- cperciva 2y agoThings stalled; there were a couple outstanding bugs (one I know has now been fixed, not sure about the other) and I had to take over FreeBSD release engineering and didn't have time to follow up on the Firecracker bits. I understand that NetBSD can boot in Firecracker using the same patches so I'm hoping they can resolve any remaining issues and prod the Firecracker developers into merging things.