10 ms·
Gokrazy: A pure-Go userland for Raspberry Pi 3 appliances
- dom96 9y agoI'm having a really hard time figuring out what problem this project solves, can anyone explain?
- leaveyou 9y agoThey say it right in the header: "For a long time, we were unhappy with having to care about security issues and Linux distribution maintenance on our various Raspberry Pis."
- kiallmacinnes 9y agoYep - for home automation style stuff (which the author seems to have built it for - see the showcase section), having a full Linux distro is actually a real pain. I've taken to building my own stuff with the ESP series of microcontrollers, home made PCBs, etc - once it's built - it just keeps chugging along, unlike my original stuff I built atop RaspberryPis with Linux.
- alexose 9y agoI've done a lot of this sort of this as well. The ESP8266 is so great for simple on/off switches and sensors. But-- there are so many applications for having a full IoT Linux distribution that I can't believe this isn't already a solved problem. I'm imagining an ultra-stable distro that is easy to deploy to a headless machine, handles security updates, handles flaky wi-fi issues, tests the SD card (I've had more than a few burn out on me), and automatically logs to the cloud for easy debugging. In other words: Why am I still maintaining my RasPis as if they are apache servers from the 1990s?
- petra 9y agoAndroid-things solves this problem at the scale of embedded linux, and mbed solves this problem at the microcontroller level.
- alexose 9y agoI'm a wary of Android only in that I'm so used to the Raspian ecosystem, but I will check it out. Thanks for the recommend!
- halbritt 9y agoResinOS does all that and more. It's container-based. Works super-well on a Pi3.
- alexose 9y agoOh, this is awesome. Can't wait to try it. Thank you!
- justincormack 9y agoLinuxkit is another option, slightly different approach.
- boromi 9y agoThanks. Where would alpine linux fit into this all. It seems like a good candidate for embedded setups.
- halbritt 9y agoYou could run whatever you like in an alpine container on ResinOS
- nickpsecurity 9y agoPeople have been building stuff like this. Often for firewall appliances (eg SCC Sidewinder, Immunix Linux, genua w/ OpenBSD). They usually do one of these: (a) get big under new management that doesnt care about either security or the little people (enterprise focus); (b) sell out to company like that; (c) go bankrupt cuz market values assurance of security the least. Sad but true. There's still a few companies selling microkernel-based solutions in defense and mobile with Linux user-lands. They'd OEM license their stacks for a new set of products aimed to do this. They cost a fortune. Whoever pays it wont get it back, though, cuz folks dont usually pay for security in that volume. So, nobody is doing it for small players for now.
- dom96 9y agoWhat is it about Linux that doesn't keep your stuff chugging along?
- IshKebab 9y agoYou have to keep it up to date because of all the security issues in other apps that are installed. If the system only contains the Linux kernel and your software you only have to worry about your security issues (unavoidable) and Linux security issues (also difficult to avoid - too much work to write a new kernel too, plus remote Linux vulnerabilities are quite rare).
- dom96 9y agoSo why not just create a stripped down Linux distribution instead of reinventing the wheel and creating a new userland from scratch (with probably a few brand new and undiscovered security issues already)?
- secure 9y agoI tried: I have used debootstrap to create minimal Debian images in the past. A big part of the motivation for gokrazy is to use a safe, high-level programming language. This allows us to re-implement the userland with small amounts of code, see e.g. the NTP client at https://github.com/gokrazy/gokrazy/blob/master/cmd/ntp/ntp.go https://github.com/gokrazy/gokrazy/blob/master/cmd/ntp/ntp.g... Replacing the implementation language eliminates whole classes of attacks, while stripping down an existing Linux distribution can only go so far. Plus, it’s not clear to me how one would maintain such a stripped-down distribution in a way that isn’t a lot of effort.
- dom96 9y agoI still find this effort odd. You already have a well tested and widely used userland, even if it is written in a non memory safe language. The easiest solution to me seems to be to grab the tools you need from the existing userland, instead of reimplementing them in Go. Currently you have reimplemented some of the userland, and packaged it up into a distro. So you have to maintain a stripped down distribution and the tools that you reimplemented. If you just took existing tools you would only have to do the latter. Am I missing something here?
- elliotec 9y agoSo it’s like it’s own OS almost? What’s a userland?
- yoavm 9y agoYes, it's pretty much the OS except for the kernel.
- mulmen 9y agoUserland is the part of the OS that is not the kernel. It is the part you actually interact with. https://en.wikipedia.org/wiki/User_space https://en.wikipedia.org/wiki/User_space
- AYBABTME 9y agoIt's everything the kernel doesn't do. In general, that means the surface with which you interact as a user. So if you're used to using a terminal, your SSH connection, your Bash session, are all userland programs (that use kernel features to do things). You can think of it as kernel is a framework like Rails and userland stuff is developers using and expending the interfaces... sort of. When your computer runs headless, the init/process tree system is userland. Background (cron) jobs are userland, daemons are userland. Etc.
- candiodari 9y agoSo it's a distribution ? ... that will have security issues, require updates, distribution maintenance, and all that stuff they claim is unacceptable ?
- secure 9y agoIt’s not a full-blown distribution; it doesn’t distribute third-party software. Instead, it allows application authors to build their own distribution (at which point “appliance” is a more fitting word than “distribution”). Updates are easy to fully automate with gokrazy (and I’ve been using that mechanism for a year at this point) because assembling a new image and installing it over the network is just one simple command. New kernel and firmware versions are provided automatically once they are verified to boot on real hardware. There is no maintenance aspect in the actual installations (i.e. you don’t need to SSH into them, ran commands to update, etc.) — the root file system is read-only, and an update overwrites it completely. Unless the applications you want to use with gokrazy introduce state, gokrazy defaults to being stateless. Hope that clarifies things, let me know if you have further questions.
- deleted 9y ago[deleted]
- velodrome 9y agoThis is a repost from last year: https://news.ycombinator.com/item?id=13792287 https://news.ycombinator.com/item?id=13792287
- bringtheaction 9y ago"Reposts" are fine. Better to formulate it as previous discussion.
- simmons 9y agoThis is great! I think it could have uses far beyond the Raspberry Pi. As someone who's worked in the consumer electronics industry for years, I think we desperately need to rethink our approach to Linux-based electronics with respect to both the userland and the primary application. A legacy-free framework written in a safe language could go a long way to improving our currently lousy reputation with respect to security and reliability. (Yes, fresh code will have fresh bugs even in a safe language -- but it's a start!) I'm also keeping my eyes open for similar ideas using Rust.
- hugelgupf 9y agoShameless self-promotion: also take a look at https://github.com/u-root/u-root https://github.com/u-root/u-root :) (The gokrazy and u-root people all work at Google, I just contacted them to see if we can help each other. Not a competing project.)
- thisacctforreal 9y agoAny relation to https://github.com/u-boot/u-boot https://github.com/u-boot/u-boot ?
- sigjuice 9y agoDoesn't look like it. U-boot is a boot loader for PowerPC/ARM/MIPS/<other CPUs> boards. A typical Linux-based embedded system would consist of U-Boot, the Linux Kernel and a filesystem. u-root would be one of the many ways to construct this filesystem.
- hugelgupf 9y agoNo, what sigjuice said is right. The name was derived off u-boot, though, and u-root can be used (and is in use) with LinuxBoot.
- pjmlp 9y agoYeah, the idea is great. There are a few Basic, Pascal, Oberon and Java (not Android related) options available, but only a minority cares for them. Still, big enough to keep those vendors doing business. Having a minimal bare metal runtime for Go would also be nice, but it is a gigantic effort to make it portable across all types of hardware.
- sigjuice 9y agoIt is really interesting that Gokrazy uses an upstream kernel (with a couple of minor patches). https://github.com/gokrazy/kernel/blob/3ecdc901da51c2c5b6bf7ab003bf764ee4b95426/cmd/gokr-build-kernel/build.go#L15 https://github.com/gokrazy/kernel/blob/3ecdc901da51c2c5b6bf7...
- colemickens 9y agoWhy is that interesting? Just that they're not having to carry special patches for the RPi3 or what?
- technofiend 9y agoEven the raspberry Pi Zero and Zero W are supported in the latest kernels, and those were the latest Pi boards until the B+ came out.
- Ixiaus 9y agoWhy is Go considered safe?
- secure 9y agoGo is memory-safe. See also https://en.wikipedia.org/wiki/Memory_safety https://en.wikipedia.org/wiki/Memory_safety
- Ixiaus 9y agoAre there not many statically typed, compiled languages that ship a runtime with a garbage collector? Why is Go a good choice here? Why is this choice better than a language that provides memory safety using linear types so you don't even need a runtime or garbage collector?
- secure 9y agoYou can read more about why I like Go at https://michael.stapelberg.de/posts/2017-08-19-golang_favorite/ https://michael.stapelberg.de/posts/2017-08-19-golang_favori.... Of course I used my favorite programming language for building gokrazy :).
- pknopf 9y agoHey, the creator of i3! I love your stuff! I enjoyed your episode on Go Time as well! Cheers!
- secure 9y agoGlad you like my work; thanks for the kind words!
- crispinb 9y agoI've only dipped into Go recently (and not for any production code) but your article very nicely summarises my impressions with it so far. I've found a kind of fluid pleasure in writing Go that wouldn't be apparent from a bullet list of its (lack of) features. It really allows my attention to fall mainly on the problem domain in a way few other languages afford. I can't resist while here thanking you for i3. I've been a mac user recently, but will likely be back to linux in the next year or so. Using i3 again will be probably be the greatest pleasure involved in that return ;)
- shurcooL 9y ago> The motivation is that @stapelberg spends way more time on C software and their various issues than he would like. Hence, he is going Go-only where feasible. This is so cool! I’m very happy to learn an effort like this exists, and looking forward to perusing the code for it... in Go (a language I really enjoy reading).
- jononor 9y agoSuper nice. Glad to see A/B partiton updating scheme, that is the way to go on appliance / embedded device. Want something like this for one of the languages that I know... Node.js, Python or Rust. I'd include C++11, though for this kind of deployment method I think one needs packages easily available in a way separate from Linux distro (since there is none).
- londons_explore 9y agoThe A/B updating scheme is a massive waste of space in most implementations... In many hardware devices, flash storage space for the OS is actually one of the most expensive components, and one of the main reasons that many devices run little hard-to-program embedded OS's rather than linux or something else high level. Instead, a filesystem like btrfs should be used which can give the appearance of multiple versions, yet only use actual storage space where versions differ. I'd prefer the root filesystem to be something using a FUSE-like layer which can store format specific binary diff/patches between versions, and then transparently apply (and cache) them on access. Taking it a step further, I wish desktop linux distros had a package-manager-fs, which is a readonly virtual filesystem which displayed all the files installed by all packages, but the underlying real filesystem only contained all the original compressed packages and a cache of recently used files.
- jononor 9y agoSure it is quite inefficient. Both in storage and in transmission of updates. But it is very simple and robust. What one tests pre rollout is identical to what will go on production devices, down to the byte. Each device with same version is identical. It can be cryptographically verified easily. Automatic rollback can be performed on update fail, including self-checks. This approach of shipping a custom userland without any of the generic Linux userland should drastically bring down the size.
- secure 9y agoI strongly agree regarding the simplicity! A minimal gokrazy root file system is 16MB in size. On a 8GB SD card, I certainly am willing to sacrifice another 16MB to get A/B updates :).
- cjdell 9y agoCould/should one use this to build a touch screen user interface for an RPi as part of a kiosk based application? I really like Go, and suspect a Go app would beat the pants off anything powered by Electron (performance wise).
- secure 9y agoI’d love to see this, but note that you’ll need to use a pure-Go graphical toolkit (or implement all graphics yourself), and likely interface the touchscreen yourself, too. If you’re not comfortable with this, it might be better to use something else.
- tmzt 9y agoMaybe a Flutter port would make sense here?
- Kabukks 9y agoLooks great. I'd love to use it. How is the GPIO/PWM support?
- secure 9y agoHave a look at https://periph.io/ https://periph.io/. In case any of periph.io’s features require changes to the kernel configuration, I’m happy to change it accordingly.
- tigeba 9y agoDoes anyone know of a good tool for building custom PI distributions based on the Raspbian / Raspbian Lite or other distributions that does not involve forking the official image builder and hacking it up? My ideal tool would allow me to do this with a Dockerfile and just export the image after the configuration is complete.