4 ms·

I've used buildroot extensively, and an occasional yockto. My impression in recent years is that these image cross build environments are just not as frequentl

by johnea 4mo ago

I've used buildroot extensively, and an occasional yockto. My impression in recent years is that these image cross build environments are just not as frequently needed as they were back in the day of their invention. My most recent embedded linux environments were just embedded archlinux. No need to cross build an image, just install and run the minimized linux environment right on the target. Of course, a big part of the need for these cross-tools is that it seems most modern embedded linux developers are running windoze on their development workstations 8-/

xyzzy_plugh 4mo ago
Are you proposing compiling on the target? For a vast number of embedded systems that is not only impossible (insufficient disk and memory) but also incredibly slow. At the end of the day you need some cross compilation just for board bring up. If you're playing with some platform for which this has already been done, then sure, but that's not really the "normal" way of doing embedded.
- KingMachiavelli 4mo ago
  Embedded just means ARM 99% of the time and it's cheaper and easier to use native ARM servers (AWS has them cheap) than to make 100% of software cross compile. Some parts of the firmware might need to be cross compiled but those projects are designed to cross compile.
  - okanat 4mo ago
    Your target build environment often differs significantly from your server environment, so you end up needing a different toolchain and all of the problems that come with cross-compilation anyway. The toolchain and ABI settings that produce small, battery or instruction cache efficient are usually not you want on servers.
    - nrdvana 4mo ago
      I'm not familiar with the problems that would cause "target build environment often differs significantly from your server environment". If you have an ARM server, then you should be able to download and run Alpine docker images, and then fetch most system utilities from the Alpine package repo and then compile a kernel and a few other binaries on top of that, then export an image. The main reason to not run Debian is that Debian usually makes lots of compile-time decisions for you, and chooses to maximize the enabled options of the software. Alpine makes the opposite choices for you, creating very minimal feature sets. It's been working well for me, but my use cases is a lot more like a custom in-house DDWRT firmware kind of a thing. If you've experienced additional constraints that make this pattern unworkable, I'd be interested to hear about them.
    - structural 4mo ago
      Run on server, chroot into target environment, configure compiler flags for eventual target, done. It's really just an afternoon of setup and then the problem is solved.
    - KingMachiavelli 4mo ago
      My toolchain for both is managed by the Nix derivation wrappers. It’s trivial to set custom C/C++/Rust/Go/NodeJS flags. As long as I have one flexible build tool, I can customize it for both server and embedded.
- johnea 4mo ago
  I'm talking about cross building an image, and flashing an entire image onto the target. This is the purpose of environments like yocto and buildroot. My preference is to cross build packages for each application on my workstation, and maintain a local package repository from which individual packages are installed on the target. I find archlinux extremely useful in this configuration, because even if I'm not building on the target, the same package maintenance and other system commands are identical to my development workstation. Additionally, archlinux provides a well documented, and fairly simple process for hosting your own local repositories.
- cbrake 4mo ago
  There are several ways to compile for the target: 1) Cross compile. Yoe build still does this where it makes sense. (Go apps, likely kernel builds in the future) 2) QEMU user mode - 5-20x slow, but fine for some things. 3) Run yoe-build on any ARM machine (AWS, rPI5, Jetson) 4) Farm unit builds out to runners on cloud ARM machines. (future) The yoe-build architecture allows for this. Choose the container, host architecture, and location that makes the most sense on a unit-by-unit basis. Caching is also part of the vision, so we never rebuild something twice, unless it changes.
  - nickthenerd 4mo ago
    Doing this on target seems terrible. What an awful development experience.
    - cbrake 4mo ago
      Agreed, building on the typical target is a terrible idea. None of my suggestions are to build on target, but rather specific units on some capable hardware of the same architecture. The main build will still run on your own workstation of any architecture. Only specific units get farmed out to another machine, but worst case can be run in QEMU user mode emulation if there is no native machine runner available. This remote build runner is not implemented yet, just ideas.

by johnea 4mo ago

xyzzy_plugh 4mo ago

Are you proposing compiling on the target? For a vast number of embedded systems that is not only impossible (insufficient disk and memory) but also incredibly slow. At the end of the day you need some cross compilation just for board bring up. If you're playing with some platform for which this has already been done, then sure, but that's not really the "normal" way of doing embedded.

KingMachiavelli 4mo ago
Embedded just means ARM 99% of the time and it's cheaper and easier to use native ARM servers (AWS has them cheap) than to make 100% of software cross compile. Some parts of the firmware might need to be cross compiled but those projects are designed to cross compile.
- okanat 4mo ago
  Your target build environment often differs significantly from your server environment, so you end up needing a different toolchain and all of the problems that come with cross-compilation anyway. The toolchain and ABI settings that produce small, battery or instruction cache efficient are usually not you want on servers.
  - nrdvana 4mo ago
    I'm not familiar with the problems that would cause "target build environment often differs significantly from your server environment". If you have an ARM server, then you should be able to download and run Alpine docker images, and then fetch most system utilities from the Alpine package repo and then compile a kernel and a few other binaries on top of that, then export an image. The main reason to not run Debian is that Debian usually makes lots of compile-time decisions for you, and chooses to maximize the enabled options of the software. Alpine makes the opposite choices for you, creating very minimal feature sets. It's been working well for me, but my use cases is a lot more like a custom in-house DDWRT firmware kind of a thing. If you've experienced additional constraints that make this pattern unworkable, I'd be interested to hear about them.
  - structural 4mo ago
    Run on server, chroot into target environment, configure compiler flags for eventual target, done. It's really just an afternoon of setup and then the problem is solved.
  - KingMachiavelli 4mo ago
    My toolchain for both is managed by the Nix derivation wrappers. It’s trivial to set custom C/C++/Rust/Go/NodeJS flags. As long as I have one flexible build tool, I can customize it for both server and embedded.
johnea 4mo ago
I'm talking about cross building an image, and flashing an entire image onto the target. This is the purpose of environments like yocto and buildroot. My preference is to cross build packages for each application on my workstation, and maintain a local package repository from which individual packages are installed on the target. I find archlinux extremely useful in this configuration, because even if I'm not building on the target, the same package maintenance and other system commands are identical to my development workstation. Additionally, archlinux provides a well documented, and fairly simple process for hosting your own local repositories.
cbrake 4mo ago
There are several ways to compile for the target: 1) Cross compile. Yoe build still does this where it makes sense. (Go apps, likely kernel builds in the future) 2) QEMU user mode - 5-20x slow, but fine for some things. 3) Run yoe-build on any ARM machine (AWS, rPI5, Jetson) 4) Farm unit builds out to runners on cloud ARM machines. (future) The yoe-build architecture allows for this. Choose the container, host architecture, and location that makes the most sense on a unit-by-unit basis. Caching is also part of the vision, so we never rebuild something twice, unless it changes.
- nickthenerd 4mo ago
  Doing this on target seems terrible. What an awful development experience.
  - cbrake 4mo ago
    Agreed, building on the typical target is a terrible idea. None of my suggestions are to build on target, but rather specific units on some capable hardware of the same architecture. The main build will still run on your own workstation of any architecture. Only specific units get farmed out to another machine, but worst case can be run in QEMU user mode emulation if there is no native machine runner available. This remote build runner is not implemented yet, just ideas.