5 ms·

I used buildroot since 2011 and yocto since 2019. Both have their strengths, and their weaknesses, but I actually prefer yocto now since it is more predictable

I used buildroot since 2011 and yocto since 2019. Both have their strengths, and their weaknesses, but I actually prefer yocto now since it is more predictable in what it builds. Buildroot is faster to launch and run, but I'm infinitely more confident that everything will build again after cleaning with Yocto than I am with buildroot because Yocto creates a sysroot for each recipe which contains only the specified dependencies, unlike buildroot which contains whatever has been installed into "target" by that point in the build process. This is even more troublesome when using multiple cores to build since the state of "target" is changing rapidly based on how quickly different packages are able to be built and installed. Yocto is slow and bitbake can be a pain until you get it, but it's really an amazing thing once you understand it and can leverage what it has to offer. Layers are wonderful if you need to support multiple devices which share a common base. By using a shared state cache, I can have other developers building complete images in minutes rather than hours. I am happy that I took the time to learn how to use Yocto, even after having a good experience with buildroot.

ephaeton 6y ago
I concur with your technical argument, except for the trust of what's being built. I have repeatedly found the shared state cache to be the achilles heel of my builds. You never want to have a false-positive hit in your cache, yet yocto has repeatedly bit me with that. I have had instances where bitbake claimed it was done, just to find wrong or missing artifacts in my image. Cleaning and rebuilding from scratch is a continuous exercise that's IME sadly necessary. ...let alone letting others use my build directories! I accept yocto's right to exist, but I'm far from being happy for using it.
- fullstop 6y ago
  I don't want to be the person who says "you're doing it wrong", but you probably are doing something wrong. I've been using this for quite a while and have never had to clean the world to get things straightened out, or had a false positive in the cache. One thing that I had to do, however, was run bitbake-hashserv when running the shared state cache. Even so, this was because I was getting cache misses. If I may ask, what are you doing when it claims to be done but things aren't in a good state? If you're deleting files / directories manually instead of using bitbake commands you're going to have a bad time.
  - rcxdude 6y ago
    I've had similar issues. It's rare but a nightmare to debug when it happens. One thing which consistently caused problems was OOM errors on compilers (fairly easy to encounter during a parallel yocto build). This was suprisingly effective at poisoning the state of the system (I'm not sure why this compared to any other failure was so problematic).
kevin_b_er 6y ago
> Yocto creates a sysroot for each recipe which contains only the specified dependencies It didn't used to. This was a major improvement in how yocto acts when it was added.
nrclark 6y ago
I've worked at a Yocto shop, and I currently work at a Buildroot shop. Between the two of them, I like Yocto much, much better. Yocto (or OpenEmbedded if ya nasty) is a much more complicated beast, for sure. But that's a feature, not a bug. I've hit Buildroot's limits a whole bunch of times. It's messy, especially if you're trying to support a family of designs from a single repo. Right off the bat, here are a few of the things Yocto does really well: 1. File dependencies. With Buildroot, I have to intentionally clean targets all the time. I can't just modify a source file somewhere and expect Buildroot to know what needs to be rebuilt when I run `make all`. Yocto has really good file-level dependency tracking, and will almost always do the right thing if your dependencies are correctly expressed. It's hard to include a source in Yocto without also tracking it in the dependency graph. 2. Hardware variants. Buildroot's kconfig system is nice for managing a single rootfs for a single piece of hardware. But as soon as you want to share configuration across multiple boards (some with different packages), it kind of blows up. All of a sudden, you're using a preprocessor to generate your defconfig from fragments, and you can't reliably use `make savedefconfig` to guarantee that you aren't missing package/config dependencies. And you also can't use `make menuconfig` any more, which is kind of the whole point of using kconfig in the first place. 3. License tracking. Yocto is really strict about licensing. It forces you to specify the license of every package, and also forces you to specify a file that it can check to make sure the license doesn't change unexpectedly. When building an image, you can easily generate license reports for every piece of software in the rootfs. You can also do things like forcing it to exclude GPLv3 software. 4. Toolchains in the rootfs. For some dumb reason, Buildroot doesn't give you recipes for building a compiler toolchain and including it in your rootfs. This is trivial in Yocto. If you support developers that like to work on-target over SSH, it's a big limitation for Buildroot. 5. Scripting. If you want to do something fancy in a package recipe (or in an image recipe), it's way easier in Yocto. You can run arbitrary Python code at parse-time, and modify your build's internal metadata if you want. Each recipe can also have arbitrary build steps that you can trigger manually if you want (or hook into the dependency graph). 6. Layers. Yocto lets you put all of your customizations in their own folder, which can be in a separate folder from the upstream sources. You can extend, override, or modify the behavior of any upstream recipe, and source-control your tweaks without having to vendor your own copy of the upstream. This is really nice for upgrading Yocto releases, because it's usually as simple as "swap out the upstream submodules, then tweak your custom bbappends (if any)". You can have as many layers as you want, and you have full control over their evaluation order. Lots of hardware vendors distribute their BSPs as a Yocto layer. 7. Packages. Yocto builds every package as a .deb (or a .rpm, or a .ipk depending on how you set up the build). This makes it very easy to build a package on your dev machine, then install it on-target if you're experimenting with some new package or you want to install $tool one time for a test.
rcxdude 6y ago
yocto and bitbake are basically slightly earlier and less formal versions of the same ideas as in nix and nixos, and they do suffer a bit from that. Given that, I would actually be very interested in a version of nixos focused on targeting embedded systems.

ephaeton 6y ago

I concur with your technical argument, except for the trust of what's being built. I have repeatedly found the shared state cache to be the achilles heel of my builds. You never want to have a false-positive hit in your cache, yet yocto has repeatedly bit me with that. I have had instances where bitbake claimed it was done, just to find wrong or missing artifacts in my image. Cleaning and rebuilding from scratch is a continuous exercise that's IME sadly necessary. ...let alone letting others use my build directories! I accept yocto's right to exist, but I'm far from being happy for using it.

fullstop 6y ago
I don't want to be the person who says "you're doing it wrong", but you probably are doing something wrong. I've been using this for quite a while and have never had to clean the world to get things straightened out, or had a false positive in the cache. One thing that I had to do, however, was run bitbake-hashserv when running the shared state cache. Even so, this was because I was getting cache misses. If I may ask, what are you doing when it claims to be done but things aren't in a good state? If you're deleting files / directories manually instead of using bitbake commands you're going to have a bad time.
- rcxdude 6y ago
  I've had similar issues. It's rare but a nightmare to debug when it happens. One thing which consistently caused problems was OOM errors on compilers (fairly easy to encounter during a parallel yocto build). This was suprisingly effective at poisoning the state of the system (I'm not sure why this compared to any other failure was so problematic).

kevin_b_er 6y ago

> Yocto creates a sysroot for each recipe which contains only the specified dependencies It didn't used to. This was a major improvement in how yocto acts when it was added.

nrclark 6y ago

I've worked at a Yocto shop, and I currently work at a Buildroot shop. Between the two of them, I like Yocto much, much better. Yocto (or OpenEmbedded if ya nasty) is a much more complicated beast, for sure. But that's a feature, not a bug. I've hit Buildroot's limits a whole bunch of times. It's messy, especially if you're trying to support a family of designs from a single repo. Right off the bat, here are a few of the things Yocto does really well: 1. File dependencies. With Buildroot, I have to intentionally clean targets all the time. I can't just modify a source file somewhere and expect Buildroot to know what needs to be rebuilt when I run `make all`. Yocto has really good file-level dependency tracking, and will almost always do the right thing if your dependencies are correctly expressed. It's hard to include a source in Yocto without also tracking it in the dependency graph. 2. Hardware variants. Buildroot's kconfig system is nice for managing a single rootfs for a single piece of hardware. But as soon as you want to share configuration across multiple boards (some with different packages), it kind of blows up. All of a sudden, you're using a preprocessor to generate your defconfig from fragments, and you can't reliably use `make savedefconfig` to guarantee that you aren't missing package/config dependencies. And you also can't use `make menuconfig` any more, which is kind of the whole point of using kconfig in the first place. 3. License tracking. Yocto is really strict about licensing. It forces you to specify the license of every package, and also forces you to specify a file that it can check to make sure the license doesn't change unexpectedly. When building an image, you can easily generate license reports for every piece of software in the rootfs. You can also do things like forcing it to exclude GPLv3 software. 4. Toolchains in the rootfs. For some dumb reason, Buildroot doesn't give you recipes for building a compiler toolchain and including it in your rootfs. This is trivial in Yocto. If you support developers that like to work on-target over SSH, it's a big limitation for Buildroot. 5. Scripting. If you want to do something fancy in a package recipe (or in an image recipe), it's way easier in Yocto. You can run arbitrary Python code at parse-time, and modify your build's internal metadata if you want. Each recipe can also have arbitrary build steps that you can trigger manually if you want (or hook into the dependency graph). 6. Layers. Yocto lets you put all of your customizations in their own folder, which can be in a separate folder from the upstream sources. You can extend, override, or modify the behavior of any upstream recipe, and source-control your tweaks without having to vendor your own copy of the upstream. This is really nice for upgrading Yocto releases, because it's usually as simple as "swap out the upstream submodules, then tweak your custom bbappends (if any)". You can have as many layers as you want, and you have full control over their evaluation order. Lots of hardware vendors distribute their BSPs as a Yocto layer. 7. Packages. Yocto builds every package as a .deb (or a .rpm, or a .ipk depending on how you set up the build). This makes it very easy to build a package on your dev machine, then install it on-target if you're experimenting with some new package or you want to install $tool one time for a test.

rcxdude 6y ago

yocto and bitbake are basically slightly earlier and less formal versions of the same ideas as in nix and nixos, and they do suffer a bit from that. Given that, I would actually be very interested in a version of nixos focused on targeting embedded systems.