7 ms·

Mender – An open-source OTA software updater for embedded Linux devices

132 points by ralphmender 10y ago

currywurst 10y ago
Neat :)! In case the Mender folks are here, have you looked into incorporating the concerns addressed bt The Update Framework (TUF) https://theupdateframework.github.io/ https://theupdateframework.github.io/
- theamk 10y ago
  Why? TUF is all about reimplementing SSL and PKI. Since mender can use regular SSL with good-old PKIs, there is no reason to go with weird solutions.
  - aseipp 10y ago
    TUF protects against more attacks than just HTTPS or regular trivial signing methods do (rollback attacks, freezes, mix and match attacks, and helps secure mirrors), and has little to do with HTTPS or raw "transport layer encryption". It absolutely compliments and suppliments HTTPS if you're using it for your downloads, it is not obsoleted by it. (Though, the subtext on the introduction page probably doesn't help this impression by saying "Like the S in HTTPS...")
    - theamk 10y ago
      Well, the reason TUF has to protect against all of the attacks is because it is choosing to support a varying set of requirements, including lack of SSL and insecure mirrors. Mender simply does not care about them, so it can be dramatically simpler: - rollback attacks -- impossible since all comms are secure, and there are no untrusted mirrors - freezes -- impossible, because SSL channel must be re-negotiated every time - mix and match attacks -- nothing to mix+match, mender only does one file (rootfs) - helps secure mirrors -- mender does not support 3rd party mirrors, so no need to secure them. You can see it right on the TUF homepage: it claims to replace application, library package and system package managers. This is a lot of work, which requires a lot of complexity, and there is no need at all to pay that price if you do not need to.
- eystein 10y ago
  Thanks! :) Yes, we have looked into it and the nice thing is that TUF seems to be quite easy to add as an additional security layer down the road. One interesting challenge is downgrade attacks. How do you allow rollback of a bad deployment while disallowing an attacker to deploy an old and vulnerable version?
PanosJee 10y ago
What are the differences against Resin.io?
- gregdistefano 10y ago
  Mender is full image based, using active/passive partitions, while Resin is container based. Mender is also on a less restrictive software license
  - imrehg 10y ago
    > Mender is also on a less restrictive software license. According to the article Mender releases under Apache 2.0, and all resin.io's open source code is also on Apache 2.0, so it should be the same permissive setup. (Source: working at resin.io)
    - gregdistefano 10y ago
      my bad!
- mpasinski 10y ago
  disclaimer: I work for Mender Mender is basically full image update solution while resin is container based. Mender is fully open source, both client and server, resin is having only client open source. Mender is more lightweight, it provides a thin layer to be integrated with the already existing stack, while resin is providing full stack you need to use to be able to incorporate update mechanism.
  - karmicthreat 10y ago
    So is there any way to make Mender do local updates that are not OTA?
    - deleted 10y ago
      [deleted]
    - 10y ago
nrclark 10y ago
This is kind of a stupid question, but why not just use .rpm/dnf or .deb/apt and a custom repo?
- kristoffer 10y ago
  Embedded Linux systems supporting OTA usually employs a dual root file system (RFS) approach where the upgrade is placed onto the currently not used partition and then after successful upgrade the RFS to boot into is replaced. It is an easy solutions which ensures integrity and has few drawbacks for typical embedded systems. The output of a typical Embedded Linux CI build is a complete RFS, having to care about individual packages would just be a headache, when you can replace the entire RFS and be done with it.
  - xyzzy_plugh 10y ago
    Exactly. I wish Mender was around 4 years ago. I built exactly this!
    - padelt 10y ago
      Interesting! How did you go about falling back to an older version if the update was bad? Is there a nice way to do this automatically? Say I update to a really botched version with the kernel panicing before it reaches userland. Does this need manual intervention?
      - mpasinski 10y ago
        disclaimer: I work for Mender It is possible to make rollback fully automatic. In order to do so you need some integration with bootloader. It needs to be configured so that it can roll back to the previously working partition if update is broken. What is more, you can add some user space runtime checks that can verify the update and if those are not passing (updated image is broken) you can rollback to the previous one as well.
        10y ago
amq 10y ago
Does Mender work with mbed OS?
- amq 10y ago
  Figured the answer: no, you need Linux.
  - ingve 10y ago
    Strictly speaking you don't need Linux. As mentioned in the blog post, Mender also works with IncludeOS [0]. A demo was shown at the OpenIoT Summit last week, the video should be available in the near future. [0] http://www.includeos.org/ http://www.includeos.org/
pjmlp 10y ago
Cool! I guess an over-the-air (OTA) software updater for embedded Linux devices could be considered some kind of systems programming...
brightball 10y ago
Is Elixir's Nerves project using anything like this?
mwcampbell 10y ago
I'm curious about why you chose Yocto over buildroot for your official integration. I figured buildroot would be better, because Yocto's opkg system is superfluous on a device with full image updates.
- eystein 10y ago
  Yocto has quite large community and is growing fast. That said, think of Yocto as the first integration not the only - buildroot is surely interesting too but we had to start somewhere. :)
veli_joza 10y ago
Can someone summarize the difference between Mender and OSTree? I see that QtOTA chose OSTree as their underlying mechanism, which is significant in embedded automotive industry.
- mwcampbell 10y ago
  From my perspective as a curious observer of both projects, OSTree certainly looks attractive, because it doesn't waste space on two rootfs partitions which have to be oversized to accommodate future growth of the image. I initially thought OSTree required btrfs, because Project Atomic used btrfs the last time I looked at it. But according to the docs, while OSTree will take advantage of btrfs features if btrfs is being used, OSTree itself will work with a variety of filesystems including ext4. Edit: An upside of the alternating rootfs partition approach is that the rootfs can be cryptographically verified at the block level. Chromium OS implemented this, and CoreOS also uses that implementation. This is probably outside the scope of Mender itself, but the updating approach used by Mender enables it.
  - eystein 10y ago
    Cryptographic signing and verification is in scope for Mender [0], and frankly it should be in scope for all updaters -- too many hacks have happened due to lack of codesigning. [0] https://tracker.mender.io/projects/MEN/issues/MEN-1020 https://tracker.mender.io/projects/MEN/issues/MEN-1020
- oytis 10y ago
  If you are interested in OSTree, there is an open-source solution for OSTree updates on embedded devices. Yocto layer: https://github.com/advancedtelematic/meta-updater https://github.com/advancedtelematic/meta-updater Quickstart project with a nice tutorial: https://github.com/advancedtelematic/garage-quickstart-rpi https://github.com/advancedtelematic/garage-quickstart-rpi And yes, we have chosen it because you don't have to waste twice as much disk space, and, more importantly (for wireless networks at least), you don't have to download the whole image.
- theamk 10y ago

132 points by ralphmender 10y ago

Neat :)! In case the Mender folks are here, have you looked into incorporating the concerns addressed bt The Update Framework (TUF) https://theupdateframework.github.io/ https://theupdateframework.github.io/

theamk 10y ago
Why? TUF is all about reimplementing SSL and PKI. Since mender can use regular SSL with good-old PKIs, there is no reason to go with weird solutions.
- aseipp 10y ago
  TUF protects against more attacks than just HTTPS or regular trivial signing methods do (rollback attacks, freezes, mix and match attacks, and helps secure mirrors), and has little to do with HTTPS or raw "transport layer encryption". It absolutely compliments and suppliments HTTPS if you're using it for your downloads, it is not obsoleted by it. (Though, the subtext on the introduction page probably doesn't help this impression by saying "Like the S in HTTPS...")
  - theamk 10y ago
    Well, the reason TUF has to protect against all of the attacks is because it is choosing to support a varying set of requirements, including lack of SSL and insecure mirrors. Mender simply does not care about them, so it can be dramatically simpler: - rollback attacks -- impossible since all comms are secure, and there are no untrusted mirrors - freezes -- impossible, because SSL channel must be re-negotiated every time - mix and match attacks -- nothing to mix+match, mender only does one file (rootfs) - helps secure mirrors -- mender does not support 3rd party mirrors, so no need to secure them. You can see it right on the TUF homepage: it claims to replace application, library package and system package managers. This is a lot of work, which requires a lot of complexity, and there is no need at all to pay that price if you do not need to.
eystein 10y ago
Thanks! :) Yes, we have looked into it and the nice thing is that TUF seems to be quite easy to add as an additional security layer down the road. One interesting challenge is downgrade attacks. How do you allow rollback of a bad deployment while disallowing an attacker to deploy an old and vulnerable version?

PanosJee 10y ago

What are the differences against Resin.io?

gregdistefano 10y ago
Mender is full image based, using active/passive partitions, while Resin is container based. Mender is also on a less restrictive software license
- imrehg 10y ago
  > Mender is also on a less restrictive software license. According to the article Mender releases under Apache 2.0, and all resin.io's open source code is also on Apache 2.0, so it should be the same permissive setup. (Source: working at resin.io)
  - gregdistefano 10y ago
    my bad!
mpasinski 10y ago
disclaimer: I work for Mender Mender is basically full image update solution while resin is container based. Mender is fully open source, both client and server, resin is having only client open source. Mender is more lightweight, it provides a thin layer to be integrated with the already existing stack, while resin is providing full stack you need to use to be able to incorporate update mechanism.
- karmicthreat 10y ago
  So is there any way to make Mender do local updates that are not OTA?
  - deleted 10y ago
    [deleted]
  - 10y ago

nrclark 10y ago

This is kind of a stupid question, but why not just use .rpm/dnf or .deb/apt and a custom repo?

kristoffer 10y ago
Embedded Linux systems supporting OTA usually employs a dual root file system (RFS) approach where the upgrade is placed onto the currently not used partition and then after successful upgrade the RFS to boot into is replaced. It is an easy solutions which ensures integrity and has few drawbacks for typical embedded systems. The output of a typical Embedded Linux CI build is a complete RFS, having to care about individual packages would just be a headache, when you can replace the entire RFS and be done with it.
- xyzzy_plugh 10y ago
  Exactly. I wish Mender was around 4 years ago. I built exactly this!
  - padelt 10y ago
    Interesting! How did you go about falling back to an older version if the update was bad? Is there a nice way to do this automatically? Say I update to a really botched version with the kernel panicing before it reaches userland. Does this need manual intervention?
    - mpasinski 10y ago
      disclaimer: I work for Mender It is possible to make rollback fully automatic. In order to do so you need some integration with bootloader. It needs to be configured so that it can roll back to the previously working partition if update is broken. What is more, you can add some user space runtime checks that can verify the update and if those are not passing (updated image is broken) you can rollback to the previous one as well.
      - 10y ago

amq 10y ago

Does Mender work with mbed OS?

amq 10y ago
Figured the answer: no, you need Linux.
- ingve 10y ago
  Strictly speaking you don't need Linux. As mentioned in the blog post, Mender also works with IncludeOS [0]. A demo was shown at the OpenIoT Summit last week, the video should be available in the near future. [0] http://www.includeos.org/ http://www.includeos.org/

pjmlp 10y ago

Cool! I guess an over-the-air (OTA) software updater for embedded Linux devices could be considered some kind of systems programming...

brightball 10y ago

Is Elixir's Nerves project using anything like this?

mwcampbell 10y ago

I'm curious about why you chose Yocto over buildroot for your official integration. I figured buildroot would be better, because Yocto's opkg system is superfluous on a device with full image updates.

eystein 10y ago
Yocto has quite large community and is growing fast. That said, think of Yocto as the first integration not the only - buildroot is surely interesting too but we had to start somewhere. :)

veli_joza 10y ago

Can someone summarize the difference between Mender and OSTree? I see that QtOTA chose OSTree as their underlying mechanism, which is significant in embedded automotive industry.

mwcampbell 10y ago
From my perspective as a curious observer of both projects, OSTree certainly looks attractive, because it doesn't waste space on two rootfs partitions which have to be oversized to accommodate future growth of the image. I initially thought OSTree required btrfs, because Project Atomic used btrfs the last time I looked at it. But according to the docs, while OSTree will take advantage of btrfs features if btrfs is being used, OSTree itself will work with a variety of filesystems including ext4. Edit: An upside of the alternating rootfs partition approach is that the rootfs can be cryptographically verified at the block level. Chromium OS implemented this, and CoreOS also uses that implementation. This is probably outside the scope of Mender itself, but the updating approach used by Mender enables it.
- eystein 10y ago
  Cryptographic signing and verification is in scope for Mender [0], and frankly it should be in scope for all updaters -- too many hacks have happened due to lack of codesigning. [0] https://tracker.mender.io/projects/MEN/issues/MEN-1020 https://tracker.mender.io/projects/MEN/issues/MEN-1020
oytis 10y ago
If you are interested in OSTree, there is an open-source solution for OSTree updates on embedded devices. Yocto layer: https://github.com/advancedtelematic/meta-updater https://github.com/advancedtelematic/meta-updater Quickstart project with a nice tutorial: https://github.com/advancedtelematic/garage-quickstart-rpi https://github.com/advancedtelematic/garage-quickstart-rpi And yes, we have chosen it because you don't have to waste twice as much disk space, and, more importantly (for wireless networks at least), you don't have to download the whole image.
theamk 10y ago