4 ms·

How does hardware acceleration work with containers?

23 points by leonheld 2y ago

throwaway42668 2y ago
> "At Toradex we build Torizon OS, which is a container-based embedded system." That sounded interesting. Then I read the article only to find a very, very loose definition of "embedded".
- leonheld 2y ago
  Yea, for sure, it might sound very loose compared to something like a microcontroller. But embedded is very broad. Some of our devices run Chromium, for example :-)
  - elcritch 2y ago
    Doing similar with some of my devices. Out of curiosity, how are you building Chromium for embedded?
    - leonheld 2y ago
      Hi! We either use Yocto recipes for 'regular' embedded (meta-imx and meta-browser are good starting points) but we also apply nxp-specific patches on top of Chromium from Debian and use the usual standard tooling like `sbuild` to run it inside the containers (renting out AWS arm64 instances in this case, because building Chromium requires a metric ton of cores and memory. Cross-building is quite challenging, so I wouldn't recommend that).
- deleted 2y ago
  [deleted]
pragma_x 2y ago
This article sent me down a rabbit-hole about device cgroup rules, since the concept was foreign to me. Bottom line here is that these rules look cryptic because they map to device _numbers and types_ as enumerated by the kernel. Some like `/dev/sda` translating to `b 8:0` are probably going to be consistent between different Linuxes. The rest? I have no idea. The rule syntax is what the kernel itself uses, so that isn't Docker's fault. Still, I would love to be able to say `/dev/sda rwm` instead of `b 8:0 rwm`, by letting the Docker daemon automate the path-to-cgroup-rule translation. It would be far more docker-compose friendly. Even better, like for the sake of this article, would be to have known semantic names like 'gpu', '3daccelerator', and 'primarystorage'. Anything to make docker-compose files more useful to other people. https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v1/devices.html https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v1...
- leonheld 2y ago
  > Some like `/dev/sda` translating to `b 8:0` are probably going to be consistent between different Linuxes. This is well documented for Linux https://www.kernel.org/doc/Documentation/admin-guide/devices.txt https://www.kernel.org/doc/Documentation/admin-guide/devices.... The real issue happes when vendors get a bit too creative and don't plan on upstreaming their drivers.
rcarmo 2y ago
I do this all the time with LXC. Makes for a nice way to share GPU resources among several “machines” without devoting a PCI slot to a VM.

23 points by leonheld 2y ago

> "At Toradex we build Torizon OS, which is a container-based embedded system." That sounded interesting. Then I read the article only to find a very, very loose definition of "embedded".

leonheld 2y ago
Yea, for sure, it might sound very loose compared to something like a microcontroller. But embedded is very broad. Some of our devices run Chromium, for example :-)
- elcritch 2y ago
  Doing similar with some of my devices. Out of curiosity, how are you building Chromium for embedded?
  - leonheld 2y ago
    Hi! We either use Yocto recipes for 'regular' embedded (meta-imx and meta-browser are good starting points) but we also apply nxp-specific patches on top of Chromium from Debian and use the usual standard tooling like `sbuild` to run it inside the containers (renting out AWS arm64 instances in this case, because building Chromium requires a metric ton of cores and memory. Cross-building is quite challenging, so I wouldn't recommend that).
deleted 2y ago
[deleted]

pragma_x 2y ago

This article sent me down a rabbit-hole about device cgroup rules, since the concept was foreign to me. Bottom line here is that these rules look cryptic because they map to device _numbers and types_ as enumerated by the kernel. Some like `/dev/sda` translating to `b 8:0` are probably going to be consistent between different Linuxes. The rest? I have no idea. The rule syntax is what the kernel itself uses, so that isn't Docker's fault. Still, I would love to be able to say `/dev/sda rwm` instead of `b 8:0 rwm`, by letting the Docker daemon automate the path-to-cgroup-rule translation. It would be far more docker-compose friendly. Even better, like for the sake of this article, would be to have known semantic names like 'gpu', '3daccelerator', and 'primarystorage'. Anything to make docker-compose files more useful to other people. https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v1/devices.html https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v1...

leonheld 2y ago
> Some like `/dev/sda` translating to `b 8:0` are probably going to be consistent between different Linuxes. This is well documented for Linux https://www.kernel.org/doc/Documentation/admin-guide/devices.txt https://www.kernel.org/doc/Documentation/admin-guide/devices.... The real issue happes when vendors get a bit too creative and don't plan on upstreaming their drivers.

rcarmo 2y ago

I do this all the time with LXC. Makes for a nice way to share GPU resources among several “machines” without devoting a PCI slot to a VM.