5 ms·

I know it's still different than what you're looking for, so you probably already know this, but many projects like this have the Dockerfile on github which sho

I know it's still different than what you're looking for, so you probably already know this, but many projects like this have the Dockerfile on github which shows exactly how they set up the image. For example: https://github.com/RadeonOpenCompute/ROCm-docker/blob/master/dev/Dockerfile-ubuntu-22.04 https://github.com/RadeonOpenCompute/ROCm-docker/blob/master... They also have some for Fedora. Looks like for this you need to install their repo: curl -sL https://repo.radeon.com/rocm/rocm.gpg.key | apt-key add - \ && printf "deb [arch=amd64] https://repo.radeon.com/rocm/apt/$ROCM_VERSION/ jammy main" | tee /etc/apt/sources.list.d/rocm.list \ && printf "deb [arch=amd64] https://repo.radeon.com/amdgpu/$AMDGPU_VERSION/ubuntu jammy main" | tee /etc/apt/sources.list.d/amdgpu.list \ then install Python, a couple other dependencies (build-essential, etc) and then the package in question: rocm-dev So they are doing the packaging. There might even be documentation elsewhere for that type of setup.

mikepurvis 3y ago
Oh yeah, I mean... having the source for the container build is kind of table stakes at this point. No one would accept a 10gb mystery meat blob as the basis of their production system. It's bad enough that we still accept binary-only drivers and proprietary libraries like TensorRT. I think my issue is more just with the mindset that it's okay to have one narrow slice of supported versions of everything that are "known to work together" and those are what's in the container and anything outside of those and you're immediately pooched. This is not hypothetical btw, I've run into real problems around it with libraries like gproto, where tensorflow's bazel build pulls in an exact version that's different from the default one in nixpkgs, and now you get symbol conflicts when something tries to link to the tensorflow c++ API while linking to another component already using the default gproto. I know these problems are solveable with symbol visibility control and whatever, but that stuff is far from universal and hard to get right, especially if the person setting up the build rules for the library doesn't themselves use it in that type of heterogeneous environment (like, everyone at Google just links the same global proto version from the monorepo so it doesn't matter).
- hotstickyballs 3y ago
  If anything, the situation with tensor rt shows that companies are absolutely willing to accept a multi gig meat blob
- mgaunard 3y ago
  I don't know what world you live in, but this is a problem for any software development. You need to ensure that there is only one version of any library used globally throughout the code and that the set of versions is compatible with each other, and preferably you also want everything to be built against the same toolchain with the same flags. That usually means onboarding third-party libraries into your own build system.
  - josephg 3y ago
    It’s not a universal problem. A lot of modern languages allow multiple versions of a library to be pulled in to the same code base, through different dependency paths. (Eg nodejs, rust). It’s not a perfect answer by any means, but it’s nice not needing to worry about some package pulling in an inconvenient version of one of its dependencies. Also, just to name it, it’s ridiculous that a specific graphics card manages to restrict the version of gproto that you’re using. You don’t have this problem with nvidia drivers, since cuda stuff is much less fiddly. AMD needs to pull a finger out and fix the bugs in their stack that make it so fragile like this.
    - deleted 3y ago
      [deleted]
  - anuraaga 3y ago
    I'd say with semver becoming far better known, this is not a problem for "any" software development. The developer gets the choice to pick libraries that are stable, often also influencing language choice. Mistakes happen, Guava broke the Java ecosystem for about two years, but it's never something that is accepted as just a fact of software development, it is a mistake. Wanting to hold Python+C ecosystem more accountable is fair I think, at least from my own experience around half a year ago, Anaconda doesn't work and you need a Dockerfile for any sort of reproducibility, which can have issues since GPU with docker isn't that easy. And this means developers from the vendors working with Anaconda, for example, on solving the issue rather than just hoping for contributors to do it. If AMD were to make easy, reproducible builds without root or VM a reality, that would be reason enough to try their hardware. If not, hopefully Nvidia does and then there really would be no way across the moat for me at least.

mikepurvis 3y ago

Oh yeah, I mean... having the source for the container build is kind of table stakes at this point. No one would accept a 10gb mystery meat blob as the basis of their production system. It's bad enough that we still accept binary-only drivers and proprietary libraries like TensorRT. I think my issue is more just with the mindset that it's okay to have one narrow slice of supported versions of everything that are "known to work together" and those are what's in the container and anything outside of those and you're immediately pooched. This is not hypothetical btw, I've run into real problems around it with libraries like gproto, where tensorflow's bazel build pulls in an exact version that's different from the default one in nixpkgs, and now you get symbol conflicts when something tries to link to the tensorflow c++ API while linking to another component already using the default gproto. I know these problems are solveable with symbol visibility control and whatever, but that stuff is far from universal and hard to get right, especially if the person setting up the build rules for the library doesn't themselves use it in that type of heterogeneous environment (like, everyone at Google just links the same global proto version from the monorepo so it doesn't matter).

hotstickyballs 3y ago
If anything, the situation with tensor rt shows that companies are absolutely willing to accept a multi gig meat blob
mgaunard 3y ago
I don't know what world you live in, but this is a problem for any software development. You need to ensure that there is only one version of any library used globally throughout the code and that the set of versions is compatible with each other, and preferably you also want everything to be built against the same toolchain with the same flags. That usually means onboarding third-party libraries into your own build system.
- josephg 3y ago
  It’s not a universal problem. A lot of modern languages allow multiple versions of a library to be pulled in to the same code base, through different dependency paths. (Eg nodejs, rust). It’s not a perfect answer by any means, but it’s nice not needing to worry about some package pulling in an inconvenient version of one of its dependencies. Also, just to name it, it’s ridiculous that a specific graphics card manages to restrict the version of gproto that you’re using. You don’t have this problem with nvidia drivers, since cuda stuff is much less fiddly. AMD needs to pull a finger out and fix the bugs in their stack that make it so fragile like this.
  - deleted 3y ago
    [deleted]
- anuraaga 3y ago
  I'd say with semver becoming far better known, this is not a problem for "any" software development. The developer gets the choice to pick libraries that are stable, often also influencing language choice. Mistakes happen, Guava broke the Java ecosystem for about two years, but it's never something that is accepted as just a fact of software development, it is a mistake. Wanting to hold Python+C ecosystem more accountable is fair I think, at least from my own experience around half a year ago, Anaconda doesn't work and you need a Dockerfile for any sort of reproducibility, which can have issues since GPU with docker isn't that easy. And this means developers from the vendors working with Anaconda, for example, on solving the issue rather than just hoping for contributors to do it. If AMD were to make easy, reproducible builds without root or VM a reality, that would be reason enough to try their hardware. If not, hopefully Nvidia does and then there really would be no way across the moat for me at least.