4 ms·

Hi! I really appreciate you taking the time to reply. I have since gotten Invoke to run and was already able to get some results I'm really quite happy with, s

Hi! I really appreciate you taking the time to reply. I have since gotten Invoke to run and was already able to get some results I'm really quite happy with, so thank you for your time and commitment working on Invoke! I understand that ROCm is still challenging, but it seems my problems were less related to ROCm or Invoke itself and more to Python dependency management. It really boiled down to getting the correct (ROCm) versions of packages installed. Installing Invoke from PyPi always removed my Torch and installed CUDA-enabled Torch (as well as cuBLAS, cuDNN, ...). Once I had the correct versions of packages, everything just worked. To me, your pyproject.toml looks perfectly sane, so I wasn't sure how to go about fixing the problem. What ended up working for me was to use one of AMD's ROCm OCI base images, manually installing all dependencies, foregoing a virtual environment, cloning your repo (, building the frontend), and then installing from there. The majority of my struggle would have been solved by a recent working Docker image containing a working setup. (The one on Docker Hub is 9 months old.) Trying to build the Dockerfile from your repo, I also ended up with a CUDA-enabled Torch. It did install the correct one first, but in a later step removed the ROCm-enabled Torch to switch it for the CUDA-enabled one. I hope you'll consider investing some resources into publishing newer, working builds of your Docker image.

sophrocyne 3y ago
You bet - Thanks for the feedback. Glad you're enjoying Invoke! We do have Docker packages hosted on GH, but I'll be the first to admit that we haven't prioritized ROCm. Contributors who have AMDs are a scant few, but maybe we'll find some help in wrangling that problem now that we know there's an avenue to do so.
- Cu3PO42 3y ago
  I hate maintaining my own build instructions as much as the next guy, so I'll try to get your Dockerfile working for me and then send a PR.
- Cu3PO42 3y ago
  As promised in my other comment, I did send a PR! https://github.com/invoke-ai/InvokeAI/pull/5714 https://github.com/invoke-ai/InvokeAI/pull/5714
doctorpangloss 3y ago
> Installing Invoke from PyPi... To me, your pyproject.toml looks perfectly sane, so I wasn't sure how to go about fixing the problem. You can't install the PyTorch that's best for the currently running platform using a pyproject.toml with a setuptools backend, for starters. Invoke would have to author a setup.py that deals with all the issues, in a way that is compatible with build isolation. > The majority of my struggle would have been solved by a recent working Docker image containing a working setup. (The one on Docker Hub is 9 months old.) Why? Given the state of the ecosystem, what guarantee is there really that the documentation for Docker Desktop with AMD ROCm device binding is going to actually work for your device? (https://rocm.docs.amd.com/projects/MIVisionX/en/latest/docker/README.html https://rocm.docs.amd.com/projects/MIVisionX/en/latest/docke...) There is a lot of ad-hoc reinvention of tooling in this space.
- Cu3PO42 3y ago
  > You can't install the PyTorch that's best for the currently running platform using a pyproject.toml with a setuptools backend, for starters. I see. I do know Python, but my knowledge of setuptools, pip, poetry and whatever else have you. To get my working setup, I specified an --index-url for my Torch installation. Does that not work while using their current setup? > Why? Given the state of the ecosystem, what guarantee is there really that the documentation for Docker Desktop with AMD ROCm device binding is going to actually work for your device? Well, they did work for me. Though I think only passing /dev/{dri,kfd} and setting seccomp=unconfined was sufficient. So for my particular case, getting a working image was the only missing step. From a more general POV: it might not make sense to invest in a ROCm OCI image from a short-term business perspective, but in the long term and based purely on principal, I do think the ecosystem should strive to be less reliant on CUDA and only CUDA.
westurner 3y ago
> AMD's ROCm OCI base images, ROCm docs > "Install ROCm Docker containers" > Base Image: https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/docker.html#base-images https://rocm.docs.amd.com/projects/install-on-linux/en/lates... links to ROCm/ROCm-docker: https://github.com/ROCm/ROCm-docker https://github.com/ROCm/ROCm-docker which is the source of docker.io/rocm/rocm-terminal: https://hub.docker.com/r/rocm/rocm-terminal https://hub.docker.com/r/rocm/rocm-terminal : docker run -it --device=/dev/kfd --device=/dev/dri --group-add video rocm/rocm-terminal ROCm docs > "Docker image support matrix": https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/docker-image-support-matrix.html https://rocm.docs.amd.com/projects/install-on-linux/en/lates... ROCm/ROCm-docker//dev/Dockerfile-centos-7-complete: https://github.com/ROCm/ROCm-docker/blob/master/dev/Dockerfile-centos-7-complete https://github.com/ROCm/ROCm-docker/blob/master/dev/Dockerfi... Bazzite is a ublue (Universal Blue) fork of the Fedora Kinoite (KDE) or Fedora Silverblue (Gnome) rpm-ostree Linux distributions; ublue-os/bazzite//Containerfile : https://github.com/ublue-os/bazzite/blob/main/Containerfile#L465 https://github.com/ublue-os/bazzite/blob/main/Containerfile#... has, in addition to fan and power controls, automatic updates on desktop, supergfxctl, system76-scheduler, and an fsync kernel: rpm-ostree install rocm-hip \ rocm-opencl \ rocm-clinfo But it's not `rpm-ostree install --apply-live` because its a Containerfile. To install a ublue-os distro, you install any of the Fedora ostree distros: {Silverblue, Kinoite, Sway Atomic, or Budgie Atomic} from e.g. a USB stick and then `rpm-ostree rebase <OCI_host_image_url>`: rpm-ostree rebase ostree-unverified-registry:ghcr.io/ublue-os/bazzite:stable rpm-ostree rebase ostree-unverified-registry:ghcr.io/ublue-os/bazzite-nvidia:stable rpm-ostree rebase ostree-image-signed: ublue-os/config//build/ublue-os-just/40-nvidia.just defines the `ujust configure-nvidia` and `ujust toggle-nvk` commands: https://github.com/ublue-os/config/blob/main/build/ublue-os-just/40-nvidia.just https://github.com/ublue-os/config/blob/main/build/ublue-os-... There's a default `distrobox` with pytorch in ublue-os/config//build/ublue-os-just/etc-distrobox/apps.ini: https://github.com/ublue-os/config/blob/main/build/ublue-os-just/etc-distrobox/apps.ini https://github.com/ublue-os/config/blob/main/build/ublue-os-... [mlbox] image=nvcr.io/nvidia/pytorch:23.08-py3 additional_packages="nano git htop" init_hooks="pip3 install huggingface_hub tokenizers transformers accelerate datasets wandb peft bitsandbytes fastcore fastprogress watermark torchmetrics deepspeed" pre-init-hooks="/init_script.sh" nvidia=true pull=true root=false replace=false docker.io/rocm/pytorch: https://hub.docker.com/r/rocm/pytorch https://hub.docker.com/r/rocm/pytorch pytorch/builder//manywheel/Dockerfile: https://github.com/pytorch/builder/blob/main/manywheel/Dockerfile#L157 https://github.com/pytorch/builder/blob/main/manywheel/Docke... ROCm/pytorch//Dockerfile: https://github.com/ROCm/pytorch/blob/main/Dockerfile https://github.com/ROCm/pytorch/blob/main/Dockerfile The ublue-os (and so also bazzite) OCI host image Containerfile has Sunshine installed; which is a 4k HDR 120fps remote desktop solution for gaming. There's a `ujust remove-sunshine` command in system_files/desktop/shared/usr/share/ublue-os/just/80-bazzite.just : https://github.com/ublue-os/bazzite/blob/main/system_files/desktop/shared/usr/share/ublue-os/just/80-bazzite.just https://github.com/ublue-os/bazzite/blob/main/system_files/d... and also kernel args for AMD: pstate-force-enable: rpm-ostree kargs --append-if-missing=amd_pstate=active ublue-os/config//Containerfile: https://github.com/ublue-os/config/blob/main/Containerfile https://github.com/ublue-os/config/blob/main/Containerfile LizardByte/Sunshine: https://github.com/LizardByte/Sunshine https://github.com/LizardByte/Sunshine moonlight-stream https://github.com/moonlight-stream https://github.com/moonlight-stream Anyways, hopefully this PR fixes the immediate issue: https://github.com/invoke-ai/InvokeAI/pull/5714/files https://github.com/invoke-ai/InvokeAI/pull/5714/files conda-forge/pytorch-cpu-feedstock > "Add ROCm variant?": https://github.com/conda-forge/pytorch-cpu-feedstock/issues/198 https://github.com/conda-forge/pytorch-cpu-feedstock/issues/... And Fedora supports OCI containers as host images and also podman container images with just systemd to respawn one or a pod of containers.

sophrocyne 3y ago

You bet - Thanks for the feedback. Glad you're enjoying Invoke! We do have Docker packages hosted on GH, but I'll be the first to admit that we haven't prioritized ROCm. Contributors who have AMDs are a scant few, but maybe we'll find some help in wrangling that problem now that we know there's an avenue to do so.

Cu3PO42 3y ago
I hate maintaining my own build instructions as much as the next guy, so I'll try to get your Dockerfile working for me and then send a PR.
Cu3PO42 3y ago
As promised in my other comment, I did send a PR! https://github.com/invoke-ai/InvokeAI/pull/5714 https://github.com/invoke-ai/InvokeAI/pull/5714

doctorpangloss 3y ago

> Installing Invoke from PyPi... To me, your pyproject.toml looks perfectly sane, so I wasn't sure how to go about fixing the problem. You can't install the PyTorch that's best for the currently running platform using a pyproject.toml with a setuptools backend, for starters. Invoke would have to author a setup.py that deals with all the issues, in a way that is compatible with build isolation. > The majority of my struggle would have been solved by a recent working Docker image containing a working setup. (The one on Docker Hub is 9 months old.) Why? Given the state of the ecosystem, what guarantee is there really that the documentation for Docker Desktop with AMD ROCm device binding is going to actually work for your device? (https://rocm.docs.amd.com/projects/MIVisionX/en/latest/docker/README.html https://rocm.docs.amd.com/projects/MIVisionX/en/latest/docke...) There is a lot of ad-hoc reinvention of tooling in this space.

Cu3PO42 3y ago
> You can't install the PyTorch that's best for the currently running platform using a pyproject.toml with a setuptools backend, for starters. I see. I do know Python, but my knowledge of setuptools, pip, poetry and whatever else have you. To get my working setup, I specified an --index-url for my Torch installation. Does that not work while using their current setup? > Why? Given the state of the ecosystem, what guarantee is there really that the documentation for Docker Desktop with AMD ROCm device binding is going to actually work for your device? Well, they did work for me. Though I think only passing /dev/{dri,kfd} and setting seccomp=unconfined was sufficient. So for my particular case, getting a working image was the only missing step. From a more general POV: it might not make sense to invest in a ROCm OCI image from a short-term business perspective, but in the long term and based purely on principal, I do think the ecosystem should strive to be less reliant on CUDA and only CUDA.

westurner 3y ago

> AMD's ROCm OCI base images, ROCm docs > "Install ROCm Docker containers" > Base Image: https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/docker.html#base-images https://rocm.docs.amd.com/projects/install-on-linux/en/lates... links to ROCm/ROCm-docker: https://github.com/ROCm/ROCm-docker https://github.com/ROCm/ROCm-docker which is the source of docker.io/rocm/rocm-terminal: https://hub.docker.com/r/rocm/rocm-terminal https://hub.docker.com/r/rocm/rocm-terminal : docker run -it --device=/dev/kfd --device=/dev/dri --group-add video rocm/rocm-terminal ROCm docs > "Docker image support matrix": https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/docker-image-support-matrix.html https://rocm.docs.amd.com/projects/install-on-linux/en/lates... ROCm/ROCm-docker//dev/Dockerfile-centos-7-complete: https://github.com/ROCm/ROCm-docker/blob/master/dev/Dockerfile-centos-7-complete https://github.com/ROCm/ROCm-docker/blob/master/dev/Dockerfi... Bazzite is a ublue (Universal Blue) fork of the Fedora Kinoite (KDE) or Fedora Silverblue (Gnome) rpm-ostree Linux distributions; ublue-os/bazzite//Containerfile : https://github.com/ublue-os/bazzite/blob/main/Containerfile#L465 https://github.com/ublue-os/bazzite/blob/main/Containerfile#... has, in addition to fan and power controls, automatic updates on desktop, supergfxctl, system76-scheduler, and an fsync kernel: rpm-ostree install rocm-hip \ rocm-opencl \ rocm-clinfo But it's not `rpm-ostree install --apply-live` because its a Containerfile. To install a ublue-os distro, you install any of the Fedora ostree distros: {Silverblue, Kinoite, Sway Atomic, or Budgie Atomic} from e.g. a USB stick and then `rpm-ostree rebase <OCI_host_image_url>`: rpm-ostree rebase ostree-unverified-registry:ghcr.io/ublue-os/bazzite:stable rpm-ostree rebase ostree-unverified-registry:ghcr.io/ublue-os/bazzite-nvidia:stable rpm-ostree rebase ostree-image-signed: ublue-os/config//build/ublue-os-just/40-nvidia.just defines the `ujust configure-nvidia` and `ujust toggle-nvk` commands: https://github.com/ublue-os/config/blob/main/build/ublue-os-just/40-nvidia.just https://github.com/ublue-os/config/blob/main/build/ublue-os-... There's a default `distrobox` with pytorch in ublue-os/config//build/ublue-os-just/etc-distrobox/apps.ini: https://github.com/ublue-os/config/blob/main/build/ublue-os-just/etc-distrobox/apps.ini https://github.com/ublue-os/config/blob/main/build/ublue-os-... [mlbox] image=nvcr.io/nvidia/pytorch:23.08-py3 additional_packages="nano git htop" init_hooks="pip3 install huggingface_hub tokenizers transformers accelerate datasets wandb peft bitsandbytes fastcore fastprogress watermark torchmetrics deepspeed" pre-init-hooks="/init_script.sh" nvidia=true pull=true root=false replace=false docker.io/rocm/pytorch: https://hub.docker.com/r/rocm/pytorch https://hub.docker.com/r/rocm/pytorch pytorch/builder//manywheel/Dockerfile: https://github.com/pytorch/builder/blob/main/manywheel/Dockerfile#L157 https://github.com/pytorch/builder/blob/main/manywheel/Docke... ROCm/pytorch//Dockerfile: https://github.com/ROCm/pytorch/blob/main/Dockerfile https://github.com/ROCm/pytorch/blob/main/Dockerfile The ublue-os (and so also bazzite) OCI host image Containerfile has Sunshine installed; which is a 4k HDR 120fps remote desktop solution for gaming. There's a `ujust remove-sunshine` command in system_files/desktop/shared/usr/share/ublue-os/just/80-bazzite.just : https://github.com/ublue-os/bazzite/blob/main/system_files/desktop/shared/usr/share/ublue-os/just/80-bazzite.just https://github.com/ublue-os/bazzite/blob/main/system_files/d... and also kernel args for AMD: pstate-force-enable: rpm-ostree kargs --append-if-missing=amd_pstate=active ublue-os/config//Containerfile: https://github.com/ublue-os/config/blob/main/Containerfile https://github.com/ublue-os/config/blob/main/Containerfile LizardByte/Sunshine: https://github.com/LizardByte/Sunshine https://github.com/LizardByte/Sunshine moonlight-stream https://github.com/moonlight-stream https://github.com/moonlight-stream Anyways, hopefully this PR fixes the immediate issue: https://github.com/invoke-ai/InvokeAI/pull/5714/files https://github.com/invoke-ai/InvokeAI/pull/5714/files conda-forge/pytorch-cpu-feedstock > "Add ROCm variant?": https://github.com/conda-forge/pytorch-cpu-feedstock/issues/198 https://github.com/conda-forge/pytorch-cpu-feedstock/issues/... And Fedora supports OCI containers as host images and also podman container images with just systemd to respawn one or a pod of containers.