You can do inference from a Docker container, just as you'd do it with NVidia. OpenAI runs a K8s cluster doing this. I have personally only worked with NVidia,

You can do inference from a Docker container, just as you'd do it with NVidia. OpenAI runs a K8s cluster doing this. I have personally only worked with NVidia, but the docs are present for AMD too. Like anything AI and AMD, you need the right card(s) and rocm version along with sheer dumb luck to get it working. AMD has Docker images with rocm support, so you could merge your app in with that as the base layer. Just pass through the GPU to the container and you should get it working. It might just be the software in a Docker image, but it removes a variable I would otherwise have to worry about during deployment. It literally is inference on AMD with Docker, if that's what you meant.