I'm not familiar with multi-GPU setup in general. GPU programming in Julia has the advantage that naive operation doesn't even need to be GPU-aware (for writers

I'm not familiar with multi-GPU setup in general. GPU programming in Julia has the advantage that naive operation doesn't even need to be GPU-aware (for writers), since GPU arrays (of any vendor backend) conforms the AbstractArray interface. If you're advanced library writer, you can leverage: https://juliagpu.github.io/KernelAbstractions.jl/stable/#Writing-your-first-kernel-1 https://juliagpu.github.io/KernelAbstractions.jl/stable/#Wri... which allows you to write kernel, in Julia, that compiles efficiently with rest of native Julia code, and that works cross-vendor! Back to multi GPU, it seems there's: https://clima.github.io/OceananigansDocumentation/stable/appendix/benchmarks/#Multi-GPU-Shallow-Water-Model https://clima.github.io/OceananigansDocumentation/stable/app... which is MPI based?