4 ms·

i’m always confused wrt how shader languages that target a bunch of other framework specific shader languages manage to map all the available features correctly

i’m always confused wrt how shader languages that target a bunch of other framework specific shader languages manage to map all the available features correctly? like tensor core support, non-uniform thread groups, different data type support, a bunch of simd group functions, stuff like that… i couldn’t find info wrt this in rust gpu, so i’m assuming it just tries to target the narrowest available feature set, that’s compatible across all shading languages?

LegNeato 2y ago
The goal is to handle things similar to how rust on the CPU does (shared traits with different implementations, intrinsics, platform-specific modules, user replaceable traits like alloc or panic handler and "asm" as a last ditch case). We already have a lot (we have many intrinsics and an `arch` module like `std::arch`, asm! to include raw spirv, etc). For example, here are the intrinsics: https://rust-gpu.github.io/rust-gpu/api/spirv_std/arch/index.html https://rust-gpu.github.io/rust-gpu/api/spirv_std/arch/index... and here is support for ray tracing (which obviously is not on every card: https://rust-gpu.github.io/rust-gpu/api/spirv_std/ray_tracing/index.html https://rust-gpu.github.io/rust-gpu/api/spirv_std/ray_tracin...). Vulkan has a way to query and specify different GPU capabilities and Rust-GPU uses that. Rust and Vulkan have many of the tools we need for progressive enhancement, we are not focused on lowest common denominator.
cosmic_quanta 2y ago
I'm not familiar with GPUs specifically, but I have seen this for ORMs that support multiple SQL dialects (e.g. [0]). A great technique is called 'tagless final encoding' [1]. Using this technique, you can specify capabilities of an embedded domain-specific language (eDSL) such that you can have a shared (but narrow) common set of features, while allowing specializations of this eDSL to support extra features. [0]: https://github.com/haskell-beam/beam https://github.com/haskell-beam/beam [1]: https://nrinaudo.github.io/articles/tagless_final.html https://nrinaudo.github.io/articles/tagless_final.html
messe 2y ago
It's mentioned near the beginning of the linked article: > These Rust GPU programs are then compiled into SPIR-V, a low-level format that most GPUs understand
- taminka 2y ago
  so it’s just a matter of rust gpu not yet supporting these features?
  - deleted 2y ago
    [deleted]
fulafel 2y ago
The GPU prorgramming language tech landscape is generally pretty low-tech on the compilers side. Tensor cores are not supported. (Culturally of course the big one is the fragmentation and the proprietary nature of everything which is the reason so little gets done on GPUs and the horror of attempting multiplatform software there)
- creata 2y ago
  > Tensor cores are not supported. Correct me if I'm wrong or misunderstanding you, but doesn't SPIR-V support tensor cores via SPV_KHR_cooperative_matrix?
  - fulafel 2y ago
    I don't know, do share if there's something you can link. At first blush it just sounds like something to allow multiple shader compute elements to work more efficiently together ("cooperate") on a single bigger matrix computation.
    - creata 2y ago
      That is what it is afaict, but NVIDIA says this in their 2019 post "Machine Learning Acceleration in Vulkan with Cooperative Matrices"[0]: > Additionally, if the GPU includes dedicated hardware for high-speed matrix operations, such as the Tensor Cores on Turing GPUs, then the Cooperative Matrix extension can tap into the power of this acceleration with no application changes. The benchmark graph doesn't look too great, though - around half the "theoretical peak tensor core performance". [0]: https://developer.nvidia.com/blog/machine-learning-acceleration-vulkan-cooperative-matrices/ https://developer.nvidia.com/blog/machine-learning-accelerat...

LegNeato 2y ago

The goal is to handle things similar to how rust on the CPU does (shared traits with different implementations, intrinsics, platform-specific modules, user replaceable traits like alloc or panic handler and "asm" as a last ditch case). We already have a lot (we have many intrinsics and an `arch` module like `std::arch`, asm! to include raw spirv, etc). For example, here are the intrinsics: https://rust-gpu.github.io/rust-gpu/api/spirv_std/arch/index.html https://rust-gpu.github.io/rust-gpu/api/spirv_std/arch/index... and here is support for ray tracing (which obviously is not on every card: https://rust-gpu.github.io/rust-gpu/api/spirv_std/ray_tracing/index.html https://rust-gpu.github.io/rust-gpu/api/spirv_std/ray_tracin...). Vulkan has a way to query and specify different GPU capabilities and Rust-GPU uses that. Rust and Vulkan have many of the tools we need for progressive enhancement, we are not focused on lowest common denominator.

cosmic_quanta 2y ago

I'm not familiar with GPUs specifically, but I have seen this for ORMs that support multiple SQL dialects (e.g. [0]). A great technique is called 'tagless final encoding' [1]. Using this technique, you can specify capabilities of an embedded domain-specific language (eDSL) such that you can have a shared (but narrow) common set of features, while allowing specializations of this eDSL to support extra features. [0]: https://github.com/haskell-beam/beam https://github.com/haskell-beam/beam [1]: https://nrinaudo.github.io/articles/tagless_final.html https://nrinaudo.github.io/articles/tagless_final.html

messe 2y ago

It's mentioned near the beginning of the linked article: > These Rust GPU programs are then compiled into SPIR-V, a low-level format that most GPUs understand

taminka 2y ago
so it’s just a matter of rust gpu not yet supporting these features?
- deleted 2y ago
  [deleted]

fulafel 2y ago

The GPU prorgramming language tech landscape is generally pretty low-tech on the compilers side. Tensor cores are not supported. (Culturally of course the big one is the fragmentation and the proprietary nature of everything which is the reason so little gets done on GPUs and the horror of attempting multiplatform software there)

creata 2y ago
> Tensor cores are not supported. Correct me if I'm wrong or misunderstanding you, but doesn't SPIR-V support tensor cores via SPV_KHR_cooperative_matrix?
- fulafel 2y ago
  I don't know, do share if there's something you can link. At first blush it just sounds like something to allow multiple shader compute elements to work more efficiently together ("cooperate") on a single bigger matrix computation.
  - creata 2y ago
    That is what it is afaict, but NVIDIA says this in their 2019 post "Machine Learning Acceleration in Vulkan with Cooperative Matrices"[0]: > Additionally, if the GPU includes dedicated hardware for high-speed matrix operations, such as the Tensor Cores on Turing GPUs, then the Cooperative Matrix extension can tap into the power of this acceleration with no application changes. The benchmark graph doesn't look too great, though - around half the "theoretical peak tensor core performance". [0]: https://developer.nvidia.com/blog/machine-learning-acceleration-vulkan-cooperative-matrices/ https://developer.nvidia.com/blog/machine-learning-accelerat...