5 ms·

I'll piggyback on your comment for a question: I only very recently started getting into the basic of GPGPU coding. I hate vendor lock in, so I started out with

by gspr 5y ago

I'll piggyback on your comment for a question: I only very recently started getting into the basic of GPGPU coding. I hate vendor lock in, so I started out with OpenCL. I had a good time, but I do understand that more and more of non-CUDA world (however small that may be) is moving away from OpenCL. What are they moving to? Where should one direct one's "compute learning energy" (if one already excludes CUDA or other proprietary solutions)?

fsloth 5y ago
Not OP, but: Unfortunately you need to pick a platform and target that. Which API you use depends on the platform. DirectX, OpenGL, Vulkan and Metal all have compute capability. So, I think you need to abstract your code at the services level, and provide an implementations for each platform you need to support (the number may be 1 or more than 1). I think you need to direct your "compute learning" energy at the algorithm level. Numerics and all that. The GPU API:s are just the obtuse implementation detail, but not the meat of the thing.
- raphlinus 5y ago
  In addition to picking a platform, you also essentially need to pick a generation. Vulkan 1.0 for example is missing subgroups and has lots of other limitations. Vulkan 1.2 on a modern GPU with an updated driver is pretty darned powerful, with an explicit memory model, control over subgroup size, a form of pointers, and a range of scalar datatype sizes (f16 is especially useful for machine learning, commonly having 2x the throughput as f32). It is possible to have multiple versions and select at runtime, but it's an additional complexity that needs to be considered, and can constrain your architecture to accommodate the compatibility modes.
- gspr 5y ago
  Thanks! That's a bit of a sad state of things :-(
- zozbot234 5y ago
  > So, I think you need to abstract your code at the services level, and provide an implementations for each platform you need to support (the number may be 1 or more than 1). There are some cross-platform capabilities via SPIRV-Cross. The LLVM folks are working on MLIR that might provide even better features wrt. a common abstraction layer going forward.

by gspr 5y ago

fsloth 5y ago

Not OP, but: Unfortunately you need to pick a platform and target that. Which API you use depends on the platform. DirectX, OpenGL, Vulkan and Metal all have compute capability. So, I think you need to abstract your code at the services level, and provide an implementations for each platform you need to support (the number may be 1 or more than 1). I think you need to direct your "compute learning" energy at the algorithm level. Numerics and all that. The GPU API:s are just the obtuse implementation detail, but not the meat of the thing.

raphlinus 5y ago
In addition to picking a platform, you also essentially need to pick a generation. Vulkan 1.0 for example is missing subgroups and has lots of other limitations. Vulkan 1.2 on a modern GPU with an updated driver is pretty darned powerful, with an explicit memory model, control over subgroup size, a form of pointers, and a range of scalar datatype sizes (f16 is especially useful for machine learning, commonly having 2x the throughput as f32). It is possible to have multiple versions and select at runtime, but it's an additional complexity that needs to be considered, and can constrain your architecture to accommodate the compatibility modes.
gspr 5y ago
Thanks! That's a bit of a sad state of things :-(
zozbot234 5y ago
> So, I think you need to abstract your code at the services level, and provide an implementations for each platform you need to support (the number may be 1 or more than 1). There are some cross-platform capabilities via SPIRV-Cross. The LLVM folks are working on MLIR that might provide even better features wrt. a common abstraction layer going forward.