3 ms·

> will run only on NVIDIA GPUs for the preview I wonder why Microsoft helps nVidia, instead of using their own technology? Here’s an example: https://github.c

> will run only on NVIDIA GPUs for the preview I wonder why Microsoft helps nVidia, instead of using their own technology? Here’s an example: https://github.com/Const-me/Cgml https://github.com/Const-me/Cgml

capableweb 3y ago
Companies of the size of Microsoft usually function more like a country with various companies within, rather than one company with a unified view and unified goals. So probably, the group wasn't even aware of that technology because it's far away by either professional connection, or by personal/relationship connections, or they knew about it but had another goal than "maximize use of own stuff" and made the call that the tradeoffs wasn't worth it.
- Const-me 3y ago
  I don’t work for Microsoft, and I don’t care whether people are using Microsoft’s technologies. As a consumer, I’m not happy with nVidia’s monopoly in the field. I find it disappointing that Microsoft chose to play along, instead of disrupting the landscape. I think the only reason for that monopoly is the mental inertia of everyone involved. The complexity of these AI models is contained within the data in the models (gigabytes of numbers in these tensors), the GPU-running code is rather simple, most of that code is basic BLAS stuff. Unlike traditional GPGPU applications (FEM, numerical simulations, fluid dynamics), the compute kernels used in AI are easily portable across GPU APIs. Maybe when I have some free time, I should port my library to Linux + Vulkan, just to prove the point.
lostmsu 3y ago
Have you considered ILGPU? https://ilgpu.net/ https://ilgpu.net/
- Const-me 3y ago
  No, I have not considered ILGPU, it’s the first time I see that thing. Very interesting, but I’m not sure I would have used it for that project. In my experience, using any non-native GPU API is asking for troubles. On Windows, the native ones are D3D 11 and 12, on Linux and Android it’s often Vulkan, and on MacOS it’s either Metal or that newer thing they have built specifically for AI. It seems ILGPU only has backends for CUDA, OpenCL, and CPU SIMD. I have general suspicion towards custom compilers. Writing a good compiler is hard, for GPUs even harder due to the weird execution model and insufficient documentation from GPU vendors. HLSL compiler is supported by Microsoft, and compute shaders are used by many millions of gamers every day. Similarly, CUDA is supported by nVidia, usually pretty stable, my only issue with CUDA is vendor lock-in. I have an impression people are often unhappy with the quality and hardware compatibility of less popular GPU APIs like ROCm and OpenCL. BTW, I asked a friend to test my program on their low-and laptop. The laptop has some Intel core i3 with integrated GPU. The performance wasn’t great at about 1 token/second (single-channel memory), but at least my code worked. I’m not sure it would have worked on that computer if the backend was based on OpenCL.
pjmlp 3y ago
Their own technology is DirectML, not raw DirectX 11. https://blogs.windows.com/windowsdeveloper/2023/11/15/elevating-the-developer-experience-on-windows-with-new-ai-tools-and-productivity-tools/ https://blogs.windows.com/windowsdeveloper/2023/11/15/elevat... https://blogs.windows.com/windowsdeveloper/2023/12/14/directml-accelerating-ai-on-windows-now-with-npus/ https://blogs.windows.com/windowsdeveloper/2023/12/14/direct...

capableweb 3y ago

Companies of the size of Microsoft usually function more like a country with various companies within, rather than one company with a unified view and unified goals. So probably, the group wasn't even aware of that technology because it's far away by either professional connection, or by personal/relationship connections, or they knew about it but had another goal than "maximize use of own stuff" and made the call that the tradeoffs wasn't worth it.

Const-me 3y ago
I don’t work for Microsoft, and I don’t care whether people are using Microsoft’s technologies. As a consumer, I’m not happy with nVidia’s monopoly in the field. I find it disappointing that Microsoft chose to play along, instead of disrupting the landscape. I think the only reason for that monopoly is the mental inertia of everyone involved. The complexity of these AI models is contained within the data in the models (gigabytes of numbers in these tensors), the GPU-running code is rather simple, most of that code is basic BLAS stuff. Unlike traditional GPGPU applications (FEM, numerical simulations, fluid dynamics), the compute kernels used in AI are easily portable across GPU APIs. Maybe when I have some free time, I should port my library to Linux + Vulkan, just to prove the point.

lostmsu 3y ago

Have you considered ILGPU? https://ilgpu.net/ https://ilgpu.net/

Const-me 3y ago
No, I have not considered ILGPU, it’s the first time I see that thing. Very interesting, but I’m not sure I would have used it for that project. In my experience, using any non-native GPU API is asking for troubles. On Windows, the native ones are D3D 11 and 12, on Linux and Android it’s often Vulkan, and on MacOS it’s either Metal or that newer thing they have built specifically for AI. It seems ILGPU only has backends for CUDA, OpenCL, and CPU SIMD. I have general suspicion towards custom compilers. Writing a good compiler is hard, for GPUs even harder due to the weird execution model and insufficient documentation from GPU vendors. HLSL compiler is supported by Microsoft, and compute shaders are used by many millions of gamers every day. Similarly, CUDA is supported by nVidia, usually pretty stable, my only issue with CUDA is vendor lock-in. I have an impression people are often unhappy with the quality and hardware compatibility of less popular GPU APIs like ROCm and OpenCL. BTW, I asked a friend to test my program on their low-and laptop. The laptop has some Intel core i3 with integrated GPU. The performance wasn’t great at about 1 token/second (single-channel memory), but at least my code worked. I’m not sure it would have worked on that computer if the backend was based on OpenCL.

pjmlp 3y ago

Their own technology is DirectML, not raw DirectX 11. https://blogs.windows.com/windowsdeveloper/2023/11/15/elevating-the-developer-experience-on-windows-with-new-ai-tools-and-productivity-tools/ https://blogs.windows.com/windowsdeveloper/2023/11/15/elevat... https://blogs.windows.com/windowsdeveloper/2023/12/14/directml-accelerating-ai-on-windows-now-with-npus/ https://blogs.windows.com/windowsdeveloper/2023/12/14/direct...