5 ms·
Nvidia Tilus: A Tile-Level GPU Kernel Programming Language
- FilosofumRex 1y agoseriously!!! there is so much more Nvidia and its billionaire managers can do to improve developers experience with CUDA/nvcc/PTX, instead of yet another barely functional, sparsely documented, rarely tested DSL
- snarfy 1y ago"...made incremental improvements to CUDA" just doesn't get you promoted like "creator of GPU Kernel Programming language Tilus"
- skavi 1y agoWhat’s the relationship between Tilus and cuTile (and maybe Warp)? A slight tangent, but I really wish Nvidia would release more details on Tile IR. Specifically on what it enables vs PTX. Is it just about moving towards more MLIR based infra? Maybe it’s higher level and thus can enable better codegen across generations?
- dachworker 1y agoMy impression of NVIDIA is that internally the teams are quite independent and there's not much in terms of broad strategy. So it could be these two efforts are not related.
- dagmx 1y agoThis is accurate. A lot of projects that come out are each teams research projects unless they’re tied to a specific product which is when they start working closer together and integrating further. However this repo was specifically part of their acquisition of CentML
- alwahi 1y agookay, I am not a systems level programmer but I am currently learning c with the aim of doing some gpgpu programming using Cuda etc., what is a tile level gpu kernel programming language? and how is it different from something like cuda? I know i can ask a llm or search on google, but i was hoping someone in the community could explain it in a way i could understand.
- taminka 1y agoprogramming w/ tiles means working w/ contiguous-ish blocks/squares of data, to minimise cache misses (often major bottleneck in gpu programming), so it's just a nod to the fact that optimisations like these are included in the language or some such...
- atq2119 1y agoI'd say the main difference is that in traditional GPU languages, the thread of execution is a single lane of a warp or wave. You typically work with ~fp32-sized values, and those are mapped by the compiler to one lane of a 32-wide vector register in a wave (or 16- to 128-wide depending on the architecture). Control flow often has to be implemented through implicit masking as different threads mapped to lanes of the same vector can make different control flow decisions (that is, an if statement in the source program gets compiled to an instruction sequence that uses masking in some way - the details vary by vendor). In tile languages, the thread of execution is an entire workgroup (or block in CUDA-speak). You typically work with large vector/matrix-sized values. The compiler decides how to distribute those values onto vector registers across waves of the workgroup. (Example: if your program has a value that is a 32x32 matrix of fp32 elements and a workgroup has 8 32-wide waves, the value will be implemented as 4 standard-sized vector registers in each wave of the workgroup.) All control flow affects the entire workgroup equally since the ToE is the entire workgroup, and so the compiler does not have to do implicit masking. Instead, tile languages usually have provisions for explicit masking using boolean vectors/matrices. Tile languages are a new phenomenon and clearly disagree on what the exact level of abstraction should be. For example, Triton mostly hides the details of shared memory from the programmer and lets the compiler take care of software pipelined loads, while in this Tilus here, it looks like the programmer has to program shared memory use explicitly.
- Archit3ch 1y agoI'm hoping that something like MoYe.jl moves from Nvidia-only to a vendor-agnostic tile DSL.
- lukax 1y agoNVIDIA does not want CUDA development (e.g. flash attention) to move to Triton because Triton also supports AMD and if ecosystem moves from pure CUDA to Triton, that's bad for NVIDIA's lock-in. That's why there is so much focus on CUDA Python (lower level) and Tilus (higher level, more similar to Triton).
- archerx 1y agoWhat’s bad for Nvidia is good for everyone else. The cuda lock-in needs to die.
- pjmlp 1y agoIt is on AMD and Intel to deliver.
- ants_everywhere 1y agoOr one of the cloud providers who doesn't want to pay lock-in prices when they'd rather pay commodity prices
- topspin 1y agoHow feasible is this for a cloud operation(s)? I imagine this work requires close collaboration with the architects and proprietary knowledge about the design.
- ants_everywhere 1y agoit seems feasible, it's more a matter of how much of a priority it is. I follow Google most closely. They design and manufacture their own accelerators. AWS I know manufactures its own CPUs, but I don't know if they're working on or already have an AI accelerator. Several of the big players are working on OpenXLA, which is designed to abstract and commoditize the GPU layer: https://openxla.org/xla https://openxla.org/xla OpenXLA mentions: > Alibaba, Amazon Web Services, AMD, Apple, Arm, Google, Intel, Meta, and NVIDIA
- ddtaylor 1y agoNvidia lock in attempt.
- Keyframe 1y agoSo, SIMD but in python and on GPU?
- camdroidw 1y agoI'm a little out of the loop these days but I thought webgpu obviated the use of platform specific language (say Cuda)?
- lights0123 1y agoThere are a lot of hardware features inaccessible from webgpu because its devs still have a lot of work to do for the current implementations—no browser is even shipping it on Linux by default. Vulkan is closer, but CUDA still exposes more features.
- pjmlp 1y agoNot quite right, Chrome is shipping with support on ChromeOS, Android and WebOS. It is GNU/Linux that Google/Chrome sees as low priority.
- bobajeff 1y agoWebGPU is a API that sits above Vulkan, DirectX and Metal. For CUDA, ROCm and oneAPI there is SYCL. But it's not used too much and it seems mostly just Intel is invested.
- pjmlp 1y agoAdditionally, designed for 2015 hardware capabilities, with JavaScript in mind, in a sandboxed environment. Some folks are reaching out to WebGPU outside the browser, because Vulkan is a pain to program for, and they misuse WebGPU as middleware, although a less capable one, due to its original design, and who is driving its standardisation process. Regarding SYCL, Intel basically bought the only company that was shipping a good developer experience for it, CodePlay, which used to do specialized compilers for game consoles, and pivoted into GPGPU.
- m-schuetz 1y agoWebGPU is the lowest common denominator over ancient mobile phones and modern hardware; as well as Vulkan, DirectX and Metal; effectively making it an ancient-mobile-phone-graphics-API. It is at the time of its releas as outdated as WebGL was when it came out.
- pjmlp 1y agoThe recent interest on NVidia making Python first class on CUDA ecosystem is what makes me wonder how successful Mojo migth becomem, if they aren't faster than NVidia with Python. All the attempts to attack CUDA fail to undestand why most researchers flock to it, instead of enduring the pain of the competion tooling, and they tend to focus on a single aspect of CUDA, be it C++, or something else, but never the polyglot support, the libraries, the IDE integration, the graphical debuggers, the compiler backends for other developers to target CUDA.
- curtisszmania 1y ago[dead]
- drdirk 1y agoThe linked paper in the GitHub repository doesn’t contain an NVIDIA email. There is an Amazon email and a bunch of university emails. How come that this paper has become an NVIDIA project? Paper: https://arxiv.org/pdf/2504.12984 https://arxiv.org/pdf/2504.12984
- moralestapia 1y agoNew thing to experiment with. Great :).
- coderatlarge 1y agobut why use “~float16” for something that in its own documentation the authors describe as a float16* ?