3 ms·
How many people here actually write custom cuda/triton kernels? An extremely small hand full of people write them (and they're all on one discord server).. whic
by lordofgibbons 1y ago
How many people here actually write custom cuda/triton kernels? An extremely small hand full of people write them (and they're all on one discord server).. which then gets integrated into Pytorch, Megatron-LM, vLLM, SGLang, etc. The rest of us in the ML/AI ecosystem have absolutely no incentive to migrate off of python due to network effects even though I think it's a terrible language for maintainable production systems.
If Mojo focuses on systems software ( and gets rid of exceptions - Chris, please <3 ) it will be a serious competitor to Rust and Go. It has all the performance and safety of Rust with a significantly easier learning curve.
- huevosabio 1y agoWhich Discord server? I want in!
- chrislattner 1y agoThe Mojo discord and forums are all listed here: https://www.modular.com/community https://www.modular.com/community
- fprog 1y agoNot OP, but my guess is GPU MODE. https://discord.gg/gpumode https://discord.gg/gpumode
- achierius 1y agoPlenty of people do, many more than are in that server -- I asked some of my former coworkers and none knew about it, but we all spent a whole lot of hours tuning CUDA kernels together :). You have one perspective on this sector, but it's not the only one! Some example motivations: - Strange synchronization/coherency requirements - Working with new hardware / new strategies that Nvidia&co haven't fine-tuned yet - Just wanting to squeeze out some extra performance
- chrislattner 1y agoMojo doesn't have C++-like exceptions, but does support throwing. The codegen approach is basically like go's (where you return a bool + error conceptually) but with the python style syntax to make it way more ergonomic than Go. We have a public roadmap and are hard charging about improving the language, check out https://docs.modular.com/mojo/roadmap/ https://docs.modular.com/mojo/roadmap/ to learn more. -Chris
- latemedium 1y agoI think part of the reason why just a few people write custom CUDA / triton kernels is that it's really hard to do well. Languages like Mojo aim to make that much easier, and so hopefully more people will be able to write them (and do other interesting things with GPUs that are too technically challenging right now)
- dogma1138 1y agoThe only question will there be a benefit in writing your own kernels in something like Mojo than to skip that part altogether and use the primitives with already highly optimized kernels that frameworks like torch provide especially when it comes to performance.
- adastra22 1y agoProbably tens of thousands of people. You do know that CUDA is used for more than just AI/ML?
- pjmlp 1y agoI guess given all the hype people tend to forget why GPGPU is used for, it is like the common memme of why CUDA when there is PyTorch.
- pzo 1y agoIf Mojo is superset of Python same like TypeScript is superset of Javascript I would really be willing to use Mojo even if just for the option to have support for mobile platforms. Right now python trying to support iOS and Android but it would take years before people will mange to compile at least so of the ecosystem.