3 ms·
Mojo (and Modular's whole stack) is pretty much completely focused at people who are interested in inference, not training nor research so much at this moment.
by fnands 1y ago
Mojo (and Modular's whole stack) is pretty much completely focused at people who are interested in inference, not training nor research so much at this moment.
So going after people who need to build low latency high-throughput inference systems.
Also as someone else pointed out, they also target all kinds of hardware, not just NVidia.
- pjmlp 1y agoCurrently looks more like CPUs and eventually AMD, from what I have been following up on their YouTube sessions, and whole blog post series about freedom from NVidia and such. They also miss CPUs on Windows, unless using WSL.
- MohamedMabrouk 1y agoAlready GPU code, kernels, and complete models can run on datacenter AMD GPUs using the same code, the same programming model, and same language constructs.
- pjmlp 1y agoLaptops?
- MohamedMabrouk 1y agonot sure, modular is focusing mainly on enterprise applications. but if you look at the current PRs you can see people hacking support for standalone consumer-grade Nvidia and AMD gpus because it is easy, you just add the missing or different intrinsics for the architecture in the lowest level (in pure mojo code) and wire it up in a few places and voila you already program and run code on this GPU. iGPU/Apple GPUs are still not supported yet but it would interesting to see their integration
- patternsplicing 1y agoYes, recent NVIDIA and AMD consumer GPUs are supported: https://docs.modular.com/max/faq/ https://docs.modular.com/max/faq/
- hogepodge 1y agoThere's pretty broad support for server-grade and consumer GPUs. It's a bit buried, but one of the most reliable lists of supported GPUs is in the Mojo info documentation.https://docs.modular.com/mojo/stdlib/gpu/host/info/ https://docs.modular.com/mojo/stdlib/gpu/host/info/
- orochimaaru 1y agoWhy not use PyO3 instead? It had a much cleaner interface than cython and c++ libraries. The primary advantage of mojo seems to be Gil-free syntax that is as close to Python as possible.
- fnands 1y agoGPU programming in Rust isn't great. In Mojo it's pretty much the whole point of the language. If you're only using CPUs, then yeah, PyO3 is a good choice.
- diggan 1y agoWhat about Candle, made by Huggingface? Seems to at least allow the basics and has lots of examples, all of them run on both CPU and GPU. Haven't dived deeper into it, but played around with it a bit and found it good enough for embedding purposes at least.
- GeekyBear 1y agoI think the big value add of Mojo is that you are no longer writing GPU code that only runs on one particular GPU architecture. In the same way that LLVM allows CPU code to target more than one CPU architecture, MLIR/Mojo allows GPU code to target multiple vendor's GPUs. There is some effort required to write the backend for a new GPU architecture, and Lattner has discussed it taking about two months for them to bring up H100 support.
- melodyogonna 1y agoIndeed, but not only GPUs but accelerators in general. Mojo will be able to target weird esoteric hardware (portably if that is important)