3 ms·
I was going to answer the same, I find the approach of machine learning compilers that directly compile models to host and device code better than having to bri
by lairv 3y ago
I was going to answer the same, I find the approach of machine learning compilers that directly compile models to host and device code better than having to bring a huge runtime. There are exciting projects in this area like TVM Unity [1], IREE [2], or torch.export [3]
[1] https://github.com/apache/tvm/tree/unity https://github.com/apache/tvm/tree/unity
[2] https://github.com/openxla/iree https://github.com/openxla/iree
[3] https://pytorch.org/get-started/pytorch-2.0/#inference-and-export https://pytorch.org/get-started/pytorch-2.0/#inference-and-e...
- brucethemoose2 3y agoThis is the precise pitch for Facebook's AITemplate as well.
- mathisfun123 3y agoNope AITemplate is just a tracing mechanism plus cutlass plus autotuning. You still need a runtime. AIT also works on very few models (it's a replacement for something called "static runtime"). It's surprising to me how many people just shoot from the hip/guess on this stuff. IKYK and if you don't maybe don't guess?
- chillee 3y agoNo, AIT is a runtime - it exports a .so that you can load from C++ and call how you like.