5 ms·
> Today, we announce torch.compile, a feature that pushes PyTorch performance to new heights and starts the move for parts of PyTorch from C++ back into Python.
by quietbritishjim 4y ago
> Today, we announce torch.compile, a feature that pushes PyTorch performance to new heights and starts the move for parts of PyTorch from C++ back into Python.
I'll admit I don't know enough about PyTorch to know what torch.compile is exactly. But does this means some features of PyTorch will no longer be available in the core C++ library? One of the nice things about PyTorch had been that you could do your training in Python then deploy with a pure C++ application.
- microtonal 4y agoOne of the nice things about PyTorch had been that you could do your training in Python then deploy with a pure C++ application. Or even train in C++ or Rust without much loss in functionality.
- synergy20 4y agoRust really has not had any presence in AI training engine yet, it's probably 100% c++.
- microtonal 4y agoI was referring to the libtorch library, which you can use through the tch crate. It is possible to make such rich bindings because so much of Torch is exposed through the C++ API. When more new functionality is moved to Python, it makes it harder to use functionality from the C++ interface and downstream bindings.
- synergy20 4y agoFacebook did similar thing to its original code PHP, it uses HHVM to 'compile' PHP(now called Hacklang) to gain performance, it seems doing similar thing with python here.
- pjmlp 4y agoIf I remember correctly, the JIT version of the PHP compiler proved itself against the AOT compilation to native code via translation to C++, so they went with the more productive variant.
- fddr 4y agoThe `torch.compile` API itself will not be available from C++. That means that you won't get the pytorch 2.0 performance gains if you use it via C++ API. There's no plan to deprecate the existing C++ API, it should keep working as it is. However, a common theme of all the changes is implementing more of pytorch in python (explicitly the goal of primtorch), so if this plan works it could happen in the long run.