4 ms·

> Meh it's poorly supported by both PyTorch and TF. This does not match my experience. Most new model architectures port fine to ONNX from pytorch, only occasi

by Vetch 3y ago

> Meh it's poorly supported by both PyTorch and TF. This does not match my experience. Most new model architectures port fine to ONNX from pytorch, only occasionally having to fill in rare functions. > Not even by a long-shot - first party compilers are generally faster because of smoother interop but even amongst third-party you have TRT and TVM. Again, in my experience, there is little by way of documentation for any of these platforms nor as broad support for as many OS and Hardware combinations. > I have no idea what anyone uses ONNX for these days Huggingface has strong support for ONNX and leverages it for improved performance in places. https://huggingface.co/docs/optimum/onnxruntime/usage_guides/trainer https://huggingface.co/docs/optimum/onnxruntime/usage_guides... https://huggingface.co/docs/optimum/onnxruntime/usage_guides/models https://huggingface.co/docs/optimum/onnxruntime/usage_guides... It's curious how impressions can differ so.

radarsat1 3y ago
It's not poorly supported by pytorch but you do have to have it in mind while writing your code. I've been having a hell of a time getting a decent ONNX model out of some pytorch code that was written with lots of dynamic logic inside the modules, it can be quite some work converting this to the necessary "static graph" design needed to export to ONNX properly. What I'd love to see is a lean, inference-only version of pytorch that just works with existing code, and is smaller without the overhead of autograd and GPU support.