4 ms·
I've tried inference on the python version of onnx and it usually varies between hitting a OOM limit (while with TF it works fine) to being an order of magnitud
by tvirosi 5y ago
I've tried inference on the python version of onnx and it usually varies between hitting a OOM limit (while with TF it works fine) to being an order of magnitude slower. Even if the codebase is still being changed I don't see much reason for people to use it other than as a convenient distribution format.
- akhundelar 5y agoInteresting, I did not encounter such discrepancies in my work with these tools. There could be multiple reasons for the degraded performance: - Are we comparing apples to apples here (heh), e.g. ResNet-50 vs ResNet-50? - Was the ONNX model ported from TF? There are known issues with that path (https://onnxruntime.ai/docs/how-to/tune-performance.html#my-converted-tensorflow-model-is-slow---why https://onnxruntime.ai/docs/how-to/tune-performance.html#my-...) - Have you tried tuning an execution provider for your specific target platform?(https://onnxruntime.ai/docs/reference/execution-providers/#summary-of-supported-execution-providers https://onnxruntime.ai/docs/reference/execution-providers/#s...)