3 ms·
Intel OpenVINO is also quite punchy for CPU inference.
by touisteur 3y ago
Intel OpenVINO is also quite punchy for CPU inference.
- ikhatri 3y agoYeah I've heard of it but never used it. Looks like they have a backend/runtime for ONNX models as well (https://pypi.org/project/onnxruntime-openvino/ https://pypi.org/project/onnxruntime-openvino/) neat! ONNX really is the universal format. If you can get your model exported to ONNX, running it on various platforms becomes much easier.* *as long as every hardware platform supports the ops you use in your network and you're not doing anything too fancy/custom :P
- touisteur 3y agoYeah I've only used it with networks in ONNX format (converted from tensorflow or torch). I was looking for high perf low latency / real-time, the C or C++ APIs for OpenVINO are quite OK if you spend some time playing with it. I hope Intel keeps investing on it... Edit: often if you go through the ONNX intermediate format, be prepared to perform some 'network surgery' to clean up some conversion cruft, but also to remove training-only stuff left in the network...