TVM is all of this. It even has compilation support for exotic devices like FPGAs, phone ASICs and such. MLIR frameworks are heading this direction too. > its

TVM is all of this. It even has compilation support for exotic devices like FPGAs, phone ASICs and such. MLIR frameworks are heading this direction too. > its trivial to quantize to your liking ...Except its not implemented in the demo. Also, quantization is far from simple. All this sounds good, but I have seen cool sounding ONNX demos for years, and (outside of some fairly quick one off TensorRT demos) I havent really seen the pavement hit the road.