3 ms·
I have been mulling over the idea of starting a project that reads ONNX files and performs a codegen on the model, building model inference code directly that d
by mynameisash 5y ago
I have been mulling over the idea of starting a project that reads ONNX files and performs a codegen on the model, building model inference code directly that doesn't rely upon an ONNX runtime. In my experience, hooking up a runtime requires a lot of effort, and inference is often fairly latent. I'm wondering if it's feasible to generate code offline from an .onnx model then compile the generated code directly into whatever project is using it.
- vivegi 5y agoCool. I have thought about the same thing a while ago, but held back. The code gen you are talking about is for data wrangling and shaping, right? I used ML.net from MSFT and it comes with an ONNX runtime that is quite easy to use, esp if we are just using the model for predictions. The data shaping was not complex. That's why I held back. (See the first link below) https://docs.microsoft.com/en-us/azure/machine-learning/how-to-use-automl-onnx-model-dotnet https://docs.microsoft.com/en-us/azure/machine-learning/how-... This one has a bit of involved processing https://docs.microsoft.com/en-us/dotnet/machine-learning/tutorials/object-detection-onnx https://docs.microsoft.com/en-us/dotnet/machine-learning/tut...
- freemint 5y agoI think Julia's ecosystem FluxML/ONNX.jl does just that and writes it into a tape. Turning Julia into standalone executables is less fun, so you are probably stuck with Julia run time which is probably bigger?