2 ms·
ONNX is horrible for anything that has variable input shapes and that is why nobody uses it for LLMs. It fundamentally is poorly designed for anything that does
by imtringued 2y ago
ONNX is horrible for anything that has variable input shapes and that is why nobody uses it for LLMs. It fundamentally is poorly designed for anything that doesn't take a fixed size image.
- zozbot234 2y agoANE itself is also limited to fixed computation "shapes" so I'm not sure how much that would matter practically.