4 ms·
How is this going to support custom layers like variations of (flash) attention that every company seems to introduce? Would it mean one won't be able to run a
by storus 1y ago
How is this going to support custom layers like variations of (flash) attention that every company seems to introduce? Would it mean one won't be able to run a specific model (or only have its bastardized version) until MS implements it in the runtime?
- Maxious 1y agoONNX already includes an escape hatch for this, there are official Microsoft maintained Execution Providers https://onnxruntime.ai/docs/execution-providers/ https://onnxruntime.ai/docs/execution-providers/ But vendors like Arm/rockchip/Huawei can also write them https://onnxruntime.ai/docs/execution-providers/community-maintained/ https://onnxruntime.ai/docs/execution-providers/community-ma...