4 ms·
But coreml utilizes ANE, right? Is there some bottleneck in coreml that requires lower level access?
by jvstokes 1y ago
But coreml utilizes ANE, right? Is there some bottleneck in coreml that requires lower level access?
- anemll 1y agoMemory bandwidth is the main bottleneck. It got better with M3/M4. ANE is really fast in FLOPS but low in memory bandwidth.