4 ms·
How long does this typically take? It sounds time consuming. Also, it seems like this could be similar to doing a GA?
by UncleOxidant 1y ago
How long does this typically take? It sounds time consuming. Also, it seems like this could be similar to doing a GA?
- jakestevens2 1y agoThat depends on the model architecture and how it was written since that informs the size of the search space. The typical range is 10 mins to 10 hours. It won't be fast but you only have to do it once and then those optimizations are set for every forward pass.
- sitkack 1y agoDo you learn the capabilities of the underlying hardware relative to the kernel src? You should be able to start predicting perf using learned static profiling.
- jakestevens2 1y agoNot today but we will implement memoization of kernels for each hardware backend, yes.
- jakestevens2 1y agoYou can also set a time budget for how long you'd like the search to run for to avoid wasting time on diminishing returns.
- s40078920 1y ago[dead]