5 ms·
Yep, we're not claiming this is the compute required for DL, and for specific tasks we expect compute required to fall over time. But better algorithms actuall
by geoffreyirving 8y ago
Yep, we're not claiming this is the compute required for DL, and for specific tasks we expect compute required to fall over time. But better algorithms actually mean compute is more important, not less, and would likely make the growth in available compute more important.
For example, if a task is parameterized (by size or difficulty, say), then a better algorithm might change the asymptotic complexity from O(n^3) to O(n^2). A 2x compute increase for the old algorithm would take us from n -> 1.25n, but the new algorithm would go from n -> 1.41n.