4 ms·
One is the UX, which they don't care about because people use their models anyway. The second would be hardware savings on the order of tens of billions of dol
by oblio 3mo ago
One is the UX, which they don't care about because people use their models anyway.
The second would be hardware savings on the order of tens of billions of dollars if they were stupid not do try all possible optimizations.
Spot the difference.
- joefourier 3mo agoYou cannot just "try all possible optimisations". It takes time, effort, and money that could otherwise be spent elsewhere (especially for training, where each training run is especially costly, and optimisations might be promising early on, but cause the final performance of the model to be worse). You need smart people interested in unglamorous work, and if you're swimming in VC money, it's far more straightforward to just throw more GPUs at the problem.