> to overfit some dataset that excel at some completely artificial leaderboard metric. This comment exemplifies the myopic narrowminded focus on costs and bein

> to overfit some dataset that excel at some completely artificial leaderboard metric. This comment exemplifies the myopic narrowminded focus on costs and being dismissive of the benefits of greater resource investments which I am criticizing here. Said 'completely artificial' tasks like ImageNet actually turn out to transfer really well to the real world and are why AI is becoming so useful, and also spurred the research into how to create efficient NNs via distillation and compression which still cannot be done without training the large models first. You need to invest if you want efficiency. Performance precedes it, it doesn't follow it. As the programming saying goes: get it correct, then get it fast.