3 ms·
You make it sound extremely manual and sequential when reality is anything but. A team with funds like Tesla, Google, FAIR is going to be using NAS and have a
by sumnuyungi 5y ago
You make it sound extremely manual and sequential when reality is anything but.
A team with funds like Tesla, Google, FAIR is going to be using NAS and have a continuous testing pipeline. Tesla has arguably the best environment for continuous testing which is the most difficult part of improving a model. Andrej even said in his talk that their supercomputer is in the top 5 for FLOPs.
SOTA on ImageNet for the past few years has been driven by pre-training on massive datasets. Vision transformers are increasingly more common and are extremely data-hungry.