3 ms·
I asked the same question after several AI talks given by large companies (Google, Nvidia, etc). The general answer is that it's still possible to try out thin
by m3nu 7y ago
I asked the same question after several AI talks given by large companies (Google, Nvidia, etc).
The general answer is that it's still possible to try out things on a single GPU or several servers and many gains come from good features and smart network designs. On the other hand, squeezing out the last 5% does require more data and budget.
Personally, I think you can still do a lot with a moderate budget and smart people. But would love to hear other opinions.
- totoglazer 7y agoLook into the modern nlp models. BERT and its many derivatives, RoBERTa, XLNet. Training all of these require roughly TB of data, and generally take days on multiple TPUs. You often can’t even fine tune on a single GPU without some clever tricks.