2 ms·

Is this really true? At least for inference I've seen data showing the trend is rather for smaller models that exploit sparsity.

• pama 6y ago
  This is a chart from before GPT-3: https://www.microsoft.com/en-us/research/blog/turing-nlg-a-17-billion-parameter-language-model-by-microsoft/ https://www.microsoft.com/en-us/research/blog/turing-nlg-a-1...
• nl 6y ago
  Yes, in NLP big models rule and they are getting bigger.