4 ms·
Is there an equivalent to Moore’s Law for language models? It feels like every week an even bigger (and supposedly better) model is announced.
by gk1 5y ago
Is there an equivalent to Moore’s Law for language models? It feels like every week an even bigger (and supposedly better) model is announced.
- visarga 5y agoScaling Laws for Neural Language Models - https://arxiv.org/abs/2001.08361 https://arxiv.org/abs/2001.08361
- lucidrains 5y agorevised scaling laws https://arxiv.org/abs/2203.15556 https://arxiv.org/abs/2203.15556 https://www.lesswrong.com/posts/midXmMb2Xg37F2Kgn/new-scaling-laws-for-large-language-models https://www.lesswrong.com/posts/midXmMb2Xg37F2Kgn/new-scalin...