3 ms·
No, there are mathematical reasons LLMs are better. They are trained with multiobjective loss (coding skills, translation skills, etc) so they understand the wo
by nshm 2y ago
No, there are mathematical reasons LLMs are better. They are trained with multiobjective loss (coding skills, translation skills, etc) so they understand the world much better than MLM. Original post discuss that but with more words and points than necessary.
- Der_Einzige 2y agoCall it a CLM vs MLM, not LLM vs MLM. Soon LMLM's will exist, which will be LLMs too...
- visarga 2y agoGPTs also get gradients from all tokens, BERT only on 15% masked tokens. GPTs are more effective.