3 ms·
I bet they'll only train on the internet snapshot from now, before LLMs. Additional non-internet training material will probably be human created, or curated a
by sejje 10mo ago
I bet they'll only train on the internet snapshot from now, before LLMs.
Additional non-internet training material will probably be human created, or curated at least.
- sosodev 10mo agoNope. Pretraining runs have been moving forward with internet snapshots that include plenty of LLM content.
- sejje 10mo agoSure, but not all of them are stupid enough to keep doing that while watching the model degrade, if it indeed does.
- pc86 10mo agoThis only makes sense if the percentage of LLM hallucinations is much higher than the percentage of things written on line being flat wrong (it's definitely not).