3 ms·
Well, you can't train LLM on "facts alone", since they are basically throwing darts at the next word, not using information to craft a message. But I hear your
by jvanderbot 4y ago
Well, you can't train LLM on "facts alone", since they are basically throwing darts at the next word, not using information to craft a message. But I hear your larger point: it'd be nice to have an agent that was trained on wikipedia information, but already spoke a language
- r3trohack3r 4y agoGPT is trained on Wikipedia though. It’s speculated libgen is in there as well.
- artificial 4y agoLet’s not forget Twitter was a training source. Early Twitter was like 4chan.
- jvanderbot 4y agoI think GGP comment started this thread about a knowledge engine, not a "word generator" if that makes sense. Training on Wikipedia in this case means "training to put together words like wikipedia authors do" right? Not "learning the facts of wikipedia to provide useful synthesis and cross referencing." though it probably does OK by accident.