3 ms·
Recursive inputs are bad, but my biggest (maybe) fear is just _obsolete_ or “tired” inputs. I.e., what happens when LLM output is so good that people just stop
by creeble 3y ago
Recursive inputs are bad, but my biggest (maybe) fear is just _obsolete_ or “tired” inputs.
I.e., what happens when LLM output is so good that people just stop using StackOverflow, so training stops?
Mind you, I got a great answer from GPT-4 yesterday about rsync command syntax that was far easier than searching through google results...
- valine 3y agoIts a “restaurant is so busy no ones goes there anymore” scenario. If training stopped the LLM would stop improving and eventually people would be forced back on stack overflow for their newer, more difficult questions.
- stOneskull 3y agomaybe the LLM starts asking the questions on stackoverflow. when the humans get the right answer, the LLM updates itself.
- yomlica8 3y agoIt will create a dark age of no alternative restaurants first though. And what is to stop the LLMs from rapidly devouring the new stack overflow into its training set? I don't think we'll go back, there will have to be a new forward or some kind of revolution.