4 ms·

Rapid degradation, like recording multiple generations of VHS tapes. The LLMs make the internet dumber, the LLMs get dumber by learning from it. Rinse and repea

by aezart 3y ago

Rapid degradation, like recording multiple generations of VHS tapes. The LLMs make the internet dumber, the LLMs get dumber by learning from it. Rinse and repeat.

valine 3y ago
Maybe, maybe not. The human brain trains on its own output without descending into chaos. It’s not hard to imagine a scheme where you use a model like GPT-4 to filter a dataset before training. Classification is easier than creation, so you’d expect performance to continue to improve regardless of how poisoned the unfiltered dataset becomes. One of the more exciting scenarios would be if it turns out that performance can improve indefinitely with a generate -> filter -> train cycle. There are certainly parallels to how humans learn.
kypro 3y ago
> The LLMs make the internet dumber, the LLMs get dumber by learning from it. I see this said a lot, but in reality it just depends on how the network is trained and how it's prompted. For example, you don't get dumber because you read children's books, you just get better at understanding what makes a good children's book. It's only if reading children's books comes at the cost of reading other content that you might be dumber as a result. Similarly, an AI doesn't automatically get dumber because it encounters dumb content. It's only if you're training exclusively on dumb content that it doesn't know what quality looks like that you'd have problems. Broad training sets (ideally pruned of as much junk as possible) and RLHF in theory should condition the network to reproduce quality content and not simply the lowest common denominator of what's found on the internet. And assuming all that fails there's nothing stopping researchers from just using past datasets with improved architecture going forward. I mean you'd have to wonder why on Earth OpenAI would even release GPT-5 if it's worse than GPT-4... There's just no scenario here in which what you're saying would actually play out in reality. One way or another companies will ensure the next iteration of their LLMs are better than their previous.
- alwaysbeconsing 3y ago
  > For example, you don't get dumber because you read children's books, you just get better at understanding what makes a good children's book. I don't think this is an accurate analogy. It's not books for children -- well-written material at a lower educational/cognitive level -- it's more like books by children -- which is necessarily lacking skill and background context (connection to reality). Think about how children constantly pass around misleading, invented, and incorrect stories among themselves -- and they don't do it maliciously, they just don't know any better. Legends like "Candyman"/"Bloody Mary" for example. They need an outside influence, an adult or a book or website, to nudge them out of that knowledge rut. (Of course the same thing can happen with a (closed) group of adults, too, but it's more of a "natural state" with children because they simply haven't had time to encounter as much knowledge.)

by aezart 3y ago

Rapid degradation, like recording multiple generations of VHS tapes. The LLMs make the internet dumber, the LLMs get dumber by learning from it. Rinse and repeat.

valine 3y ago

Maybe, maybe not. The human brain trains on its own output without descending into chaos. It’s not hard to imagine a scheme where you use a model like GPT-4 to filter a dataset before training. Classification is easier than creation, so you’d expect performance to continue to improve regardless of how poisoned the unfiltered dataset becomes. One of the more exciting scenarios would be if it turns out that performance can improve indefinitely with a generate -> filter -> train cycle. There are certainly parallels to how humans learn.

kypro 3y ago

> The LLMs make the internet dumber, the LLMs get dumber by learning from it. I see this said a lot, but in reality it just depends on how the network is trained and how it's prompted. For example, you don't get dumber because you read children's books, you just get better at understanding what makes a good children's book. It's only if reading children's books comes at the cost of reading other content that you might be dumber as a result. Similarly, an AI doesn't automatically get dumber because it encounters dumb content. It's only if you're training exclusively on dumb content that it doesn't know what quality looks like that you'd have problems. Broad training sets (ideally pruned of as much junk as possible) and RLHF in theory should condition the network to reproduce quality content and not simply the lowest common denominator of what's found on the internet. And assuming all that fails there's nothing stopping researchers from just using past datasets with improved architecture going forward. I mean you'd have to wonder why on Earth OpenAI would even release GPT-5 if it's worse than GPT-4... There's just no scenario here in which what you're saying would actually play out in reality. One way or another companies will ensure the next iteration of their LLMs are better than their previous.

alwaysbeconsing 3y ago
> For example, you don't get dumber because you read children's books, you just get better at understanding what makes a good children's book. I don't think this is an accurate analogy. It's not books for children -- well-written material at a lower educational/cognitive level -- it's more like books by children -- which is necessarily lacking skill and background context (connection to reality). Think about how children constantly pass around misleading, invented, and incorrect stories among themselves -- and they don't do it maliciously, they just don't know any better. Legends like "Candyman"/"Bloody Mary" for example. They need an outside influence, an adult or a book or website, to nudge them out of that knowledge rut. (Of course the same thing can happen with a (closed) group of adults, too, but it's more of a "natural state" with children because they simply haven't had time to encounter as much knowledge.)