2 ms·

> Training an LLM on content produced by the LLM gives it the equivalent of a digital prion disease Is it foolish of me to have expected more from a blog post

> Training an LLM on content produced by the LLM gives it the equivalent of a digital prion disease Is it foolish of me to have expected more from a blog post on kernel.org?

theandrewbailey 1mo ago
Are you trying to say that's bad writing? I think it's a good metaphor for a documented phenomenon: https://en.wikipedia.org/wiki/Model_collapse https://en.wikipedia.org/wiki/Model_collapse
- Lerc 1mo ago
  As the article states, this phenomenon may be documented, but there is no consensus that it describes any practical reality. The predicted consequences have now had time to manifest, and have not done so. This makes the claim either false or overstated. Perhaps there will be issues in the future, but to date there have been many claims that AI development will stall (for a variety of reasons). If they were the critical weaknesses they have been portrayed as, models would not have advanced to the level they are today. If you have a hypothesis, make a clear prediction based upon it. If you start pushing the date forward after each failed prediction, you end up looking like a hapless doomsday cult. If your hypothesis is correct however, your prediction should actually happen. Then provided you have not made so many predictions to get one right by chance, people will take what you have to say seriously.
  - desterothx 1mo ago
    You state it as if its only the quality of the hypothesis that matters, but you are ignoring an important part of it, timing. During the 08 financial crisis Burry had a hypothesis that was correct, however he almost went bankrupt still because he thought it would happen earlier than it did because of the government bailouts. He was pushing the day forward, and was looking like a "hapless doomsday cult". His hypothesis still turned out correct
    - Lerc 1mo ago
      If you can't say when something will happen you are just playing with statistics on another domain. Seems like this chaps problem was that the actions were based upon a different hypothesis than the one he stated. X will cause Y by the end of the year is considerably different to X will eventually cause Y.
- petesergeant 1mo ago
  Training on large quantities of LLM-generated synthetic data is an important part of training LLMs.

> Training an LLM on content produced by the LLM gives it the equivalent of a digital prion disease Is it foolish of me to have expected more from a blog post on kernel.org?

theandrewbailey 1mo ago

Are you trying to say that's bad writing? I think it's a good metaphor for a documented phenomenon: https://en.wikipedia.org/wiki/Model_collapse https://en.wikipedia.org/wiki/Model_collapse

Lerc 1mo ago
As the article states, this phenomenon may be documented, but there is no consensus that it describes any practical reality. The predicted consequences have now had time to manifest, and have not done so. This makes the claim either false or overstated. Perhaps there will be issues in the future, but to date there have been many claims that AI development will stall (for a variety of reasons). If they were the critical weaknesses they have been portrayed as, models would not have advanced to the level they are today. If you have a hypothesis, make a clear prediction based upon it. If you start pushing the date forward after each failed prediction, you end up looking like a hapless doomsday cult. If your hypothesis is correct however, your prediction should actually happen. Then provided you have not made so many predictions to get one right by chance, people will take what you have to say seriously.
- desterothx 1mo ago
  You state it as if its only the quality of the hypothesis that matters, but you are ignoring an important part of it, timing. During the 08 financial crisis Burry had a hypothesis that was correct, however he almost went bankrupt still because he thought it would happen earlier than it did because of the government bailouts. He was pushing the day forward, and was looking like a "hapless doomsday cult". His hypothesis still turned out correct
  - Lerc 1mo ago
    If you can't say when something will happen you are just playing with statistics on another domain. Seems like this chaps problem was that the actions were based upon a different hypothesis than the one he stated. X will cause Y by the end of the year is considerably different to X will eventually cause Y.
petesergeant 1mo ago
Training on large quantities of LLM-generated synthetic data is an important part of training LLMs.