3 ms·

Thank you for making this comment, because it exposes some logical gaps. Firstly, Go, Chess, and other games have objective rules and win criteria. (There is n

Thank you for making this comment, because it exposes some logical gaps. Firstly, Go, Chess, and other games have objective rules and win criteria. (There is no “subjective opinion” as to whether Fischer or Spassky won their match.) Language, the output of LLMs, does not have an objective function. Consider the following to sentences: “My lips, two blushing pilgrims, ready stand.” “My red lips are ready to kiss your red lips.” Both are grammatically correct English sentences and both mean basically the same thing, but clearly the former by Shakespeare has a subjective poetic quality which the latter lacks. Even if we make evaluation rules to target (for example, “do not repeat phrases”, “use descriptive adjectives”, etc.) AI still seems to favor certain (for example “delve”) that are valid but not commonly used in human-originated English. There is then a positive feedback loop where these preferences are used to further train models, hence the next generation of models have no way of knowing whether the now frequent usage of “delve” is a human-originated or AI-originated phenomenon. Lastly, regarding works of fiction, the concern is less about the content of stories—though that is also a concern—but more about the quality of language. (Consider above alternate take on Romeo and Juliet, for example.)

patrickhogan1 2y ago
So you are arguing that the world does not have objective rule criteria, like Physics? And that an AI could not model the world and then run simulations and have each simulation generate data and learn from that, similar to AlphaZero. Here is a possible objective win environment Model complex multicellular organisms that become capable of passing the Turing test that also self-replicate.
- dudeinjapan 2y ago
  My argument here is narrowly scoped to human language and literature. (We already know the objective rule criteria of life is 42.) It may very well be possible for an AI to read all of literature, and figure out what makes Hemingway, Tolstoy, and Dylan "good writing" vs. "bad writing". That has not yet been achieved. The problem, as the OP implies, is that by polluting the universe of literature with current-gen AI output, we may making the task of generating "good writing" in the future harder. Then again, maybe not. Perhaps we have enough pre-AI works that we can train on them versus the mountains of AI generating schlock, and determine the objective function.
  - patrickhogan1 2y ago
    You seem to restate the argument of the submitted research paper in a narrow interpretation that is much different than its main conclusion of Model Collapse from synthetic data creation. But I will follow you down this interpretation. Why does a human have to judge something as good for it to be functionally useful? Humans never came up with the move that AlphaZero used to win 1 of the 4 out of 5 games it won. Was that a bad move? Did AlphaZero devolve to Model Collapse because it made that move?
    - patrickhogan1 2y ago
      And what’s interesting here is people will get annoyed with this. Am I pro-human? Absolutely Can non-human outputs be functionally valuable to humans and not per se harmful? Absolutely, we live in a biosphere that humans didn’t create or if humans did create it, it was humans more evolved than humans we are aware of.

patrickhogan1 2y ago

So you are arguing that the world does not have objective rule criteria, like Physics? And that an AI could not model the world and then run simulations and have each simulation generate data and learn from that, similar to AlphaZero. Here is a possible objective win environment Model complex multicellular organisms that become capable of passing the Turing test that also self-replicate.

dudeinjapan 2y ago
My argument here is narrowly scoped to human language and literature. (We already know the objective rule criteria of life is 42.) It may very well be possible for an AI to read all of literature, and figure out what makes Hemingway, Tolstoy, and Dylan "good writing" vs. "bad writing". That has not yet been achieved. The problem, as the OP implies, is that by polluting the universe of literature with current-gen AI output, we may making the task of generating "good writing" in the future harder. Then again, maybe not. Perhaps we have enough pre-AI works that we can train on them versus the mountains of AI generating schlock, and determine the objective function.
- patrickhogan1 2y ago
  You seem to restate the argument of the submitted research paper in a narrow interpretation that is much different than its main conclusion of Model Collapse from synthetic data creation. But I will follow you down this interpretation. Why does a human have to judge something as good for it to be functionally useful? Humans never came up with the move that AlphaZero used to win 1 of the 4 out of 5 games it won. Was that a bad move? Did AlphaZero devolve to Model Collapse because it made that move?
  - patrickhogan1 2y ago
    And what’s interesting here is people will get annoyed with this. Am I pro-human? Absolutely Can non-human outputs be functionally valuable to humans and not per se harmful? Absolutely, we live in a biosphere that humans didn’t create or if humans did create it, it was humans more evolved than humans we are aware of.