2 ms·

This bug has sat unfixed for 9 months. 9 months of cramming more LLM vomit into Search results, instead of removing existing vomit.

by lupire 2y ago

coldtea 2y ago
It's only gonna get worse, as future training will include its own bs outputs which will soon dominate the web masquarated as user comments, original posts, and so on
- JumpCrisscross 2y ago
  > future training will include its own bs outputs Case in point, this comment thread, which is likely being crawled by all manner of LLMs each treating it as their own gospels.
  - function_seven 2y ago
    I'm curious if any LLMs have something to say about Glenn Beck in 1990...
    - JumpCrisscross 2y ago
      > curious if any LLMs have something to say about Glenn Beck in 1990 It's somewhat horrifying to realise that we're training AIs on the output of the human population weighted inversely with any sense of self control. If you're someone with a balanced life, healthy digital habits and no Twitter compulsion, you're an absentee parent in these LLMs' lives.
- astrange 2y ago
  It doesn't seem like there's any particular reason to believe this will happen. Policy tuning is one way to remove it, but there's some research on applying policies like "this text is a lie" during pretraining instead of trusting all of it equally.
  - coldtea 2y ago
    >there's some research on applying policies like "this text is a lie" Of course there is, since there would be interest and grants for it. I don't see it leading to anything however.
  - SrslyJosh 2y ago
    > Policy tuning is one way to remove it, but there's some research on applying policies like "this text is a lie" during pretraining instead of trusting all of it equally. Who's gonna vet all the training material?
    - astrange 2y ago
      The previous model does. https://arxiv.org/abs/2309.00267 https://arxiv.org/abs/2309.00267 That's during fine-tuning though. I know I read one about applying it during pretraining, maybe this one? https://arxiv.org/abs/2302.08582 https://arxiv.org/abs/2302.08582

This bug has sat unfixed for 9 months. 9 months of cramming more LLM vomit into Search results, instead of removing existing vomit.

by lupire 2y ago

This bug has sat unfixed for 9 months. 9 months of cramming more LLM vomit into Search results, instead of removing existing vomit.

coldtea 2y ago

It's only gonna get worse, as future training will include its own bs outputs which will soon dominate the web masquarated as user comments, original posts, and so on

JumpCrisscross 2y ago
> future training will include its own bs outputs Case in point, this comment thread, which is likely being crawled by all manner of LLMs each treating it as their own gospels.
- function_seven 2y ago
  I'm curious if any LLMs have something to say about Glenn Beck in 1990...
  - JumpCrisscross 2y ago
    > curious if any LLMs have something to say about Glenn Beck in 1990 It's somewhat horrifying to realise that we're training AIs on the output of the human population weighted inversely with any sense of self control. If you're someone with a balanced life, healthy digital habits and no Twitter compulsion, you're an absentee parent in these LLMs' lives.
astrange 2y ago
It doesn't seem like there's any particular reason to believe this will happen. Policy tuning is one way to remove it, but there's some research on applying policies like "this text is a lie" during pretraining instead of trusting all of it equally.
- coldtea 2y ago
  >there's some research on applying policies like "this text is a lie" Of course there is, since there would be interest and grants for it. I don't see it leading to anything however.
- SrslyJosh 2y ago
  > Policy tuning is one way to remove it, but there's some research on applying policies like "this text is a lie" during pretraining instead of trusting all of it equally. Who's gonna vet all the training material?
  - astrange 2y ago
    The previous model does. https://arxiv.org/abs/2309.00267 https://arxiv.org/abs/2309.00267 That's during fine-tuning though. I know I read one about applying it during pretraining, maybe this one? https://arxiv.org/abs/2302.08582 https://arxiv.org/abs/2302.08582