4 ms·

Do you ever have novel ideas while walking or in the shower? Well, you’re learning from data you generate.

nkozyra 2y ago
> Well, you’re learning from data you generate. Sure. I'm producing human output from human input in a generally unconstrained, limitless way. This is producing approximated human output from approximated human input. That second level of abstraction will be constrained, ultimately, by the limits of input.
deleted 2y ago
[deleted]
sottol 2y ago
I don't necessarily think it is the equivalent. Maybe it's more akin to a high-schooler reading a single book and then being asked to write several new books that hold equal weight in teaching the next generation of children. These new text books could be great at simplifying the subject matter and making the material accessible or they may just never have fully understood the materials and are misleading. Now imagine that over and over again, imo it's pretty likely to introduce inaccuracies if just taking a naiive approach.
breck 2y ago
Well put. Also, ever read your old journals? You are training on generated data.
- Filligree 2y ago
  Ever dream? It’s the same thing. Of course we still need real world data, but it seems like generated data should also play a role. Humans don’t weight dreams equally with reality, however, and that’s a distinction I feel is missing here.
a_wild_dandan 2y ago
This is correct. To give more synthetic data examples: 1. Generate inverted problems which are easier to produce than solve. For instance, create an integration math exercise by differentiating a hairy function and reversing the steps. 2. Create (simulated) environmental data. 3. Use adversarial model competition, e.g. self-playing chess or training an artificial image generator/detector model pair. There's evidently a commonplace myth that information quality starts pristine and exclusively gets degraded by systems thereafter. That's just easily, demonstrably false in myriad ways. That's why it's absurd to conclude that LLM output being in internet training data will cause model collapse. Information-rich synthetic data can be created without humans, and it works. (Check out Phi, for instance.)
vineyardlabs 2y ago
https://arxiv.org/pdf/2305.17493 https://arxiv.org/pdf/2305.17493 There's some cursory indication that in the long tail, training LLMs on LLM-generated data causes model collapse. Kind of like how if you photocopy a photocopy too many times the document becomes unreadable. This isn't really surprising though. Neural networks at large are a form of lossy compression. You can't do lossy compression on artifacts recovered from lossy compression too many times. The losses stack.
xboxnolifes 2y ago
Even hear an idea from another person? You just trained one model with another.

> Well, you’re learning from data you generate. Sure. I'm producing human output from human input in a generally unconstrained, limitless way. This is producing approximated human output from approximated human input. That second level of abstraction will be constrained, ultimately, by the limits of input.

deleted 2y ago

[deleted]

sottol 2y ago

I don't necessarily think it is the equivalent. Maybe it's more akin to a high-schooler reading a single book and then being asked to write several new books that hold equal weight in teaching the next generation of children. These new text books could be great at simplifying the subject matter and making the material accessible or they may just never have fully understood the materials and are misleading. Now imagine that over and over again, imo it's pretty likely to introduce inaccuracies if just taking a naiive approach.

breck 2y ago

Well put. Also, ever read your old journals? You are training on generated data.

Filligree 2y ago
Ever dream? It’s the same thing. Of course we still need real world data, but it seems like generated data should also play a role. Humans don’t weight dreams equally with reality, however, and that’s a distinction I feel is missing here.

a_wild_dandan 2y ago

This is correct. To give more synthetic data examples: 1. Generate inverted problems which are easier to produce than solve. For instance, create an integration math exercise by differentiating a hairy function and reversing the steps. 2. Create (simulated) environmental data. 3. Use adversarial model competition, e.g. self-playing chess or training an artificial image generator/detector model pair. There's evidently a commonplace myth that information quality starts pristine and exclusively gets degraded by systems thereafter. That's just easily, demonstrably false in myriad ways. That's why it's absurd to conclude that LLM output being in internet training data will cause model collapse. Information-rich synthetic data can be created without humans, and it works. (Check out Phi, for instance.)

vineyardlabs 2y ago

https://arxiv.org/pdf/2305.17493 https://arxiv.org/pdf/2305.17493 There's some cursory indication that in the long tail, training LLMs on LLM-generated data causes model collapse. Kind of like how if you photocopy a photocopy too many times the document becomes unreadable. This isn't really surprising though. Neural networks at large are a form of lossy compression. You can't do lossy compression on artifacts recovered from lossy compression too many times. The losses stack.

xboxnolifes 2y ago

Even hear an idea from another person? You just trained one model with another.