4 ms·

> Does it, really? Yes. It's information theory. You can add information, by making rules like the class doesn't change if the thing is a different color or s

> Does it, really? Yes. It's information theory. You can add information, by making rules like the class doesn't change if the thing is a different color or size or position, or on a different background. But you can't automatically create new images from a distribution that has been learned and expect them to add information.

flangola7 3y ago
Then how do humans do it? Our knowledge relies heavily on sending the output of humans to the input of humans (verbal communication, books, self-thought, etc). At one point total human knowledge was little more than "some berries are bad" but through this recursive process arrived at the rich garden of information we have today.
- version_five 3y ago
  Humans have nothing in common with neural networks (other than if you choose to misunderstand the literal definition of those two words), there's no direct analogy to be made.
  - flangola7 3y ago
    I didn't say anything about neural networks, I asked how humans do it.
    - tempusalaria 3y ago
      To start with, humans have access to far more data than just what is generated by other humans. We can look at nature, other animals, mathematical laws and many other things. Humans also have far more ability than current model architectures to use memory (particularly in training), flexible context (Model context always directly follows the input distribution in training whereas we can flexibly combine different parts of our input data as we like ) and logic. We can also do live learning in ways current models can’t.
- cubefox 3y ago
  Humans don't just try to imitate text from some group of humans, like language models. But if we wanted to imitate the text of, say, 18th century novelists, then only those 18th century novels seem to matter. If we also started to imitate our own imitations of 18th century novels (synthetic data) the result would probably get worse, since our imitations are imperfect.
  - cguess 3y ago
    To consider your analogy a bit further: Imagine you had access to only... five 18th century novels, that's it, no other knowledge about the period, the people or anything. Then you try to write your own. You can make up your own stories that are a merging of various parts of those five novels, but they'll always only contain what's in those, or some combination. If all the books deal with France then Germans or English will never make an appearance because it's literally impossible to guess that they exist since they're not in the original books (the books here are training data if that's not obvious).
- time_to_smile 3y ago
  Because humans (and your language is confusing here between individual humans as models and human society as the model) are constantly receiving new information. Even before humans learned "some berries are bad" they initially thought either "all known eaten berries are bad" or "all known eaten berries are good", then somebody at a good/bad berry and updated their believes (i.e. their model of the world). Humans learn from other humans because humans don't individually share the same information and model of the world. A science teach can speed up how you learn science by taking the compressed information and explaining it quickly (essentially what is happening in the post), but out scientific model is expanded when we have experiences that call into question the strength of our current model. However both individually and as a society we are constantly taking in new information (sometimes more sometimes less) and using that to update our model.
kadoban 3y ago
How does this train of thought apply to something like AlphaZero? There, computation/time seems to be spent creating data. Why can't the same thing in concept apply here?
- version_five 3y ago
  Good question. I'm not an RL expert, but those kind of models are sampling the possible space of game states, based on the rules of the game, to learn a winning policy. Complex games can have effectively infinite different states. What we're talking about here is approximating a distribution from a finite and comparatively small set of images. There's no "game" you can play to get new images, the training data is fixed. The analogy might be say sampling images from a computer animation, which shows the object your interested in in different plausible poses and backgrounds, lighting, angles. There's a question of making sure this generalizes to "real" images, but that can help. But it relies on the animators knowledge of what the object can look like in different poses. That's how information gets added.
bheadmaster 3y ago
> You can add information, by making rules like the class doesn't change if the thing is a different color or size or position, or on a different background. But you can't automatically create new images from a distribution that has been learned and expect them to add information. I am not quite sure I follow. Why would any of those things be true for all models?
sebzim4500 3y ago
Here's a scenario where training on the output of a model could improve its own performance: Suppose you want to optimize the ability for an LLM to do arithmetic. You prompt the LLM to generate a bunch of arithmetic questions, prompting it to show its working. Then you take that output, remove the intermediate steps, and train on the results. I think you would improve some benchmarks by doing this, possibly at the expense of others. I'm sure you could do similar things with other types of model, but I think thissimple example shows the point adaquately. In any case, information theory is not especially relevant here. It puts an upper bound on model performance but we are so incredibly far away from that bound it isn't worth worrying about.
- time_to_smile 3y ago
  > You prompt the LLM to generate a bunch of arithmetic questions, prompting it to show its working. Then you take that output, remove the intermediate steps, and train on the results. Removing or editing the output of the model is providing new information to the model, what's improving the performance of the model in this scenario is that you are explicitly adding new information and fine tuning it on these new cases. > information theory is not especially relevant here It's extremely relevant because people seem to be arguing with about mathematical facts as though they were somehow opinions. You cannot improve the performance of a model without adding new information to that model.
  - sebzim4500 3y ago
    What information is being provided? I'm not suggesting that you would do any kind of check to see if the new training data is correct. Just take the output and remove every line between the first and last. Deterministically removing data is not adding information, unless you are defining information in an extremely unusual way.
    - deleted 3y ago
      [deleted]
    - YeGoblynQueenne 3y ago

flangola7 3y ago

Then how do humans do it? Our knowledge relies heavily on sending the output of humans to the input of humans (verbal communication, books, self-thought, etc). At one point total human knowledge was little more than "some berries are bad" but through this recursive process arrived at the rich garden of information we have today.

version_five 3y ago
Humans have nothing in common with neural networks (other than if you choose to misunderstand the literal definition of those two words), there's no direct analogy to be made.
- flangola7 3y ago
  I didn't say anything about neural networks, I asked how humans do it.
  - tempusalaria 3y ago
    To start with, humans have access to far more data than just what is generated by other humans. We can look at nature, other animals, mathematical laws and many other things. Humans also have far more ability than current model architectures to use memory (particularly in training), flexible context (Model context always directly follows the input distribution in training whereas we can flexibly combine different parts of our input data as we like ) and logic. We can also do live learning in ways current models can’t.
cubefox 3y ago
Humans don't just try to imitate text from some group of humans, like language models. But if we wanted to imitate the text of, say, 18th century novelists, then only those 18th century novels seem to matter. If we also started to imitate our own imitations of 18th century novels (synthetic data) the result would probably get worse, since our imitations are imperfect.
- cguess 3y ago
  To consider your analogy a bit further: Imagine you had access to only... five 18th century novels, that's it, no other knowledge about the period, the people or anything. Then you try to write your own. You can make up your own stories that are a merging of various parts of those five novels, but they'll always only contain what's in those, or some combination. If all the books deal with France then Germans or English will never make an appearance because it's literally impossible to guess that they exist since they're not in the original books (the books here are training data if that's not obvious).
time_to_smile 3y ago
Because humans (and your language is confusing here between individual humans as models and human society as the model) are constantly receiving new information. Even before humans learned "some berries are bad" they initially thought either "all known eaten berries are bad" or "all known eaten berries are good", then somebody at a good/bad berry and updated their believes (i.e. their model of the world). Humans learn from other humans because humans don't individually share the same information and model of the world. A science teach can speed up how you learn science by taking the compressed information and explaining it quickly (essentially what is happening in the post), but out scientific model is expanded when we have experiences that call into question the strength of our current model. However both individually and as a society we are constantly taking in new information (sometimes more sometimes less) and using that to update our model.

kadoban 3y ago

How does this train of thought apply to something like AlphaZero? There, computation/time seems to be spent creating data. Why can't the same thing in concept apply here?

version_five 3y ago
Good question. I'm not an RL expert, but those kind of models are sampling the possible space of game states, based on the rules of the game, to learn a winning policy. Complex games can have effectively infinite different states. What we're talking about here is approximating a distribution from a finite and comparatively small set of images. There's no "game" you can play to get new images, the training data is fixed. The analogy might be say sampling images from a computer animation, which shows the object your interested in in different plausible poses and backgrounds, lighting, angles. There's a question of making sure this generalizes to "real" images, but that can help. But it relies on the animators knowledge of what the object can look like in different poses. That's how information gets added.

bheadmaster 3y ago

> You can add information, by making rules like the class doesn't change if the thing is a different color or size or position, or on a different background. But you can't automatically create new images from a distribution that has been learned and expect them to add information. I am not quite sure I follow. Why would any of those things be true for all models?

sebzim4500 3y ago

Here's a scenario where training on the output of a model could improve its own performance: Suppose you want to optimize the ability for an LLM to do arithmetic. You prompt the LLM to generate a bunch of arithmetic questions, prompting it to show its working. Then you take that output, remove the intermediate steps, and train on the results. I think you would improve some benchmarks by doing this, possibly at the expense of others. I'm sure you could do similar things with other types of model, but I think thissimple example shows the point adaquately. In any case, information theory is not especially relevant here. It puts an upper bound on model performance but we are so incredibly far away from that bound it isn't worth worrying about.

time_to_smile 3y ago
> You prompt the LLM to generate a bunch of arithmetic questions, prompting it to show its working. Then you take that output, remove the intermediate steps, and train on the results. Removing or editing the output of the model is providing new information to the model, what's improving the performance of the model in this scenario is that you are explicitly adding new information and fine tuning it on these new cases. > information theory is not especially relevant here It's extremely relevant because people seem to be arguing with about mathematical facts as though they were somehow opinions. You cannot improve the performance of a model without adding new information to that model.
- sebzim4500 3y ago
  What information is being provided? I'm not suggesting that you would do any kind of check to see if the new training data is correct. Just take the output and remove every line between the first and last. Deterministically removing data is not adding information, unless you are defining information in an extremely unusual way.
  - deleted 3y ago
    [deleted]
  - YeGoblynQueenne 3y ago