4 ms·

If the output of prompts can be improved with "think step by step", "tree of thoughts", etc., one can produce high quality training data with the LLM itself whi

If the output of prompts can be improved with "think step by step", "tree of thoughts", etc., one can produce high quality training data with the LLM itself which the next iteration of that model can use. Rinse and repeat.

wg0 3y ago
Like law of conservation energy, matter and momentum, I guess there must be law of conservation of knowledge in context of LLMs?
- TaylorAlexander 3y ago
  Oof I really don’t think there must be such a law. The basic physics of the universe and the mathematical structure of a human engineered LLM are just very different things.
- imranq 3y ago
  There must be. Since there is a finite amount of real world reward signal captured by these models, there can only be a finite amount of grounded knowledge that can come out. Usually without reasoning, the output knowledge "yield" will be far lower than input. The interesting piece is if we can get LLMs to reason well, that finite reward capture can take LLMs much farther than possibly humans could do with the same signal.
- davidkunz 3y ago
  The "compressed" knowledge is fixed. But reasoning can create new knowledge, that's what humans do all the time.
- FeepingCreature 3y ago
  You can't do better than perfect. But if the default is sufficiently bad, you can seem to violate conservation by putting more effort in. Similarly to when a machine has an efficiency of 1%, optimizing it to 10% does not violate thermodynamics. There is a limit to what you can learn from limited knowledge. Every bit of information you learn can only divide the space of possible theories by half. However, current LLMs are ludicrously far from that limit.
- reportgunner 3y ago
  My Pareto law senses are tingling.
- tehsauce 3y ago
  Nope! That surely doesn't apply to humans, why would it necessarily apply to learning machines?
  - JoshuaDavid 3y ago
    The human equivalent would be the process of armchair philosophizing. Humans get around the limit by being able to go and look at the world to obtain more knowledge -- if you didn't let humans look at the world, they too would have a "knowledge limit". And if you _do_ let LLMs look at the world, and train on that, they also wouldn't have that limitation.
riku_iki 3y ago
> one can produce high quality training data with the LLM itself which the next iteration of that model can use. Rinse and repeat. it may be higher quality, but far from perfection. After many rinses and repeats dataset will accumulate significant amount of errors.
- Tostino 3y ago
  That depends. You can implement an error checking algorithm with the LLM itself to check for drift in meaning and loss of original context from version to version as you improve your training dataset, as well as measure performance as you train new versions of your LLM on that dataset.

wg0 3y ago

Like law of conservation energy, matter and momentum, I guess there must be law of conservation of knowledge in context of LLMs?

TaylorAlexander 3y ago
Oof I really don’t think there must be such a law. The basic physics of the universe and the mathematical structure of a human engineered LLM are just very different things.
imranq 3y ago
There must be. Since there is a finite amount of real world reward signal captured by these models, there can only be a finite amount of grounded knowledge that can come out. Usually without reasoning, the output knowledge "yield" will be far lower than input. The interesting piece is if we can get LLMs to reason well, that finite reward capture can take LLMs much farther than possibly humans could do with the same signal.
davidkunz 3y ago
The "compressed" knowledge is fixed. But reasoning can create new knowledge, that's what humans do all the time.
FeepingCreature 3y ago
You can't do better than perfect. But if the default is sufficiently bad, you can seem to violate conservation by putting more effort in. Similarly to when a machine has an efficiency of 1%, optimizing it to 10% does not violate thermodynamics. There is a limit to what you can learn from limited knowledge. Every bit of information you learn can only divide the space of possible theories by half. However, current LLMs are ludicrously far from that limit.
reportgunner 3y ago
My Pareto law senses are tingling.
tehsauce 3y ago
Nope! That surely doesn't apply to humans, why would it necessarily apply to learning machines?
- JoshuaDavid 3y ago
  The human equivalent would be the process of armchair philosophizing. Humans get around the limit by being able to go and look at the world to obtain more knowledge -- if you didn't let humans look at the world, they too would have a "knowledge limit". And if you _do_ let LLMs look at the world, and train on that, they also wouldn't have that limitation.

riku_iki 3y ago

> one can produce high quality training data with the LLM itself which the next iteration of that model can use. Rinse and repeat. it may be higher quality, but far from perfection. After many rinses and repeats dataset will accumulate significant amount of errors.

Tostino 3y ago
That depends. You can implement an error checking algorithm with the LLM itself to check for drift in meaning and loss of original context from version to version as you improve your training dataset, as well as measure performance as you train new versions of your LLM on that dataset.