4 ms·

Like law of conservation energy, matter and momentum, I guess there must be law of conservation of knowledge in context of LLMs?

by wg0 3y ago

TaylorAlexander 3y ago
Oof I really don’t think there must be such a law. The basic physics of the universe and the mathematical structure of a human engineered LLM are just very different things.
imranq 3y ago
There must be. Since there is a finite amount of real world reward signal captured by these models, there can only be a finite amount of grounded knowledge that can come out. Usually without reasoning, the output knowledge "yield" will be far lower than input. The interesting piece is if we can get LLMs to reason well, that finite reward capture can take LLMs much farther than possibly humans could do with the same signal.
davidkunz 3y ago
The "compressed" knowledge is fixed. But reasoning can create new knowledge, that's what humans do all the time.
FeepingCreature 3y ago
You can't do better than perfect. But if the default is sufficiently bad, you can seem to violate conservation by putting more effort in. Similarly to when a machine has an efficiency of 1%, optimizing it to 10% does not violate thermodynamics. There is a limit to what you can learn from limited knowledge. Every bit of information you learn can only divide the space of possible theories by half. However, current LLMs are ludicrously far from that limit.
reportgunner 3y ago
My Pareto law senses are tingling.
tehsauce 3y ago
Nope! That surely doesn't apply to humans, why would it necessarily apply to learning machines?
- JoshuaDavid 3y ago
  The human equivalent would be the process of armchair philosophizing. Humans get around the limit by being able to go and look at the world to obtain more knowledge -- if you didn't let humans look at the world, they too would have a "knowledge limit". And if you _do_ let LLMs look at the world, and train on that, they also wouldn't have that limitation.

Like law of conservation energy, matter and momentum, I guess there must be law of conservation of knowledge in context of LLMs?

by wg0 3y ago

Like law of conservation energy, matter and momentum, I guess there must be law of conservation of knowledge in context of LLMs?

TaylorAlexander 3y ago

Oof I really don’t think there must be such a law. The basic physics of the universe and the mathematical structure of a human engineered LLM are just very different things.

imranq 3y ago

There must be. Since there is a finite amount of real world reward signal captured by these models, there can only be a finite amount of grounded knowledge that can come out. Usually without reasoning, the output knowledge "yield" will be far lower than input. The interesting piece is if we can get LLMs to reason well, that finite reward capture can take LLMs much farther than possibly humans could do with the same signal.

davidkunz 3y ago

The "compressed" knowledge is fixed. But reasoning can create new knowledge, that's what humans do all the time.

FeepingCreature 3y ago

You can't do better than perfect. But if the default is sufficiently bad, you can seem to violate conservation by putting more effort in. Similarly to when a machine has an efficiency of 1%, optimizing it to 10% does not violate thermodynamics. There is a limit to what you can learn from limited knowledge. Every bit of information you learn can only divide the space of possible theories by half. However, current LLMs are ludicrously far from that limit.

reportgunner 3y ago

My Pareto law senses are tingling.

tehsauce 3y ago

Nope! That surely doesn't apply to humans, why would it necessarily apply to learning machines?

JoshuaDavid 3y ago
The human equivalent would be the process of armchair philosophizing. Humans get around the limit by being able to go and look at the world to obtain more knowledge -- if you didn't let humans look at the world, they too would have a "knowledge limit". And if you _do_ let LLMs look at the world, and train on that, they also wouldn't have that limitation.