4 ms·
If we are to get to AGI why do we need to train on all data? That's silly, and all we get is compression and probabliatic retrieval. Intelligence by definition
by miraculixx 2y ago
If we are to get to AGI why do we need to train on all data? That's silly, and all we get is compression and probabliatic retrieval.
Intelligence by definition is not compression, but ability to think and act according to new data, based on experience.
Trully AGI models will work on the this principle, not on best compression of as much data as possible.
We need a new approach.
- eigenvalue 2y agoActually, compression is an incredibly good way to think about intelligence. If you understand something really well then you can compress it a lot. If you can compress most of human knowledge effectively without much reconstruction error while shrinking it down by 99.5%, then you must have in the process arrived at a coherent and essentially correct world model, which is the basis of effective cognition.
- chpatrick 2y ago“If you can't explain it to a six year old, you don't understand it yourself.” -> "If you can compress knowledge, you understand it."
- AnotherGoodName 2y agoFwiw there's highly cited papers that literally map AGI to compression. As in they map to the same thing and people write papers on this fact that are widely respected. Basically a prediction engine can be used to make a compression tool and an AI equally. The tldr; if given inputs and a system that can accurately predict the next sequence you can either compress that data using that prediction (arithmetic coding) or you can take actions based on that prediction to achieve an end goal mapping predictions of new inputs to possible outcomes and then taking the path to a goal (AGI). They boil down to one and the same. So it's weird to have someone state they are not the same when it's widely accepted they absolutely are.