3 ms·
> The sum total of the human-generated knowledge was derived in a similar manner, with each generation learning from the one before and expanding the pool of kn
by zkry 2y ago
> The sum total of the human-generated knowledge was derived in a similar manner, with each generation learning from the one before and expanding the pool of knowledge incrementally.
Is human knowledge really derived in a similar manner though? That reduction of biological processes to compression algorithms seems like a huge oversimplification.
It's almost like saying that all of of human knowledge derives from Einstein's Field Equations, the Standard Model Lagrangian, and the Second Law of Thermodynamics (what else could human knowledge really derive from?) and all we have to do to create artificial intelligence is just to model these forces to a high enough fidelity and with enough computation.
- ethbr1 2y agoHuman knowledge also tends to be tied to an objective, mostly constant reality.
- jiggawatts 2y agoThe AIs could also learn form and interact with reality, same as humans.
- dartos 2y agoNot really. The models we use nowadays operate on discrete tokens. To overly reduce the process of human learning, we take a constant stream of realtime information. It never ends and it’s never discrete. Nor do we learn in an isolated “learn” stage in which we’re not interacting with our environment. If you try taking reality and breaking into discrete (ordered in the case of LLMs) parts, you lose information.
- mannykannot 2y agoIt's not just any compression algorithm, though, it's a specific sort of algorithm that does not have the purpose of compression, even if compression is necessary for achieving its purpose. It could not be replaced by most other compression algorithms. Having said that, I think this picture is missing something: when we teach each new generation what we know, part of that process involves recapitulating the steps by which we got to where we are. It is a highly selective (compressed?) history, however, focusing on the things that made a difference and putting aside most of the false starts, dead ends and mistaken notions (except when the topic is history, of course, and often even then.) I do not know if this view has any significance for AI.