4 ms·
I like this approach. It's not that intelligence is in the brain or the model, and it should not be a noun. It is a process, a search process based on explorati
by visarga 1y ago
I like this approach. It's not that intelligence is in the brain or the model, and it should not be a noun. It is a process, a search process based on exploration and learning.
Think of a river - the water carves the banks, the banks channel the water. Which is the real river?
A model is like the river banks, it is not intelligent in itself. Neither is activity by itself. Intelligence comes from their co-constitutive relation.
You can see the structure+flow coupling in many places. The model weights (banks) vs the training set (water). The contextual tokens (banks) vs the mass of probability for the next token (water). Agent environment (banks) vs agent activity (water).
The trick here is that neither is capable of explaining the process, or more fundamental than the other. It is a recursive process rewriting its own rules.
And we know from Godel, Turing and Chaitin that recursive processes exhibit incompleteness, undecidability and incompressibility. There is no way to predict them from outside, they can only be known only through unrolling, you have to be inside, or you have to be it to know it.
- chrisweekly 1y ago+1, Brilliant metaphor, thanks for your thought-provoking comment!
- patcon 1y agoYou are one of the few random HN users who have your own page in my personal knowledgebase, because I consistently see your wonderful comments and i can only hope to be swimming in parallel to your perspectives :) Thanks for your great contributions, especially your dedication to metaphor!
- close04 1y ago> Think of a river - the water carves the banks, the banks channel the water. Which is the real river? Fantastic metaphor. I also can't help but draw a parallel (or even an overlap) with how the process works for humans. Once you put aside the intrinsic "model" (brain) differences (nature vs. nurture), humans develop their intelligence the same way: relentless exploration, and decision making to guide that exploration. We don't just funnel new info into a child's brain, they're allowed to explore threads around that, and take different paths which then guide how to process the next bit of info. If we're looking at building anything like human intelligence, the only advanced general intelligence we know and understand to a degree, exploration will be critical.
- AndrewKemendo 1y agoYou might like my latest work id be curious your feedback https://kemendo.com/GTC.pdf https://kemendo.com/GTC.pdf