4 ms·

Some good intuition around what is common sense for most people, that a loop with some matrix multiplication never becomes intelligent. What's missed (at least

Some good intuition around what is common sense for most people, that a loop with some matrix multiplication never becomes intelligent. What's missed (at least in my skim) is that the "intelligent" part of current AI is in the training, not the inference, which as pointed out is fixed, and literally just some multiplication. Inference doesn't think about stuff but training does (for some definition), and doesn't converge necessarily. I still think it's ridiculous to equate neural networks with intelligence, but a stronger argument has to deal with training as well.

fleischhauf 3y ago
I'm not sure humans work differently. You might not learn something if you are coming to conclusions, only if the conclusions turn out to be correct or wrong you'll learn. the process of learning might then be different.
fnordpiglet 3y ago
I think people get confused that LLM is the beginning and end of AI. It’s just a piece. I think AI is an ensemble of all the classic AI techniques and IR and other information, computational, optimization, agent, etc systems. LLMs are remarkable in their abductive reasoning abilities, which is something we’ve generally failed to produce so far. But they’re not complete, and the criticism that they’re worthless because they’re incomplete misses the point. They don’t need to play chess, because we already have much better chess playing systems. But they fill a gap that all our other techniques have been unable to fill, and that is where the magic lives. It’s unnecessary for them to do everything, as those things already have good solutions.
lostmsu 3y ago
This is pretty much 100% false. There are known trivial Turing machines (which can be implemented in matrix multiplications) that are intelligent. They just don't think fast enough to be of any use.
- nmilo 3y ago
  Example?
  - klyrs 3y ago
    Without offering anything to anybody trying to characterize "intelligence," I do like silly languages. FRACTRAN is integer×rational multiplication in a double for-loop. Not an exact match, but if you work in a factorized representation, I think you can massage the integer×rational into a matrix multiplication.
  - version_five 3y ago
    I'd guess he's misunderstanding some of the finite automata?
  - lostmsu 3y ago
    I can not find the better variant that was more theoretical with exhaustiveness theorem, but on the practical side this applies to more or less any symbolic system that performs tree search among statements. E.g. https://en.m.wikipedia.org/wiki/Logic_Theorist https://en.m.wikipedia.org/wiki/Logic_Theorist
QuadmasterXLII 3y ago
If reality is actually the schroedinger equation, then you are just matrix multiplication in a for loop
stevenhuang 3y ago
Ah it's you again! It's quite funny seeing the dearth of knowledge you routinely insist on displaying whenever this topic comes up. Consider doing some reading on the information theoretic view of cognition and neurobiology. You'll find leading theories for how our brains work have many similarities to NNs. https://en.m.wikipedia.org/wiki/Predictive_coding https://en.m.wikipedia.org/wiki/Predictive_coding Another thing, you don't seem to understand that all physical processes can be modeled completely through matmuls. You should also read up on the definition of intelligence because you seem confused on what it means. Pretty much every assertion you make here is wrong and you don't even know it. You assert "it's common sense" something that does matrix multiplication in a loop can't be intelligent, yet get this, it's literally what your brain is doing. Please have some epistemic humility and consider refraining from forming strong opinions on topics you don't have a clue on.
- throwaway465465 3y ago
  [flagged]
  - hnfong 3y ago
    > major in philosophy Yep, checks out.
  - stevenhuang 3y ago
    Oh I've read and participated in so many variations of this topic. Nothing you said is new to me nor do I necessarily disagree with. It was a single statement to point out that similarities exist to someone who believed there were none. Unfortunately your giddiness to point out this false equivalence is misplaced as I was careful to make no such assertion. Ah and there is indeed not a universal agreed upon definition of intelligence, but the parent looks to be operating on a set of definitions that preclude them all (hint: general reasoning ability). The nuance you miss however is we do in fact have a pretty good idea what human intelligence looks like, and parent is excluding NNs of even that (general reasoning and problem solving, again). > Brains are embodied organs in an overall organism which lives in a dynamic changing world where ecology, cybernetics, control theory, complexity and dynamical systems theory, semantics and semiotics all will play a part in understanding it. Not just neural networks. All true, but guess what. There may be different roads to intelligence. Random walk through evolution and physical processes is one such pathway. Possibly an alternative pathway in a digital medium may reduce all these physical analogues to "mere" NN expression and matmuls. https://en.m.wikipedia.org/wiki/Universal_approximation_theorem https://en.m.wikipedia.org/wiki/Universal_approximation_theo... I'm interested in updating any of my priors if you have resources to point to. Also since you're a philosophy major I would hope you agree with me when I point out how undeveloped OP's understanding is about philosophical matters, when he says stuff like > Inference doesn't think about stuff but training does Like there's so much confusion there it's hard to unpack.

fleischhauf 3y ago

I'm not sure humans work differently. You might not learn something if you are coming to conclusions, only if the conclusions turn out to be correct or wrong you'll learn. the process of learning might then be different.

fnordpiglet 3y ago

I think people get confused that LLM is the beginning and end of AI. It’s just a piece. I think AI is an ensemble of all the classic AI techniques and IR and other information, computational, optimization, agent, etc systems. LLMs are remarkable in their abductive reasoning abilities, which is something we’ve generally failed to produce so far. But they’re not complete, and the criticism that they’re worthless because they’re incomplete misses the point. They don’t need to play chess, because we already have much better chess playing systems. But they fill a gap that all our other techniques have been unable to fill, and that is where the magic lives. It’s unnecessary for them to do everything, as those things already have good solutions.

lostmsu 3y ago

This is pretty much 100% false. There are known trivial Turing machines (which can be implemented in matrix multiplications) that are intelligent. They just don't think fast enough to be of any use.

nmilo 3y ago
Example?
- klyrs 3y ago
  Without offering anything to anybody trying to characterize "intelligence," I do like silly languages. FRACTRAN is integer×rational multiplication in a double for-loop. Not an exact match, but if you work in a factorized representation, I think you can massage the integer×rational into a matrix multiplication.
- version_five 3y ago
  I'd guess he's misunderstanding some of the finite automata?
- lostmsu 3y ago
  I can not find the better variant that was more theoretical with exhaustiveness theorem, but on the practical side this applies to more or less any symbolic system that performs tree search among statements. E.g. https://en.m.wikipedia.org/wiki/Logic_Theorist https://en.m.wikipedia.org/wiki/Logic_Theorist

QuadmasterXLII 3y ago

If reality is actually the schroedinger equation, then you are just matrix multiplication in a for loop

stevenhuang 3y ago

Ah it's you again! It's quite funny seeing the dearth of knowledge you routinely insist on displaying whenever this topic comes up. Consider doing some reading on the information theoretic view of cognition and neurobiology. You'll find leading theories for how our brains work have many similarities to NNs. https://en.m.wikipedia.org/wiki/Predictive_coding https://en.m.wikipedia.org/wiki/Predictive_coding Another thing, you don't seem to understand that all physical processes can be modeled completely through matmuls. You should also read up on the definition of intelligence because you seem confused on what it means. Pretty much every assertion you make here is wrong and you don't even know it. You assert "it's common sense" something that does matrix multiplication in a loop can't be intelligent, yet get this, it's literally what your brain is doing. Please have some epistemic humility and consider refraining from forming strong opinions on topics you don't have a clue on.

throwaway465465 3y ago
[flagged]
- hnfong 3y ago
  > major in philosophy Yep, checks out.
- stevenhuang 3y ago
  Oh I've read and participated in so many variations of this topic. Nothing you said is new to me nor do I necessarily disagree with. It was a single statement to point out that similarities exist to someone who believed there were none. Unfortunately your giddiness to point out this false equivalence is misplaced as I was careful to make no such assertion. Ah and there is indeed not a universal agreed upon definition of intelligence, but the parent looks to be operating on a set of definitions that preclude them all (hint: general reasoning ability). The nuance you miss however is we do in fact have a pretty good idea what human intelligence looks like, and parent is excluding NNs of even that (general reasoning and problem solving, again). > Brains are embodied organs in an overall organism which lives in a dynamic changing world where ecology, cybernetics, control theory, complexity and dynamical systems theory, semantics and semiotics all will play a part in understanding it. Not just neural networks. All true, but guess what. There may be different roads to intelligence. Random walk through evolution and physical processes is one such pathway. Possibly an alternative pathway in a digital medium may reduce all these physical analogues to "mere" NN expression and matmuls. https://en.m.wikipedia.org/wiki/Universal_approximation_theorem https://en.m.wikipedia.org/wiki/Universal_approximation_theo... I'm interested in updating any of my priors if you have resources to point to. Also since you're a philosophy major I would hope you agree with me when I point out how undeveloped OP's understanding is about philosophical matters, when he says stuff like > Inference doesn't think about stuff but training does Like there's so much confusion there it's hard to unpack.