6 ms·

> AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution I would push back on this a l

> AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution I would push back on this a little bit. While it has not helped us to understand our own intelligence, it has made me question whether such a thing even exists. Perhaps there are no simple and beautiful natural laws, like those that exists in Physics, that can explain how humans think and make decisions. When CNNs learned to recognize faces through a series of hierarchical abstractions that make intuitive sense it's hard to deny the similarities to what we're doing as humans. Perhaps it's all just emergent properties of some messy evolved substrate. The big lesson from the AI development in the last 10 years from me has been "I guess humans really aren't so special after all" which is similar to what we've been through with Physics. Theories often made the mistake of giving human observers some kind of special importance, which was later discovered to be the cause of theories not generalizing.

lenkite 1y ago
It is possible that we simply haven't yet discovered those natural laws for "emergent behavior" from the "messy substrate".
user_7832 1y ago
> The big lesson from the AI development in the last 10 years from me has been "I guess humans really aren't so special after all" Instead I would take the opposite take. How wonderful is it, that with naturally evolved processes and neural structures, have we been able to create what we have. Van Gogh’s paintings came out of the human brain. The Queens of the Skies - hundreds of tons of metal and composites - flying across continents in the form of a Boeing 747 or an A380 - was designed by the human brain. We went to space, have studied nature (and have conservation programs for organisms we have found to need help), took pictures the pillars of creation that are so incredibly far… all with such a “puny” structure a few cm in diameter? I think that’s freaking amazing.
- OccamsMirror 1y ago
  "I guess humans really aren't so special after all" This is a crazy take to me. As compared to what? The machines that we built? Until we discover comparably intelligent life in the universe I think it's fair to say that we are indeed very special.
  - pineaux 1y ago
    Its like saying: Ah, but these wizards created a magical entity that can also do magic! Wizards must not be so special after all...
  - globnomulous 1y ago
    https://davidhume.org/texts/empl1/dm https://davidhume.org/texts/empl1/dm
    - johnisgood 1y ago
      I am pleasantly surprised that David Hume's writings have been mentioned. I love his works.
      - globnomulous 1y ago
        I have to confess this is the only essay of his I know, though it's an all-time favorite. What other Hume pieces would you recommend?
        johnisgood 1y ago
        What initially drew me to David Hume was a quote from his discussions of miracles in "An Enquiry Concerning Human Understanding" (name of chapter is "Of Miracles"). That said, I began with "A Treatise of Human Nature" around the age of 17, translated to my native language (his works are not an easy read in English, IMO), due to my interest in both philosophy and psychology. If you haven't read them yet, I would certainly recommend them. I would recommend the latter I mentioned even if you are not interested in psychology (but may be interested in epistemology, philosophy of mind, and/or ethics), as he gets into detail about his "impressions" vs "ideas". Additionally, he is famously known for his "problem of induction" which you may already know.
- maaaaattttt 1y ago
  "Brain_s_". I find we (me included) generally overlook/underestimate the distributed nature of human intelligence, included in the AI field. That's why when I first heard of mixture of experts I was thrilled about the idea and the potential. (One could also see similarities in random forest). I believe a path to AGI(tm) would be to reproduce the evolution of human intelligence artificially. Start with small models training bigger and bigger models and let the bigger successfull models (insert RL, genetic algos, etc.) "reproduce" and teach newer models from scratch. Having different model architecture cohabit could maybe even lead to the kind of specializations we see in parts of the brain
csomar 1y ago
> it has made me question whether such a thing even exists I was reading a reddit post the other day where the guy lost his crypto holdings because he input his recovery phrase somewhere. We question the intelligence of LLMs because they might open a website, read something nefarious, and then do it. But here we have real humans doing the exact same thing... > I guess humans really aren't so special after all No they are not. But we are still far from getting there with the current LLMs and I suspect mimicking the human brain won't be the best path forward.
- krige 1y ago
  > But here we have real humans doing the exact same thing... I'd wager that a motivation in designing these systems it so they do not make these mistakes. Otherwise what's the point, really.
  - sayamqazi 1y ago
    I think a system too perfect will not show any creativity. Maybe wild new ideas require taking risks which means a system that can invent new things will end up making bad choices.
  - johnisgood 1y ago
    Our own, and other people's mistakes shape us, and our understanding. What even is perfect, anyway?
    - krige 1y ago
      I'm not interested in navel gazing. I'm interested in getting my taxes done properly.
imadierich 1y ago
[dead]
_glass 1y ago
I think it is important to realize, that we need to understand language on our own terms. The logic of LLMs is not unlike alien technology to us. That being said, the minimalist program of Chomsky lead to nowhere, because just like programming, it found edge case after edge case, reducing it further and further, until there was no program anymore that resembled a real theory. But it is wrong to assume that the big progress in linguistics is in vain, the same reason Prolog, Theorem provers, type theory, category theory is in vain, when we have LLMs that can produce everything in C++. We can use the technology of linguistics to ground our knowledge, and in some dark corner of the LLM it might already have integrated this. I think the original divide between the sciences and the humanities might be deeper and more fundamental than we think. We need linguistic as a discipline of the humanities, and maybe huge swaths of Computer Science is just that.
- js8 1y ago
  I agree with you. I think the fundamental problem is we don't have a good unified theory of fuzzy reasoning. We have a lot of different formal approaches but they all have flaws. Now LLMs made a big breakthrough that they showed we can do decent fuzzy reasoning in practice. But at the cost of nobody understanding the underlying process formally. If we had a good unified (formal) theory of fuzzy reasoning, we could build models that reason better (or at least more predictably). But we won't get a better theory by scaling the existing models, I think Chomsky is right about that. We lack the goal, not the means. If I am asking LLM a question, what answer do I want? A playfully creative one? A strictly logical one? A pleasingly sycophantic one? A harshly critical one? An out of the box devil's advocate one? A beautiful one? A practical one? We have no clue how to express these modes in logical reasoning.
- vrighter 1y ago
  but we don't have llms that can "produce everything in c++". We have LLMs that can get some boilerplate right if you use it in a greenfield project, and will repeatedly mess up your code once it grows enough for you to actually need assistance grokking it.
- suddenlybananas 1y ago
  >That being said, the minimalist program of Chomsky lead to nowhere, because just like programming, it found edge case after edge case, reducing it further and further, until there was no program anymore that resembled a real theory As someone who has worked in linguistics, I don't really see what you're talking about. Minimalism is not full of exceptions (please elaborate on a specific example if you have one). Minimalism was created to make the old theory, Government and Binding, simpler.
mykowebhn 1y ago
> Perhaps there are no simple and beautiful natural laws, like those that exists in Physics, that can explain how humans think and make decisions. Isn't Physics trying to describe the natural world? I'm guessing you are taking two positions here that are causing me confusion with your statement: 1) that our minds can be explained strictly through physical processes, and 2) our minds, including our intelligence, are outside of the domain of Physics. If you take 1) to be true, then it follows that Physics, at least theoretically, should be able to explain intelligence. It may be intractably hard, like it might be intractably hard to have physics decribe and predict the motions of more than two planetary bodies. I guess I'm saying that Physical laws ARE natural laws. I think you might be thinking that natural laws refer solely to all that messy, living stuff.
- jodrellblank 1y ago
  I think their emphasis is on simple and beautiful; not that human intelligence is outside the laws of physics, but that there will never be a “Maxwell’s equations” modelling the workings of human intelligence, it will just be a big pile of hacks and complex interactions of many distinct parts; nothing like the couple of recursive LISP macros people of the 1960s might have hoped to find.
Balgair 1y ago
Neuroscientist here: > Perhaps there are no simple and beautiful natural laws, like those that exists in Physics, that can explain how humans think and make decisions...Perhaps it's all just emergent properties of some messy evolved substrate. Yeah, it is very likely that there are not laws that will do this, it's the substrate. The fruit fly brain (let alone human) has been mapped, and we've figured out that it's not just the synapse count, but the 'weights' that matter too [0]. Mind you, those weights adjust in real time when a living animal is out there. You'll see in literature that there are people with some 'lucky' form of hydranencephaly where their brain is as thin as paper. But they vote, get married, have kids, and for some strange reason seem to work in mailrooms (not a joke). So we know it's something about the connectome that's the 'magic' of a human. My pet theory: We need memristors [2] to better represent things. But that takes redesigning the computer from the metal on up, so is unlikely to occur any time soon with this current AI craze. > The big lesson from the AI development in the last 10 years from me has been "I guess humans really aren't so special after all" which is similar to what we've been through with Physics. Yeah, biologists get there too, just the other way abouts, with animals and humans. Like, dogs make vitamin C internally, and humans have that gene too, it's just dormant, ready for evolution (or genetic engineering) to reactivate. That said, these neuroscience issues with us and the other great apes are somewhat large and strange. I'm not big into that literature, but from what little I know, the exact mechanisms and processes that get you from tool using ourangs to tool using humans, well, those seem to be a bit strange and harder to grasp for us. Again, not in that field though. In the end though, humans are special. We're the only ones on the planet that ever really asked a question. There's a lot to us and we're actually pretty strange in the end. There's many centuries of work to do with biology, we're just at the wading stage of that ocean. [0] https://en.wikipedia.org/wiki/Drosophila_connectome https://en.wikipedia.org/wiki/Drosophila_connectome [1] https://en.wikipedia.org/wiki/Hydranencephaly https://en.wikipedia.org/wiki/Hydranencephaly [2] https://en.wikipedia.org/wiki/Memristor https://en.wikipedia.org/wiki/Memristor
- southernplaces7 1y ago
  >You'll see in literature that there are people with some 'lucky' form of hydranencephaly where their brain is as thin as paper. But they vote, get married, have kids, and for some strange reason seem to work in mailrooms (not a joke). So we know it's something about the connectome that's the 'magic' of a human. These cases seem totally fascinating. Have you any links to examples or more information (i'm also curious about the curious detail of them tending to work in mail rooms)?

lenkite 1y ago

It is possible that we simply haven't yet discovered those natural laws for "emergent behavior" from the "messy substrate".

user_7832 1y ago

> The big lesson from the AI development in the last 10 years from me has been "I guess humans really aren't so special after all" Instead I would take the opposite take. How wonderful is it, that with naturally evolved processes and neural structures, have we been able to create what we have. Van Gogh’s paintings came out of the human brain. The Queens of the Skies - hundreds of tons of metal and composites - flying across continents in the form of a Boeing 747 or an A380 - was designed by the human brain. We went to space, have studied nature (and have conservation programs for organisms we have found to need help), took pictures the pillars of creation that are so incredibly far… all with such a “puny” structure a few cm in diameter? I think that’s freaking amazing.

OccamsMirror 1y ago
"I guess humans really aren't so special after all" This is a crazy take to me. As compared to what? The machines that we built? Until we discover comparably intelligent life in the universe I think it's fair to say that we are indeed very special.
- pineaux 1y ago
  Its like saying: Ah, but these wizards created a magical entity that can also do magic! Wizards must not be so special after all...
- globnomulous 1y ago
  https://davidhume.org/texts/empl1/dm https://davidhume.org/texts/empl1/dm
  - johnisgood 1y ago
    I am pleasantly surprised that David Hume's writings have been mentioned. I love his works.
    - globnomulous 1y ago
      I have to confess this is the only essay of his I know, though it's an all-time favorite. What other Hume pieces would you recommend?
      - johnisgood 1y ago
        What initially drew me to David Hume was a quote from his discussions of miracles in "An Enquiry Concerning Human Understanding" (name of chapter is "Of Miracles"). That said, I began with "A Treatise of Human Nature" around the age of 17, translated to my native language (his works are not an easy read in English, IMO), due to my interest in both philosophy and psychology. If you haven't read them yet, I would certainly recommend them. I would recommend the latter I mentioned even if you are not interested in psychology (but may be interested in epistemology, philosophy of mind, and/or ethics), as he gets into detail about his "impressions" vs "ideas". Additionally, he is famously known for his "problem of induction" which you may already know.
maaaaattttt 1y ago
"Brain_s_". I find we (me included) generally overlook/underestimate the distributed nature of human intelligence, included in the AI field. That's why when I first heard of mixture of experts I was thrilled about the idea and the potential. (One could also see similarities in random forest). I believe a path to AGI(tm) would be to reproduce the evolution of human intelligence artificially. Start with small models training bigger and bigger models and let the bigger successfull models (insert RL, genetic algos, etc.) "reproduce" and teach newer models from scratch. Having different model architecture cohabit could maybe even lead to the kind of specializations we see in parts of the brain

csomar 1y ago

> it has made me question whether such a thing even exists I was reading a reddit post the other day where the guy lost his crypto holdings because he input his recovery phrase somewhere. We question the intelligence of LLMs because they might open a website, read something nefarious, and then do it. But here we have real humans doing the exact same thing... > I guess humans really aren't so special after all No they are not. But we are still far from getting there with the current LLMs and I suspect mimicking the human brain won't be the best path forward.

krige 1y ago
> But here we have real humans doing the exact same thing... I'd wager that a motivation in designing these systems it so they do not make these mistakes. Otherwise what's the point, really.
- sayamqazi 1y ago
  I think a system too perfect will not show any creativity. Maybe wild new ideas require taking risks which means a system that can invent new things will end up making bad choices.
- johnisgood 1y ago
  Our own, and other people's mistakes shape us, and our understanding. What even is perfect, anyway?
  - krige 1y ago
    I'm not interested in navel gazing. I'm interested in getting my taxes done properly.

imadierich 1y ago

[dead]

_glass 1y ago

I think it is important to realize, that we need to understand language on our own terms. The logic of LLMs is not unlike alien technology to us. That being said, the minimalist program of Chomsky lead to nowhere, because just like programming, it found edge case after edge case, reducing it further and further, until there was no program anymore that resembled a real theory. But it is wrong to assume that the big progress in linguistics is in vain, the same reason Prolog, Theorem provers, type theory, category theory is in vain, when we have LLMs that can produce everything in C++. We can use the technology of linguistics to ground our knowledge, and in some dark corner of the LLM it might already have integrated this. I think the original divide between the sciences and the humanities might be deeper and more fundamental than we think. We need linguistic as a discipline of the humanities, and maybe huge swaths of Computer Science is just that.

js8 1y ago
I agree with you. I think the fundamental problem is we don't have a good unified theory of fuzzy reasoning. We have a lot of different formal approaches but they all have flaws. Now LLMs made a big breakthrough that they showed we can do decent fuzzy reasoning in practice. But at the cost of nobody understanding the underlying process formally. If we had a good unified (formal) theory of fuzzy reasoning, we could build models that reason better (or at least more predictably). But we won't get a better theory by scaling the existing models, I think Chomsky is right about that. We lack the goal, not the means. If I am asking LLM a question, what answer do I want? A playfully creative one? A strictly logical one? A pleasingly sycophantic one? A harshly critical one? An out of the box devil's advocate one? A beautiful one? A practical one? We have no clue how to express these modes in logical reasoning.
vrighter 1y ago
but we don't have llms that can "produce everything in c++". We have LLMs that can get some boilerplate right if you use it in a greenfield project, and will repeatedly mess up your code once it grows enough for you to actually need assistance grokking it.
suddenlybananas 1y ago
>That being said, the minimalist program of Chomsky lead to nowhere, because just like programming, it found edge case after edge case, reducing it further and further, until there was no program anymore that resembled a real theory As someone who has worked in linguistics, I don't really see what you're talking about. Minimalism is not full of exceptions (please elaborate on a specific example if you have one). Minimalism was created to make the old theory, Government and Binding, simpler.

mykowebhn 1y ago

> Perhaps there are no simple and beautiful natural laws, like those that exists in Physics, that can explain how humans think and make decisions. Isn't Physics trying to describe the natural world? I'm guessing you are taking two positions here that are causing me confusion with your statement: 1) that our minds can be explained strictly through physical processes, and 2) our minds, including our intelligence, are outside of the domain of Physics. If you take 1) to be true, then it follows that Physics, at least theoretically, should be able to explain intelligence. It may be intractably hard, like it might be intractably hard to have physics decribe and predict the motions of more than two planetary bodies. I guess I'm saying that Physical laws ARE natural laws. I think you might be thinking that natural laws refer solely to all that messy, living stuff.

jodrellblank 1y ago
I think their emphasis is on simple and beautiful; not that human intelligence is outside the laws of physics, but that there will never be a “Maxwell’s equations” modelling the workings of human intelligence, it will just be a big pile of hacks and complex interactions of many distinct parts; nothing like the couple of recursive LISP macros people of the 1960s might have hoped to find.

Balgair 1y ago

Neuroscientist here: > Perhaps there are no simple and beautiful natural laws, like those that exists in Physics, that can explain how humans think and make decisions...Perhaps it's all just emergent properties of some messy evolved substrate. Yeah, it is very likely that there are not laws that will do this, it's the substrate. The fruit fly brain (let alone human) has been mapped, and we've figured out that it's not just the synapse count, but the 'weights' that matter too [0]. Mind you, those weights adjust in real time when a living animal is out there. You'll see in literature that there are people with some 'lucky' form of hydranencephaly where their brain is as thin as paper. But they vote, get married, have kids, and for some strange reason seem to work in mailrooms (not a joke). So we know it's something about the connectome that's the 'magic' of a human. My pet theory: We need memristors [2] to better represent things. But that takes redesigning the computer from the metal on up, so is unlikely to occur any time soon with this current AI craze. > The big lesson from the AI development in the last 10 years from me has been "I guess humans really aren't so special after all" which is similar to what we've been through with Physics. Yeah, biologists get there too, just the other way abouts, with animals and humans. Like, dogs make vitamin C internally, and humans have that gene too, it's just dormant, ready for evolution (or genetic engineering) to reactivate. That said, these neuroscience issues with us and the other great apes are somewhat large and strange. I'm not big into that literature, but from what little I know, the exact mechanisms and processes that get you from tool using ourangs to tool using humans, well, those seem to be a bit strange and harder to grasp for us. Again, not in that field though. In the end though, humans are special. We're the only ones on the planet that ever really asked a question. There's a lot to us and we're actually pretty strange in the end. There's many centuries of work to do with biology, we're just at the wading stage of that ocean. [0] https://en.wikipedia.org/wiki/Drosophila_connectome https://en.wikipedia.org/wiki/Drosophila_connectome [1] https://en.wikipedia.org/wiki/Hydranencephaly https://en.wikipedia.org/wiki/Hydranencephaly [2] https://en.wikipedia.org/wiki/Memristor https://en.wikipedia.org/wiki/Memristor

southernplaces7 1y ago
>You'll see in literature that there are people with some 'lucky' form of hydranencephaly where their brain is as thin as paper. But they vote, get married, have kids, and for some strange reason seem to work in mailrooms (not a joke). So we know it's something about the connectome that's the 'magic' of a human. These cases seem totally fascinating. Have you any links to examples or more information (i'm also curious about the curious detail of them tending to work in mail rooms)?