5 ms·

Only in name, given by people in machine learning. Really it’s more accurate to say they’re both networks, except one is a million times more complicated in its

Only in name, given by people in machine learning. Really it’s more accurate to say they’re both networks, except one is a million times more complicated in its design.

ben_w 2y ago
These days it's looking like closer to a factor a thousand than a million. Appearances can be deceptive and intelligence in brains may be more complex than they currently looks, but appearances are also the only thing most of us can judge it by at this point — while there's more stuff going on in living cells, nothing I've seen says the stuff needed to keep the cells themselves alive is directly contributing to the intelligence of the wet network in my skull. It's quite surprising that such a simple mind as an LLM is so very capable. What is all the rest of our human brain doing, when the LLMs demonstrate that a mere cubic centimetre of brain tissue could be reorganised to speak 50 languages fluently, let alone all the other things?
- mrbungie 2y ago
  This post is full of speculation. For starters, human minds do more than just language. There are some arguments (Moravec's paradox) that posit that language is easy, while keeping the rest of the body working is the real hard problem. I've yet to see an humanoid robot controlled by an LLM that can tackle difficult physical problems like riding a bike or skiing in for the first time real-time, just as humans, that need to coordinate at least 5 senses (some say up to 20 but I digress) plus emotions and a train of thought to do so. Plus, sure, LLMs are simpler (that only works if you don't count the full complexity of its substrate: a lot of GPUs, interconnects, Data Centers, etc), yet they consume an insane amount of energy and matter when compared to humans. It is surprising that LLMs are so capable, but for 2000kcal a day humans are still more impressive to me. People see a model dominating language and start thinking the only thing humans do is high-level language and reasoning. That may be why the brilliant minds of this decade talk about replacing white-collars, but no plate-cleaning bot in sight. Metacommentary: a lot of LLM-human equality apologists seem to have little knowledge of the AI field and its history, as these themes are not new at all.
  - ben_w 2y ago
    > For starters, human minds do more than just language. These days, so do the AI (even though these ones are still called LLMs, which I think is now a bad name even though I continue to use the term without always being mindful of this). > I've yet to see an humanoid robot controlled by an LLM that can tackle difficult physical problems like riding a bike or skiing in for the first time real-time, just as humans, that need to coordinate at least 5 senses (some say up to 20 but I digress) plus emotions and a train of thought to do so. 1) I'd agree that AI are slow learners (as measured by number of examples rather than wall clock). For me, this is more significant than the difference in wattage. 2) 13 years ago, before the Transformer model: https://www.youtube.com/watch?v=mT3vfSQePcs https://www.youtube.com/watch?v=mT3vfSQePcs 3) We don't use 5 senses to ride a bike; I can believe touch, vision, proprioception, and balance are all involved, but that's only 4. Why would hearing, smell, or taste be involved? For emotion, my weakly-held belief is that this is what motivates us and tells us what the concept of "good" even is — i.e. it doesn't give us reasoning beyond being a motivation to learn to reason. > It is surprising that LLMs are so capable, but for 2000kcal a day humans are still more impressive to me. Our biological energy efficiency sure is nice, though progress with silicon is continuing even if the rate of improvement is decreasing: https://en.wikipedia.org/wiki/Koomey%27s_law https://en.wikipedia.org/wiki/Koomey%27s_law That said, I don't consider the substrate efficiency to be an indication of if the thing running on it is or isn't "reasoning" — if the same silicon was running a quantum chemistry simulation of the human brain it would be an even bigger power hog and yet I would say it "must be" reasoning, and conversely we use phrases such "turn your brain off and enjoy" to describe categories of film where the plot holes become pot holes if you stop to think about them (the novel I'm trying to write is due to me being nerd-sniped in this way by everything wrong with Independence Day). > That may be why the brilliant minds of this decade talk about replacing white-collars, but no plate-cleaning bot in sight. I mean, I've got a dishwasher already… ;) But more seriously, even though I'm getting cynical about press releases that turn out to be smoke and mirrors, (humanoid) robots doing housework is "in sight" in at least the same kind of way as self driving cars (which has been a decade of "next year honest" mirages, so I'm not giving a timeline): https://www.youtube.com/watch?v=Sq1QZB5baNw https://www.youtube.com/watch?v=Sq1QZB5baNw > Metacommentary: a lot of LLM-human equality apologists seem to have little knowledge of the AI field and its history, as these themes are not new at all. I've been feeling the same, but on both sides, pro and con. The Turing paper laid out all the same talking points I've been seeing over the last few years, and those talking points weren't new when he wrote them down.
ninetyninenine 2y ago
Right, and again. You’re stating obvious information that usually doesn’t need to be stated. The fundamental mathematical concept of a neuron is what both neural networks operate on in the end.
- azinman2 2y ago
  What’s the mathematical concept? A connected graph with weights and activations functions? You’re making gigantic simplifications of our brain, most of which isn’t even understood let alone “obeys a fundamental mathematical concept”.

Only in name, given by people in machine learning. Really it’s more accurate to say they’re both networks, except one is a million times more complicated in its design.

ben_w 2y ago

These days it's looking like closer to a factor a thousand than a million. Appearances can be deceptive and intelligence in brains may be more complex than they currently looks, but appearances are also the only thing most of us can judge it by at this point — while there's more stuff going on in living cells, nothing I've seen says the stuff needed to keep the cells themselves alive is directly contributing to the intelligence of the wet network in my skull. It's quite surprising that such a simple mind as an LLM is so very capable. What is all the rest of our human brain doing, when the LLMs demonstrate that a mere cubic centimetre of brain tissue could be reorganised to speak 50 languages fluently, let alone all the other things?

mrbungie 2y ago
This post is full of speculation. For starters, human minds do more than just language. There are some arguments (Moravec's paradox) that posit that language is easy, while keeping the rest of the body working is the real hard problem. I've yet to see an humanoid robot controlled by an LLM that can tackle difficult physical problems like riding a bike or skiing in for the first time real-time, just as humans, that need to coordinate at least 5 senses (some say up to 20 but I digress) plus emotions and a train of thought to do so. Plus, sure, LLMs are simpler (that only works if you don't count the full complexity of its substrate: a lot of GPUs, interconnects, Data Centers, etc), yet they consume an insane amount of energy and matter when compared to humans. It is surprising that LLMs are so capable, but for 2000kcal a day humans are still more impressive to me. People see a model dominating language and start thinking the only thing humans do is high-level language and reasoning. That may be why the brilliant minds of this decade talk about replacing white-collars, but no plate-cleaning bot in sight. Metacommentary: a lot of LLM-human equality apologists seem to have little knowledge of the AI field and its history, as these themes are not new at all.
- ben_w 2y ago
  > For starters, human minds do more than just language. These days, so do the AI (even though these ones are still called LLMs, which I think is now a bad name even though I continue to use the term without always being mindful of this). > I've yet to see an humanoid robot controlled by an LLM that can tackle difficult physical problems like riding a bike or skiing in for the first time real-time, just as humans, that need to coordinate at least 5 senses (some say up to 20 but I digress) plus emotions and a train of thought to do so. 1) I'd agree that AI are slow learners (as measured by number of examples rather than wall clock). For me, this is more significant than the difference in wattage. 2) 13 years ago, before the Transformer model: https://www.youtube.com/watch?v=mT3vfSQePcs https://www.youtube.com/watch?v=mT3vfSQePcs 3) We don't use 5 senses to ride a bike; I can believe touch, vision, proprioception, and balance are all involved, but that's only 4. Why would hearing, smell, or taste be involved? For emotion, my weakly-held belief is that this is what motivates us and tells us what the concept of "good" even is — i.e. it doesn't give us reasoning beyond being a motivation to learn to reason. > It is surprising that LLMs are so capable, but for 2000kcal a day humans are still more impressive to me. Our biological energy efficiency sure is nice, though progress with silicon is continuing even if the rate of improvement is decreasing: https://en.wikipedia.org/wiki/Koomey%27s_law https://en.wikipedia.org/wiki/Koomey%27s_law That said, I don't consider the substrate efficiency to be an indication of if the thing running on it is or isn't "reasoning" — if the same silicon was running a quantum chemistry simulation of the human brain it would be an even bigger power hog and yet I would say it "must be" reasoning, and conversely we use phrases such "turn your brain off and enjoy" to describe categories of film where the plot holes become pot holes if you stop to think about them (the novel I'm trying to write is due to me being nerd-sniped in this way by everything wrong with Independence Day). > That may be why the brilliant minds of this decade talk about replacing white-collars, but no plate-cleaning bot in sight. I mean, I've got a dishwasher already… ;) But more seriously, even though I'm getting cynical about press releases that turn out to be smoke and mirrors, (humanoid) robots doing housework is "in sight" in at least the same kind of way as self driving cars (which has been a decade of "next year honest" mirages, so I'm not giving a timeline): https://www.youtube.com/watch?v=Sq1QZB5baNw https://www.youtube.com/watch?v=Sq1QZB5baNw > Metacommentary: a lot of LLM-human equality apologists seem to have little knowledge of the AI field and its history, as these themes are not new at all. I've been feeling the same, but on both sides, pro and con. The Turing paper laid out all the same talking points I've been seeing over the last few years, and those talking points weren't new when he wrote them down.

ninetyninenine 2y ago

Right, and again. You’re stating obvious information that usually doesn’t need to be stated. The fundamental mathematical concept of a neuron is what both neural networks operate on in the end.

azinman2 2y ago
What’s the mathematical concept? A connected graph with weights and activations functions? You’re making gigantic simplifications of our brain, most of which isn’t even understood let alone “obeys a fundamental mathematical concept”.