6 ms·

If you compare "thinking" to "text generators", don't you need to understand what "thinking" actually is, before you can conclude that these are unequal?

by smokel 3y ago

marcus0x62 3y ago
Yeah. Now, imagine the hubris required to think we've invented something we functionally cannot define.
- usernomdeguerre 3y ago
  Isn't scientific advancement inventing things we can't fully define? Each working atop the other to define more?
- astrange 3y ago
  That's pretty easy to do. We don't know how Tylenol works.
  - semi-extrinsic 3y ago
    Yes we do, it's a cyclooxygenase inhibitor. E.g. from the abstract of [1]: "The mode of action of paracetamol has been uncertain, but it is now generally accepted that it inhibits COX-1 and COX-2 through metabolism by the peroxidase function of these isoenzymes." Furthermore, the active ingredient in Tylenol is a very simple molecule from a synthetic perspective, and it was developed while trying to overcome the toxicity of an even simpler molecule called acetanilide. Even though it was developed 150 years ago, there was a systematic understanding of organic chemistry and the interaction with the human body already at that time. [1] https://link.springer.com/article/10.1007/s10787-013-0172-x https://link.springer.com/article/10.1007/s10787-013-0172-x
    - tpush 3y ago
      I mean, there's plenty of other meds whose exact precise functioning we don't know, just what results and side effects it has.
  - marcus0x62 3y ago
    When tylenol was invented, did the inventors have to precisely specify the molecular interactions between the compound and the human body in the same way a programmer has to precisely specify a program? No? Maybe the analogy you are suggesting isn’t the most accurate.
- __loam 3y ago
  Humans invent things they don't fully understand basically always.
deleted 3y ago
[deleted]
ryanklee 3y ago
Generally, people in HN don't let that small detail get in the way of making strong assertions on these matters. These threads are always massive, embarrassing trainwrecks of naive takes on cognition and consciousness. It's like HN just woke up and refuses to acknowledge the relevance of the extensive prior literature on these subjects. Instead, people seem mostly fine with making shit up. Embarrassing.
- sho_hn 3y ago
  > Embarrassing. It's fine. We're social animals, we discuss topics even if we lack perfect expertise, at our present level of understanding. I've certainly had, for example, many opinions on programming that were later superseded by deeper understanding or evolving priorities. It's just important to remember there's probably a bigger and better expert out there, and stay humble and interested, and let yourself be corrected.
  - ryanklee 3y ago
    Yes, true, however, what I find egregious in this case is the comparative lack of standards. Among the other technical and scientific topics often discussed here, there is a basic general expectation of grounded reference to existing research and bodies of knowledge. Deviations from this standard are usually met with corrections. With the topic of cognition and consciousness, this is not so (generally). The standard is to absolutely not reference existing research and to basically pretend it doesn't exist. Instead, the vacuum of ignorance is filled with whatever comes to mind and threads engage in freeform speculation. This is also ironic since these topics are usually about how LLMs don't understand and just fill the vacuum of its ignorance with whatever happens to be next on the statistical chain, regardless of veracity.
    - sho_hn 3y ago
      I think it's because a lot of people presume the topic at hand to be relatively novel and not have a significant body of work behind it. At to some extent this is because they have been told this is the case - even by experts or their reporters. With physics, for example, people have an understanding that there's a large existing body of work, and that physics has been remarkably successful at building on earlier knowledge for decades without having to debunk a lot of earlier results. It's a familiar story. With AI, however, the popular narrative is that earlier approaches to the problem were all laughing stocks that went nowhere. There's not a good sense for how old some the ideas that work now actually are, and how many of the older theoretical and thought frameworks are still perfectly valid, in particular at the interfaces toward other branches of science and engineering. There's some more nuanced and balanced primers on the topic (e.g. the Wooldridge book), but an awful lot more writing on AI Winters and what not. This snowballs into a lack of awareness of the overlap between "AI" and, say, the body of knowledge in statistics. In other words: Sure, I agree - HN collectively is less knowledge on this than on some other topics (myself included!), and it's worth taking stock of that, I suppose.
swatcoder 3y ago
No more than I need to understand every dynamic of walking to know that my goldfish doesn’t do it. For a lot of us, LLM behavior and thinking are so plainly and obviously dissimilar that the endless comparisons look somewhere between naive or manipulative, depending on who’s making them.
- stavros 3y ago
  The difference is that your goldfish doesn't look like it's walking. I'm sure my door doesn't think. I'm not sure the LLM doesn't.
  - antisthenes 3y ago
    > I'm not sure the LLM doesn't. If you turn off the power to the computer hosting the LLM, you can also be sure it doesn't think.
- digging 3y ago
  > For a lot of us, LLM behavior and thinking are so plainly and obviously dissimilar But what's obvious to you is not obvious to everyone. Even though you don't have to mathematically prove a goldfish doesn't walk for me to believe you, that's because we can both agree on a very good definition of walking. If it came down to it, I am sure we could sit down with pen an paper and some textbooks and agree upon a physical definition as robust as any. We're just skipping that step because it's been done before by others. I am positive that we can't agree on a robust definition of thinking, because the definition of thinking is beyond human understanding. We have to keep an open mind. I don't believe it's likely that any LLMs are experiencing anything we'd call thought, but without knowing how thought works, it would be foolish of me to say it's impossible. The problem in AI discussions is not the positions people are taking but the certainty with which they are taking them.
  - YeGoblynQueenne 3y ago
    >> I am positive that we can't agree on a robust definition of thinking, because the definition of thinking is beyond human understanding. In that case, shouldn't the people who say LLMs can think do the hard work of explaining what "thinking" means and why they er think it's done by LLMs? Surely the default position should be that LLMs don't think because they don't belong to the class of entities that we know can think. If that default assumption is wrong, well, then, someone has to do the hard work of rejecting it. But just claiming that we think they think because who knows what thinking is, is just an excuse to not do the work.
RC_ITR 3y ago
No. We know exactly what "text generators" do and we also know that is not what the brain does. That can be true even if you don't fully understand what the brain does. An easy way to prove the two are different is 'What is the equivalent to the endocrine system for an LLM and why doesn't it get tired like a mind?' The logical problems that a lot of people run into are twofold: 1) At a very basic level, we modeled these architectures on brains (and named them after our brains), so at a very very basic metaphorical level, you can say 'these things are acting like our brains act.' This ignores the complexity of the brain. 2) We like to think that inconveniences of our minds (like the need for sleep) are "problems" that we can engineer away, rather than intrinsic components of the process. People won't like hearing this, but there's no formal reason why 'the need to sleep' is not a necessary criteria for 'being conscious,' because as far as we know, the only conscious beings out there also sleep. It's just human nature to assume we can engineer the 'good' parts of a biological system while avoiding the 'bad' while still essentially replicating the system, but that's usually not the case. [0]https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2525690/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2525690/
- nomel 3y ago
  > What is the equivalent to the endocrine system for an LLM Are you suggesting that the brain is the only implementation that can “think”? Or that an endocrine system is required? The difference isn’t the interesting, or necessarily useful, part; the practical similarities of the output are. If we can make a system that appears to "think", I can’t understand how it can be so easily dismissed when we don’t know what’s going on in it or us.
  - RC_ITR 3y ago
    I do think nervous systems (the brain being the most complex example we know of) are the only things that do what we call thinking. What else does (please do not mention any deterministic counting machines like semiconductors - neurons are not deterministic and thought isn’t a deterministic set of calculations). I would argue that the difference in process is extremely important, as evidenced by how easily we are fooled by optical illusions. Just because we think two things appear the same does not mean they are, and understanding why they appear the same but are different relies on a study of the “how” To put your argument differently - “if we can make drawings that appear to be three dimensional to our eyes, why bother understanding the difference between 2-D and 3-D space?”

If you compare "thinking" to "text generators", don't you need to understand what "thinking" actually is, before you can conclude that these are unequal?

by smokel 3y ago

If you compare "thinking" to "text generators", don't you need to understand what "thinking" actually is, before you can conclude that these are unequal?

marcus0x62 3y ago

Yeah. Now, imagine the hubris required to think we've invented something we functionally cannot define.

usernomdeguerre 3y ago
Isn't scientific advancement inventing things we can't fully define? Each working atop the other to define more?
astrange 3y ago
That's pretty easy to do. We don't know how Tylenol works.
- semi-extrinsic 3y ago
  Yes we do, it's a cyclooxygenase inhibitor. E.g. from the abstract of [1]: "The mode of action of paracetamol has been uncertain, but it is now generally accepted that it inhibits COX-1 and COX-2 through metabolism by the peroxidase function of these isoenzymes." Furthermore, the active ingredient in Tylenol is a very simple molecule from a synthetic perspective, and it was developed while trying to overcome the toxicity of an even simpler molecule called acetanilide. Even though it was developed 150 years ago, there was a systematic understanding of organic chemistry and the interaction with the human body already at that time. [1] https://link.springer.com/article/10.1007/s10787-013-0172-x https://link.springer.com/article/10.1007/s10787-013-0172-x
  - tpush 3y ago
    I mean, there's plenty of other meds whose exact precise functioning we don't know, just what results and side effects it has.
- marcus0x62 3y ago
  When tylenol was invented, did the inventors have to precisely specify the molecular interactions between the compound and the human body in the same way a programmer has to precisely specify a program? No? Maybe the analogy you are suggesting isn’t the most accurate.
__loam 3y ago
Humans invent things they don't fully understand basically always.

deleted 3y ago

[deleted]

ryanklee 3y ago

Generally, people in HN don't let that small detail get in the way of making strong assertions on these matters. These threads are always massive, embarrassing trainwrecks of naive takes on cognition and consciousness. It's like HN just woke up and refuses to acknowledge the relevance of the extensive prior literature on these subjects. Instead, people seem mostly fine with making shit up. Embarrassing.

sho_hn 3y ago
> Embarrassing. It's fine. We're social animals, we discuss topics even if we lack perfect expertise, at our present level of understanding. I've certainly had, for example, many opinions on programming that were later superseded by deeper understanding or evolving priorities. It's just important to remember there's probably a bigger and better expert out there, and stay humble and interested, and let yourself be corrected.
- ryanklee 3y ago
  Yes, true, however, what I find egregious in this case is the comparative lack of standards. Among the other technical and scientific topics often discussed here, there is a basic general expectation of grounded reference to existing research and bodies of knowledge. Deviations from this standard are usually met with corrections. With the topic of cognition and consciousness, this is not so (generally). The standard is to absolutely not reference existing research and to basically pretend it doesn't exist. Instead, the vacuum of ignorance is filled with whatever comes to mind and threads engage in freeform speculation. This is also ironic since these topics are usually about how LLMs don't understand and just fill the vacuum of its ignorance with whatever happens to be next on the statistical chain, regardless of veracity.
  - sho_hn 3y ago
    I think it's because a lot of people presume the topic at hand to be relatively novel and not have a significant body of work behind it. At to some extent this is because they have been told this is the case - even by experts or their reporters. With physics, for example, people have an understanding that there's a large existing body of work, and that physics has been remarkably successful at building on earlier knowledge for decades without having to debunk a lot of earlier results. It's a familiar story. With AI, however, the popular narrative is that earlier approaches to the problem were all laughing stocks that went nowhere. There's not a good sense for how old some the ideas that work now actually are, and how many of the older theoretical and thought frameworks are still perfectly valid, in particular at the interfaces toward other branches of science and engineering. There's some more nuanced and balanced primers on the topic (e.g. the Wooldridge book), but an awful lot more writing on AI Winters and what not. This snowballs into a lack of awareness of the overlap between "AI" and, say, the body of knowledge in statistics. In other words: Sure, I agree - HN collectively is less knowledge on this than on some other topics (myself included!), and it's worth taking stock of that, I suppose.

swatcoder 3y ago

No more than I need to understand every dynamic of walking to know that my goldfish doesn’t do it. For a lot of us, LLM behavior and thinking are so plainly and obviously dissimilar that the endless comparisons look somewhere between naive or manipulative, depending on who’s making them.

stavros 3y ago
The difference is that your goldfish doesn't look like it's walking. I'm sure my door doesn't think. I'm not sure the LLM doesn't.
- antisthenes 3y ago
  > I'm not sure the LLM doesn't. If you turn off the power to the computer hosting the LLM, you can also be sure it doesn't think.
digging 3y ago
> For a lot of us, LLM behavior and thinking are so plainly and obviously dissimilar But what's obvious to you is not obvious to everyone. Even though you don't have to mathematically prove a goldfish doesn't walk for me to believe you, that's because we can both agree on a very good definition of walking. If it came down to it, I am sure we could sit down with pen an paper and some textbooks and agree upon a physical definition as robust as any. We're just skipping that step because it's been done before by others. I am positive that we can't agree on a robust definition of thinking, because the definition of thinking is beyond human understanding. We have to keep an open mind. I don't believe it's likely that any LLMs are experiencing anything we'd call thought, but without knowing how thought works, it would be foolish of me to say it's impossible. The problem in AI discussions is not the positions people are taking but the certainty with which they are taking them.
- YeGoblynQueenne 3y ago
  >> I am positive that we can't agree on a robust definition of thinking, because the definition of thinking is beyond human understanding. In that case, shouldn't the people who say LLMs can think do the hard work of explaining what "thinking" means and why they er think it's done by LLMs? Surely the default position should be that LLMs don't think because they don't belong to the class of entities that we know can think. If that default assumption is wrong, well, then, someone has to do the hard work of rejecting it. But just claiming that we think they think because who knows what thinking is, is just an excuse to not do the work.

RC_ITR 3y ago

No. We know exactly what "text generators" do and we also know that is not what the brain does. That can be true even if you don't fully understand what the brain does. An easy way to prove the two are different is 'What is the equivalent to the endocrine system for an LLM and why doesn't it get tired like a mind?' The logical problems that a lot of people run into are twofold: 1) At a very basic level, we modeled these architectures on brains (and named them after our brains), so at a very very basic metaphorical level, you can say 'these things are acting like our brains act.' This ignores the complexity of the brain. 2) We like to think that inconveniences of our minds (like the need for sleep) are "problems" that we can engineer away, rather than intrinsic components of the process. People won't like hearing this, but there's no formal reason why 'the need to sleep' is not a necessary criteria for 'being conscious,' because as far as we know, the only conscious beings out there also sleep. It's just human nature to assume we can engineer the 'good' parts of a biological system while avoiding the 'bad' while still essentially replicating the system, but that's usually not the case. [0]https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2525690/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2525690/

nomel 3y ago
> What is the equivalent to the endocrine system for an LLM Are you suggesting that the brain is the only implementation that can “think”? Or that an endocrine system is required? The difference isn’t the interesting, or necessarily useful, part; the practical similarities of the output are. If we can make a system that appears to "think", I can’t understand how it can be so easily dismissed when we don’t know what’s going on in it or us.
- RC_ITR 3y ago
  I do think nervous systems (the brain being the most complex example we know of) are the only things that do what we call thinking. What else does (please do not mention any deterministic counting machines like semiconductors - neurons are not deterministic and thought isn’t a deterministic set of calculations). I would argue that the difference in process is extremely important, as evidenced by how easily we are fooled by optical illusions. Just because we think two things appear the same does not mean they are, and understanding why they appear the same but are different relies on a study of the “how” To put your argument differently - “if we can make drawings that appear to be three dimensional to our eyes, why bother understanding the difference between 2-D and 3-D space?”