5 ms·

> Does a calculator truly understand math when it spits out a correct answer? Of course not. Unless you're using a definition of "understand" that implies cons

> Does a calculator truly understand math when it spits out a correct answer? Of course not. Unless you're using a definition of "understand" that implies conscience of self, I would argue that a calculator is a device that understands nothing except (a subset of) math. That's what makes a calculator reliable in ways that ChatGPT is not. Philosophically speaking it could be argued that no software understands anything, but I think in the context of this discussion "understands" means "has a model of its context and the way one interacts with it", which is something a calculator (and plenty other software) definitely has and ChatGPT has not.

unshavedyak 4y ago
Depends on what the topic of understanding is. In this case it's actually token relationships, right? It does know that very, very well. And there's a lot (.. potentially, hah) that we can do with token relationships. By itself it's unlikely to ever be knowledge of course.. i see it more akin to NLP than knowledge. Which is to say, a general purpose language parsing tool which we can hand the result to something else. A conversational API, if you will, but we'll still need layers to actually run logic. To know math if you will. Disclaimer: I know very little on the subject. Pure speculation.
- SpicyLemonZest 4y ago
  The question is how sure we should be that the kind of knowledge you and I have is fundamentally different than token relationships.
  - unshavedyak 4y ago
    I definitely have that question too. I view us as big LLMs. But, even if we drop that interesting edge case i suspect we can make something very useful with the primitive that LLMs offer.. in the calculator example. ChainLang and co seem a really interesting tool for LLMs.
  - btown 4y ago
    And additionally, whether our memory and long term learning - and even our goal-choosing - is fundamentally different from an indexed storage of strings of tokens that can be brought back into short-term context when “triggered” by their embedding-similarity to the current context.
- Sharlin 4y ago
  The question is what happens when you go multimodal (which these things can do) and GPT(N+1) learns the associations between words and images/video, as well as the relationships between successive frames of video, at what point does it become unreasonable to claim that it doesn't "understand" something? How good at general-purpose predicting does an AI have to be in order for people to accept that it obviously has an internal model of things and is capable of abstractions? (Assuming that this happens, of course. Diminishing returns could make scaling infeasible past some point, for instance.)
JeremyNT 4y ago
I think this is where things can get kind of interesting, because future integrations of ChatGPT can farm the "real work" out to systems and tools which do have better models of the specific query. The LLM approach may not be able to replicate the "knowledge" your calculator has, but it (or some pre/postprocessor) may be able to recognize that a given question is actually something a calculator can answer concretely, and then it can delegate the computation to traditional software that really does "know" how to answer the question.
- snickerbockers 4y ago
  That would work, but it seems antithetical to AI to have to treat every operation as a special case like that. They want GPT to be able to write computer programs but it'll never be able to work completely independent of humans if every possible domain needs its own plugin to be reliable.
  - Gud 4y ago
    Why though? Our brains have different processing centers. Why would it be different for a general purpose A.I?
    - snickerbockers 4y ago
      Because of you have a special case plug-in for everything then the AI is just a natural language processor, and there's no deep learning for the actual functionality. >Our brains have different processing centers. Uhh, no they don't? Did you know everything you know now about math when you were born, and are you also incapable of learning new things about math? Because that's how the wolfram plug-in works.
  - JeremyNT 4y ago
    I know lots of the SV VC oligarch cult just wants to race forward and create something like AGI that will help them conquer the world and achieve immortality somehow, but hopefully this remains in the realm of science fiction. As RMS says, LLMs at least don't seem to be "it" no matter how much user generated data they ingest, because they do have these inherent limitations. The far more practical (profitable) outcome for what they currently built is to just make a useful tool, a "smarter" wolfram alpha, and that can be iterated upon by delegating relevant operations to specific techniques that are more applicable to the question at hand.
lazyant 4y ago
The Chinese Room Argument holds that a digital computer executing a program cannot have a "mind", "understanding", or "consciousness",[a] regardless of how intelligently or human-like the program may make the computer behave. https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
- alalv 4y ago
  Although the argument holds that the computer "cannot" have a mind, I think the experiment only shows that it "needs not" have a mind.
  - smitty1e 4y ago
    The Chinese Room experiment shows that pattern matching would return correct results for staged inputs, one would not "learn" enough to evaluate an expression not contained in the data.
    - barrkel 4y ago
      The Chinese Room thought experiment is not convincing to software engineers generally. It relies heavily on an intuition that looking things up in a book is clearly not "thinking". Software engineers know better: that "looking things up", if you can do it billions and trillions of times a second, can simulate a process which has a close correspondence to reasoning.
      - smitty1e 4y ago
        Calculating a hash value for a string and hitting a lookup table hardly seems "thinking", even at scale.
        barrkel 4y ago
        Addition and multiplication are trivially implemented using lookup, if you had a machine without arithmetic and only control flow and memory operations. You don't need much more than that for matrix operations, and now you have ChatGPT, a decent simulation of apparent thinking - which is all that is necessary to kill the intuition dead.
barrkel 4y ago
Calculators don't understand anything about arithmetic. They have no circuits for understanding, no code for understanding, nothing that could represent what humans mean by understanding. They implement a set of physical processes that, when operated and interpreted by humans, can be mapped into a subset of arithmetic. There's a correspondence. Correspondence is the most useful way to think about it IMO. If there's a correspodence between what the machine does, and things we humans understand, then the machine, as a tool, is useful. Understanding is a loaded word. It has implications beyond correspondence when humans use it; it has aspects of qualia, of fact vs fiction, of situatedness in a graph of comprehension, of consonance or dissonance with a set of other concepts, and so on. LLMs in my opinion have a good "situatedness" for words and concepts, relative to other concepts. Qualia - consciousness - arguably doesn't matter. Fact vs fiction, they're very shaky on. Consonance vs dissonance, they're useless at - LLMs IME tend to flatter the prompt, constructing arguments in whatever direction a loaded question leads. There's little to no coherence there at all.

unshavedyak 4y ago

Depends on what the topic of understanding is. In this case it's actually token relationships, right? It does know that very, very well. And there's a lot (.. potentially, hah) that we can do with token relationships. By itself it's unlikely to ever be knowledge of course.. i see it more akin to NLP than knowledge. Which is to say, a general purpose language parsing tool which we can hand the result to something else. A conversational API, if you will, but we'll still need layers to actually run logic. To know math if you will. Disclaimer: I know very little on the subject. Pure speculation.

SpicyLemonZest 4y ago
The question is how sure we should be that the kind of knowledge you and I have is fundamentally different than token relationships.
- unshavedyak 4y ago
  I definitely have that question too. I view us as big LLMs. But, even if we drop that interesting edge case i suspect we can make something very useful with the primitive that LLMs offer.. in the calculator example. ChainLang and co seem a really interesting tool for LLMs.
- btown 4y ago
  And additionally, whether our memory and long term learning - and even our goal-choosing - is fundamentally different from an indexed storage of strings of tokens that can be brought back into short-term context when “triggered” by their embedding-similarity to the current context.
Sharlin 4y ago
The question is what happens when you go multimodal (which these things can do) and GPT(N+1) learns the associations between words and images/video, as well as the relationships between successive frames of video, at what point does it become unreasonable to claim that it doesn't "understand" something? How good at general-purpose predicting does an AI have to be in order for people to accept that it obviously has an internal model of things and is capable of abstractions? (Assuming that this happens, of course. Diminishing returns could make scaling infeasible past some point, for instance.)

JeremyNT 4y ago

I think this is where things can get kind of interesting, because future integrations of ChatGPT can farm the "real work" out to systems and tools which do have better models of the specific query. The LLM approach may not be able to replicate the "knowledge" your calculator has, but it (or some pre/postprocessor) may be able to recognize that a given question is actually something a calculator can answer concretely, and then it can delegate the computation to traditional software that really does "know" how to answer the question.

snickerbockers 4y ago
That would work, but it seems antithetical to AI to have to treat every operation as a special case like that. They want GPT to be able to write computer programs but it'll never be able to work completely independent of humans if every possible domain needs its own plugin to be reliable.
- Gud 4y ago
  Why though? Our brains have different processing centers. Why would it be different for a general purpose A.I?
  - snickerbockers 4y ago
    Because of you have a special case plug-in for everything then the AI is just a natural language processor, and there's no deep learning for the actual functionality. >Our brains have different processing centers. Uhh, no they don't? Did you know everything you know now about math when you were born, and are you also incapable of learning new things about math? Because that's how the wolfram plug-in works.
- JeremyNT 4y ago
  I know lots of the SV VC oligarch cult just wants to race forward and create something like AGI that will help them conquer the world and achieve immortality somehow, but hopefully this remains in the realm of science fiction. As RMS says, LLMs at least don't seem to be "it" no matter how much user generated data they ingest, because they do have these inherent limitations. The far more practical (profitable) outcome for what they currently built is to just make a useful tool, a "smarter" wolfram alpha, and that can be iterated upon by delegating relevant operations to specific techniques that are more applicable to the question at hand.

lazyant 4y ago

The Chinese Room Argument holds that a digital computer executing a program cannot have a "mind", "understanding", or "consciousness",[a] regardless of how intelligently or human-like the program may make the computer behave. https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room

alalv 4y ago
Although the argument holds that the computer "cannot" have a mind, I think the experiment only shows that it "needs not" have a mind.
- smitty1e 4y ago
  The Chinese Room experiment shows that pattern matching would return correct results for staged inputs, one would not "learn" enough to evaluate an expression not contained in the data.
  - barrkel 4y ago
    The Chinese Room thought experiment is not convincing to software engineers generally. It relies heavily on an intuition that looking things up in a book is clearly not "thinking". Software engineers know better: that "looking things up", if you can do it billions and trillions of times a second, can simulate a process which has a close correspondence to reasoning.
    - smitty1e 4y ago
      Calculating a hash value for a string and hitting a lookup table hardly seems "thinking", even at scale.
      - barrkel 4y ago
        Addition and multiplication are trivially implemented using lookup, if you had a machine without arithmetic and only control flow and memory operations. You don't need much more than that for matrix operations, and now you have ChatGPT, a decent simulation of apparent thinking - which is all that is necessary to kill the intuition dead.

barrkel 4y ago

Calculators don't understand anything about arithmetic. They have no circuits for understanding, no code for understanding, nothing that could represent what humans mean by understanding. They implement a set of physical processes that, when operated and interpreted by humans, can be mapped into a subset of arithmetic. There's a correspondence. Correspondence is the most useful way to think about it IMO. If there's a correspodence between what the machine does, and things we humans understand, then the machine, as a tool, is useful. Understanding is a loaded word. It has implications beyond correspondence when humans use it; it has aspects of qualia, of fact vs fiction, of situatedness in a graph of comprehension, of consonance or dissonance with a set of other concepts, and so on. LLMs in my opinion have a good "situatedness" for words and concepts, relative to other concepts. Qualia - consciousness - arguably doesn't matter. Fact vs fiction, they're very shaky on. Consonance vs dissonance, they're useless at - LLMs IME tend to flatter the prompt, constructing arguments in whatever direction a loaded question leads. There's little to no coherence there at all.