4 ms·

The problem is that it is not fixable in the language model without implementing special logic for mortgage calculations, like "if the user asks about mortgage

The problem is that it is not fixable in the language model without implementing special logic for mortgage calculations, like "if the user asks about mortgage calculations, do this (e.g. invoke python program) instead of regular processing. That is, this class of questions should be "special-cased" somehow. But the number of special cases like this is potentially infinite. Even to answer "I don't know how to answer this question", the engine has to be specifically instructed to do so.

MPSimmons 4y ago
Yeah, all specialty subjects are like this (or math that's sufficiently different from its learned text - have it square a very, very large number with a bunch of random digits and compare that against the actual answer from a calculator). To actually solve this, we need AGI.
- iainmerrick 4y ago
  To actually solve this, we need AGI. That’s good to know -- once we reach that last milestone, we can finally stop moving those goalposts!!
  - Jensson 4y ago
    "Goalpost moving" is just a strawman, what is happening is that people give counterexamples of what the AI can't do, and then you guys the AI positivists treats those as goalposts, but they weren't. For example if I say "this bot isn't smart, it can't even play chess", I don't mean that if you hardcode a chess engine into the AI that it is then smart, I just mentioned one of the myriad of things I expect a smart bot to be able to learn. So when you come back and hardcoded chess in it, then I'll mention one of the myriad of other things I expect it to be able to learn, like playing Go for example, that wasn't me moving the goalpost, that was you interpreting my counterexample as a goalpost, the goalpost was always the same: make an AI that can learn and perform the same intellectual tasks as a typical human. Me mentioning specific counterexamples aren't goalposts, those are just the easiest way to refute the claim "this is general intelligence!". And when people like you treat those as goalposts it means that you aren't even understanding the conversation, there is no way you will convince anyone by behaving like that.
    - iainmerrick 4y ago
      Sorry to be pedantic, but the GP said "to solve this, we need AGI", implying that it is a goalpost. More generally, I don't think you have to be an "AI positivist" to note that expectations have shifted a lot, which is really all I meant by "moving the goalposts". If you had told me even 10 years ago some of the things that can be done with AI now, I'd have thought it was completely impossible -- the kinds of things done routinely now are exactly those that we used to think were particularly hard tasks for AI. And yet the consensus (which I agree with!) is that AI is still not actually intelligent. There are a couple of downsides to your strict "counterexample" approach. First, it can smack of gatekeeping a bit -- it's like trying to define art. If you can't actually say what art is, except to point out which artworks you like and which you don't like, are you saying anything useful? Is "art" even a real concept? I would like to think it is, but it's a very complicated question. Second, a lot of humans have trouble with the same kinds of things that AI has trouble with! Does that mean those humans aren't intelligent? Or are there different kinds of intelligence -- but if so, why can't AI be one of them? Right now of course you can generally tell the difference, because AI tends to fail in certain in-human ways. But if that's not a goalpost, what will it mean when it becomes difficult or impossible to tell the difference between AI errors and human errors?
      - MPSimmons 4y ago
        To be very clear, what modern LLMs can do is very, very impressive. Because they can seem so human, we start to anthropomorphize them and ascribe tendencies and behaviors, or even preferences, when those specific words belie what's actually happening to produce the output we see. I do it too. But when you give a sufficiently advanced LLM a chessboard and ask it to checkmate black, and it does, it has not solved the problem of playing chess. It has solved the problem of giving you output that it thinks meets the conditions that satisfy your query the most closely of the potential outcomes it can produce. It is still an LLM. I think that if you wanted an LLM to solve limitless arbitrary problems, you would need to represent all potentials in the weights, and that seems impossible. And even then, it's not actually solving the problems you ask, it's solving the problem of you asking.
  - resource0x 4y ago
    The goalpost remains where it was: Turing test. The necessary condition for the machine to pass the test is to stop generating non-sequiturs.
PeterisP 4y ago
To be frank, if we take a human schoolkid who knows some natural language well, getting them to apply this "language model" to make mortgage calculations requires either to give them an external tool for that or do many weeks (at least) of training in mathematics..

MPSimmons 4y ago

Yeah, all specialty subjects are like this (or math that's sufficiently different from its learned text - have it square a very, very large number with a bunch of random digits and compare that against the actual answer from a calculator). To actually solve this, we need AGI.

iainmerrick 4y ago
To actually solve this, we need AGI. That’s good to know -- once we reach that last milestone, we can finally stop moving those goalposts!!
- Jensson 4y ago
  "Goalpost moving" is just a strawman, what is happening is that people give counterexamples of what the AI can't do, and then you guys the AI positivists treats those as goalposts, but they weren't. For example if I say "this bot isn't smart, it can't even play chess", I don't mean that if you hardcode a chess engine into the AI that it is then smart, I just mentioned one of the myriad of things I expect a smart bot to be able to learn. So when you come back and hardcoded chess in it, then I'll mention one of the myriad of other things I expect it to be able to learn, like playing Go for example, that wasn't me moving the goalpost, that was you interpreting my counterexample as a goalpost, the goalpost was always the same: make an AI that can learn and perform the same intellectual tasks as a typical human. Me mentioning specific counterexamples aren't goalposts, those are just the easiest way to refute the claim "this is general intelligence!". And when people like you treat those as goalposts it means that you aren't even understanding the conversation, there is no way you will convince anyone by behaving like that.
  - iainmerrick 4y ago
    Sorry to be pedantic, but the GP said "to solve this, we need AGI", implying that it is a goalpost. More generally, I don't think you have to be an "AI positivist" to note that expectations have shifted a lot, which is really all I meant by "moving the goalposts". If you had told me even 10 years ago some of the things that can be done with AI now, I'd have thought it was completely impossible -- the kinds of things done routinely now are exactly those that we used to think were particularly hard tasks for AI. And yet the consensus (which I agree with!) is that AI is still not actually intelligent. There are a couple of downsides to your strict "counterexample" approach. First, it can smack of gatekeeping a bit -- it's like trying to define art. If you can't actually say what art is, except to point out which artworks you like and which you don't like, are you saying anything useful? Is "art" even a real concept? I would like to think it is, but it's a very complicated question. Second, a lot of humans have trouble with the same kinds of things that AI has trouble with! Does that mean those humans aren't intelligent? Or are there different kinds of intelligence -- but if so, why can't AI be one of them? Right now of course you can generally tell the difference, because AI tends to fail in certain in-human ways. But if that's not a goalpost, what will it mean when it becomes difficult or impossible to tell the difference between AI errors and human errors?
    - MPSimmons 4y ago
      To be very clear, what modern LLMs can do is very, very impressive. Because they can seem so human, we start to anthropomorphize them and ascribe tendencies and behaviors, or even preferences, when those specific words belie what's actually happening to produce the output we see. I do it too. But when you give a sufficiently advanced LLM a chessboard and ask it to checkmate black, and it does, it has not solved the problem of playing chess. It has solved the problem of giving you output that it thinks meets the conditions that satisfy your query the most closely of the potential outcomes it can produce. It is still an LLM. I think that if you wanted an LLM to solve limitless arbitrary problems, you would need to represent all potentials in the weights, and that seems impossible. And even then, it's not actually solving the problems you ask, it's solving the problem of you asking.
- resource0x 4y ago
  The goalpost remains where it was: Turing test. The necessary condition for the machine to pass the test is to stop generating non-sequiturs.

PeterisP 4y ago

To be frank, if we take a human schoolkid who knows some natural language well, getting them to apply this "language model" to make mortgage calculations requires either to give them an external tool for that or do many weeks (at least) of training in mathematics..