43 ms·

Sure but AI boosters alternately ignore these issues or dismiss them every time they come up. If it’s an extremely error prone tool with limited use cases let’s

Sure but AI boosters alternately ignore these issues or dismiss them every time they come up. If it’s an extremely error prone tool with limited use cases let’s be honest and call it what it is. “People also make mistakes” isn’t a good enough defense for a technology with this much hype and funding.

ryandrake 2y ago
Yea, people make math mistakes all the time, but I’d expect my calculator app to multiply correctly every time I used it. We should hold computers to a higher standard if we are going to rely on them.
- II2II 2y ago
  Yet the Windows 3.1 calculator couldn't subtract properly. While I bring that example up in jest, there are real limitations to how computers do math. The calculator app may produce correct results for everyday problems. Yet there are many domains where you must know how floating point numbers are handled, how the computer handles trigonometric functions, etc.. It's not that the computer is wrong. There are simply limitations due to how floating point numbers are represented. Even integers can be problematic due to their own limitations.
  - ryandrake 2y ago
    > Yet the Windows 3.1 calculator couldn't subtract properly. OK, but I'm sure that Microsoft treated that as a bug to be fixed, rather than as an inherent limitation of computers that we just need to understand and deal with.
  - cageface 2y ago
    This would be a useful analogy if Microsoft promoted the calculator as the next epoch-making trillion dollar revolution in computing and then swept aside the numerous mistakes it made on simple inputs as no worse than the average human.
- brookst 2y ago
  I think this is the heart of most of the angst around AI: it runs on computers, computers are precise and deterministic, therefore AI must be precise and deterministic. But… it just doesn’t work that way. There is tons of room for improvement in safety and reliability, but expecting a multi-billion parameter neural network to have the same accuracy properties as a software calculator is always going to lead to frustration. Complex systems have complex failure modes. There is a reason we use hammers and not CNC machine presses for nails.
  - cageface 2y ago
    Right but this is way too often glossed over in the rush to hype the new models. I see even many people that should know better failing to treat their output with appropriate skepticism. So much money is being dumped into this stuff now there's a huge incentive to sweep the shortcomings under the rug.
    - brookst 2y ago
      Perhaps? But I'm not sure I see the value in saying that other people aren't doing a good job of setting expectations with yet other people. Presumably we around here know, right? And it's always fraught to imagine problems third hand.
      - cageface 2y ago
        Presumably we around here know, right? No that's the problem. Here and also among other smart, informed people I know I keep seeing people post unchecked LLM output.

ryandrake 2y ago

Yea, people make math mistakes all the time, but I’d expect my calculator app to multiply correctly every time I used it. We should hold computers to a higher standard if we are going to rely on them.

II2II 2y ago
Yet the Windows 3.1 calculator couldn't subtract properly. While I bring that example up in jest, there are real limitations to how computers do math. The calculator app may produce correct results for everyday problems. Yet there are many domains where you must know how floating point numbers are handled, how the computer handles trigonometric functions, etc.. It's not that the computer is wrong. There are simply limitations due to how floating point numbers are represented. Even integers can be problematic due to their own limitations.
- ryandrake 2y ago
  > Yet the Windows 3.1 calculator couldn't subtract properly. OK, but I'm sure that Microsoft treated that as a bug to be fixed, rather than as an inherent limitation of computers that we just need to understand and deal with.
- cageface 2y ago
  This would be a useful analogy if Microsoft promoted the calculator as the next epoch-making trillion dollar revolution in computing and then swept aside the numerous mistakes it made on simple inputs as no worse than the average human.
brookst 2y ago
I think this is the heart of most of the angst around AI: it runs on computers, computers are precise and deterministic, therefore AI must be precise and deterministic. But… it just doesn’t work that way. There is tons of room for improvement in safety and reliability, but expecting a multi-billion parameter neural network to have the same accuracy properties as a software calculator is always going to lead to frustration. Complex systems have complex failure modes. There is a reason we use hammers and not CNC machine presses for nails.
- cageface 2y ago
  Right but this is way too often glossed over in the rush to hype the new models. I see even many people that should know better failing to treat their output with appropriate skepticism. So much money is being dumped into this stuff now there's a huge incentive to sweep the shortcomings under the rug.
  - brookst 2y ago
    Perhaps? But I'm not sure I see the value in saying that other people aren't doing a good job of setting expectations with yet other people. Presumably we around here know, right? And it's always fraught to imagine problems third hand.
    - cageface 2y ago
      Presumably we around here know, right? No that's the problem. Here and also among other smart, informed people I know I keep seeing people post unchecked LLM output.