9 ms·

How often are calculators confidently wrong?

by tricksforfree 4y ago

lolsowrong 4y ago
Depends on the calculator. Floating point imprecision is well documented.
- tricksforfree 4y ago
  That's true. But we know exactly why it can't do it.
  - ben_w 4y ago
    How important is the "knowing why" if the mistakes are still there? And in reverse, we "know" GPT doesn't use a calculator unless specially pointed at one. Floating point errors creeping in is why we have to use quaternions instead of matrixes for 3D games. Apparently. I'd already given up on doing my own true-3D game engine by that point. In some sense we "know why" humans make mistakes too — and in many fields from advertising to political zeitgeist we manipulate using knowledge of common human flaws. On this basis I think the application of pedagogical and psychological studies to AI will be increasingly important.
- kgwxd 4y ago
  Well documented is the key difference.
alex_sf 4y ago
How often are people?
- SketchySeaBeast 4y ago
  I think the difference, right now at least, is that people will go, "well, I'm not sure about this so I think we should look it up, but this is what I think" - the AI doesn't do that. It lies in the same exact way it tells truths. How are you supposed to make decisions based off of that information?
  - marcusjt 4y ago
    Does it lie? Or just get things wrong sometimes? Lying requires knowledge that what you are saying is not the truth, and usually there's a motive for doing so. I don't think ChatGPT is there yet... or is it?
    - SketchySeaBeast 4y ago
      Sure, it's not lying, you're right, there's no will there, I'm anthropomorphism. It is producing entirely wrong facts / pseudo-opinions (as it can't actually have an opinion).
      - ben_w 4y ago
        I was about to suggest "pathologically dishonest", but then I looked up the term and that seems to require being biased in favour of the speaker and knowing that you're saying falsehoods. "Confabulate" however, appears to be a good description. Confabulation is, I'm told, associated with Alzheimer's, and GPT's output does sometimes remind me of a few things my mum said while she was ill.
    - giaour 4y ago
      Technically, what ChatGPT is doing is bullshitting because it doesn't have any knowledge of or concern for truthfulness. https://en.m.wikipedia.org/wiki/On_Bullshit https://en.m.wikipedia.org/wiki/On_Bullshit
  - alex_sf 4y ago
    Presumably the same way you make decisions on any piece of information. You should not be blindly trusting a single source.
    - SketchySeaBeast 4y ago
      I was too vague I think. The only place where I can see it being acceptable right now is code because I have a whole other system that will call out failures - I can rely on my IDE and my own expertise to hopefully catch issues when they appear. Outside of the code use case, what should I rely on ChatGPT for that won't have me also looking for the information somewhere else? I suppose subjective soft things, like writing communications. But I can't rely on it for information.
japhyr 4y ago
As a math teacher, this is such a funny comparison to keep reading. Yes, there's a difference between a deterministic outcome and a non-deterministic one. But throw humans into the loop, and it becomes more interesting. I can't count the number of times I've listened to someone argue their answer must be right because they got it from the calculator. And it's not just students; as a teacher I've always paid attention to how adults use math. With calculators or GPT tools, or any other automated assistant, judgement and validation continues to matter.
- Wowfunhappy 4y ago
  > I can't count the number of times I've listened to someone argue their answer must be right because they got it from the calculator. Answers from calculators are always right! But the human may have asked the wrong question.
  - lolsowrong 4y ago
    Go type .1*.2 into any JavaScript console. Edit: slapping a few more in here: https://learn.microsoft.com/en-us/office/troubleshoot/excel/floating-point-arithmetic-inaccurate-result https://learn.microsoft.com/en-us/office/troubleshoot/excel/... https://daviddeley.com/pentbug/index.htm https://daviddeley.com/pentbug/index.htm
    - tricksforfree 4y ago
      One deck is made of wood. One deck is made made of steel. They will behave differently after years of weathering. Just because they are both both decks doesn't mean they are the same.
      - lolsowrong 4y ago
        Are both useful in some context?
        whynaut 4y ago
fnordpiglet 4y ago
Every time the human supervising it makes a mistake in their role in the relationship between user and tool.
Veen 4y ago
There's a lot of work happening at the moment around self-reflection and getting LLMs to identify and correct their own hallucinations and mistakes. https://arxiv.org/pdf/2303.11366.pdf https://arxiv.org/pdf/2303.11366.pdf
- lanstin 4y ago
  I suspect some temporality will need to be added. There are times when writing the code you have a question because the code exposes an unexpressed choice in the requirements. When you are coding in linear time, you then know to go ask the question. I am not sure that just generating the most likely or most rewarded response will do that easily. It seems to just arbitrarily pick the most likely requirement.
ipaddr 4y ago
As often as people put in incorrect values. As often as someone goes beyond the range. Always when someone tries to add yellow to blue. Often when the wrong formula is used. And in this case you don't get to put in the formula.
wkat4242 4y ago
This is the thing. ChatGPT shouldn't be that confident in its wording IMO.. If it just said "according to me" instead of stating things as fact, people would have much less problems with it. We know this wording is just show but people still get swayed by it and believe it must be true.
- AlecSchueler 4y ago
  Yeah and it can actually reflect on itself and its mistakes when prompted, so I can see a fix like this coming soon. Sometimes just asking, Are you sure? is enough for it to apologize and say a part of its answer wasn't based in known fact. Also another point while I'm here: Many many humans I've met are often confident when incorrect as well and can and will bullshit an answer when it suits their comfort.
  - lanstin 4y ago
    But no one asserts the existence of those people will forever change society.
hn_throwaway_99 4y ago
People keep saying this, pointing out the "mistakes with confidence" aspect of LLMs, but as someone who is continually amazed by ChatGPT and finds it very useful in my day-to-day, it's hard for me to take this objection seriously if presented as a reason not to use AI. That is, for me, the output of ChatGPT or other AI tools is the starting point of my investigation, not the end output. Yes, if you just blindly paste the output from an AI tool you're going to have a bad time, but we also standardize code reviews into the human code-writing process - this isn't that different. Just giving one specific example, I find ChatGPT to my an incredibly efficient "documentation lookup tool". E.g. it's great if I'm working with a new technology or API and I want to know "what my options are", but I don't know what keywords to search for, it can help give me a really good "lay of the land", and from there I can read on my own to get more specifics.
- the_doctah 4y ago
  Maybe you haven't used it enough. ChatGPT is wrong all the time for me, sometimes insultingly wrong. The confidence in it's incorrect answers just makes it that much worse. I can't buy any of this hype for a "word-putting-together" algorithm. It's not real intelligence.
  - hn_throwaway_99 4y ago
    Please give some examples then. I've found the GPT-4 version to be remarkably accurate, and when it makes mistakes it's not hard to spot them. For example, I commented last week that I've found ChatGPT to be a great tool for managing my task list, and for whatever reason the "verbal" back-and-forth works much better for my brain than a simple checklist-based todo app: https://news.ycombinator.com/item?id=35390644 https://news.ycombinator.com/item?id=35390644 . But, I also pointed out how it will get the sums for my "task estimate totals by group" wrong. But it's so easy to see this mistake, and after using it for a while I have a good understanding for when it's likely to occur, that it doesn't lessen the value I get from using the tool.
    - timr 4y ago
pwinnski 4y ago
There are three types of lies, as the saying goes: lies, damned lies, and statistics. But why are statistics considered lies? Because of how they're used. If you think of AI as a source of truth, obviously you're going to run into trouble: it "lies"! But if instead of thinking of it in isolation, you think of the person+AI producing results, then you should trust that person exactly as much as you would whether or not they use AI.
RcouF1uZ4gsC 4y ago
Calculators use floating point and can have catastrophic errors if not used correctly. So yes, calculators can be confidently wrong.
onion2k 4y ago
AI is confidently wrong about a lot of things, but that doesn't mean it's useless. It means you need to verify what it generates. Doing that for code is much easier than prose. AI that produces wrong code is immediately and obviously wrong. It can't really fool you. It's easy to test. You can even ask AI to write tests for the code it produces to demonstrate it's correct.
- weatherlite 4y ago
  Tests help but tests can be wrong as well, if this was all so easy we wouldn't have any bugs.
- lanstin 4y ago
  Just because it is making obvious errors doesn’t mean it isn’t also making subtle errors.
sharemywin 4y ago
People are confidently wrong all the time and yet we still seem to get stuff done. A tool is a tool. it has good uses and not good uses. as the human you figure out where it works and where it doesn't.
carlosjobim 4y ago
Every time they are wrong, which is every time the user slips on a key.

How often are calculators confidently wrong?

lolsowrong 4y ago

Depends on the calculator. Floating point imprecision is well documented.

tricksforfree 4y ago
That's true. But we know exactly why it can't do it.
- ben_w 4y ago
  How important is the "knowing why" if the mistakes are still there? And in reverse, we "know" GPT doesn't use a calculator unless specially pointed at one. Floating point errors creeping in is why we have to use quaternions instead of matrixes for 3D games. Apparently. I'd already given up on doing my own true-3D game engine by that point. In some sense we "know why" humans make mistakes too — and in many fields from advertising to political zeitgeist we manipulate using knowledge of common human flaws. On this basis I think the application of pedagogical and psychological studies to AI will be increasingly important.
kgwxd 4y ago
Well documented is the key difference.

alex_sf 4y ago

How often are people?

SketchySeaBeast 4y ago
I think the difference, right now at least, is that people will go, "well, I'm not sure about this so I think we should look it up, but this is what I think" - the AI doesn't do that. It lies in the same exact way it tells truths. How are you supposed to make decisions based off of that information?
- marcusjt 4y ago
  Does it lie? Or just get things wrong sometimes? Lying requires knowledge that what you are saying is not the truth, and usually there's a motive for doing so. I don't think ChatGPT is there yet... or is it?
  - SketchySeaBeast 4y ago
    Sure, it's not lying, you're right, there's no will there, I'm anthropomorphism. It is producing entirely wrong facts / pseudo-opinions (as it can't actually have an opinion).
    - ben_w 4y ago
      I was about to suggest "pathologically dishonest", but then I looked up the term and that seems to require being biased in favour of the speaker and knowing that you're saying falsehoods. "Confabulate" however, appears to be a good description. Confabulation is, I'm told, associated with Alzheimer's, and GPT's output does sometimes remind me of a few things my mum said while she was ill.
  - giaour 4y ago
    Technically, what ChatGPT is doing is bullshitting because it doesn't have any knowledge of or concern for truthfulness. https://en.m.wikipedia.org/wiki/On_Bullshit https://en.m.wikipedia.org/wiki/On_Bullshit
- alex_sf 4y ago
  Presumably the same way you make decisions on any piece of information. You should not be blindly trusting a single source.
  - SketchySeaBeast 4y ago
    I was too vague I think. The only place where I can see it being acceptable right now is code because I have a whole other system that will call out failures - I can rely on my IDE and my own expertise to hopefully catch issues when they appear. Outside of the code use case, what should I rely on ChatGPT for that won't have me also looking for the information somewhere else? I suppose subjective soft things, like writing communications. But I can't rely on it for information.

japhyr 4y ago

As a math teacher, this is such a funny comparison to keep reading. Yes, there's a difference between a deterministic outcome and a non-deterministic one. But throw humans into the loop, and it becomes more interesting. I can't count the number of times I've listened to someone argue their answer must be right because they got it from the calculator. And it's not just students; as a teacher I've always paid attention to how adults use math. With calculators or GPT tools, or any other automated assistant, judgement and validation continues to matter.

Wowfunhappy 4y ago
> I can't count the number of times I've listened to someone argue their answer must be right because they got it from the calculator. Answers from calculators are always right! But the human may have asked the wrong question.
- lolsowrong 4y ago
  Go type .1*.2 into any JavaScript console. Edit: slapping a few more in here: https://learn.microsoft.com/en-us/office/troubleshoot/excel/floating-point-arithmetic-inaccurate-result https://learn.microsoft.com/en-us/office/troubleshoot/excel/... https://daviddeley.com/pentbug/index.htm https://daviddeley.com/pentbug/index.htm
  - tricksforfree 4y ago
    One deck is made of wood. One deck is made made of steel. They will behave differently after years of weathering. Just because they are both both decks doesn't mean they are the same.
    - lolsowrong 4y ago
      Are both useful in some context?
      - whynaut 4y ago

fnordpiglet 4y ago

Every time the human supervising it makes a mistake in their role in the relationship between user and tool.

Veen 4y ago

There's a lot of work happening at the moment around self-reflection and getting LLMs to identify and correct their own hallucinations and mistakes. https://arxiv.org/pdf/2303.11366.pdf https://arxiv.org/pdf/2303.11366.pdf

lanstin 4y ago
I suspect some temporality will need to be added. There are times when writing the code you have a question because the code exposes an unexpressed choice in the requirements. When you are coding in linear time, you then know to go ask the question. I am not sure that just generating the most likely or most rewarded response will do that easily. It seems to just arbitrarily pick the most likely requirement.

ipaddr 4y ago

As often as people put in incorrect values. As often as someone goes beyond the range. Always when someone tries to add yellow to blue. Often when the wrong formula is used. And in this case you don't get to put in the formula.

wkat4242 4y ago

This is the thing. ChatGPT shouldn't be that confident in its wording IMO.. If it just said "according to me" instead of stating things as fact, people would have much less problems with it. We know this wording is just show but people still get swayed by it and believe it must be true.

AlecSchueler 4y ago
Yeah and it can actually reflect on itself and its mistakes when prompted, so I can see a fix like this coming soon. Sometimes just asking, Are you sure? is enough for it to apologize and say a part of its answer wasn't based in known fact. Also another point while I'm here: Many many humans I've met are often confident when incorrect as well and can and will bullshit an answer when it suits their comfort.
- lanstin 4y ago
  But no one asserts the existence of those people will forever change society.

hn_throwaway_99 4y ago

People keep saying this, pointing out the "mistakes with confidence" aspect of LLMs, but as someone who is continually amazed by ChatGPT and finds it very useful in my day-to-day, it's hard for me to take this objection seriously if presented as a reason not to use AI. That is, for me, the output of ChatGPT or other AI tools is the starting point of my investigation, not the end output. Yes, if you just blindly paste the output from an AI tool you're going to have a bad time, but we also standardize code reviews into the human code-writing process - this isn't that different. Just giving one specific example, I find ChatGPT to my an incredibly efficient "documentation lookup tool". E.g. it's great if I'm working with a new technology or API and I want to know "what my options are", but I don't know what keywords to search for, it can help give me a really good "lay of the land", and from there I can read on my own to get more specifics.

the_doctah 4y ago
Maybe you haven't used it enough. ChatGPT is wrong all the time for me, sometimes insultingly wrong. The confidence in it's incorrect answers just makes it that much worse. I can't buy any of this hype for a "word-putting-together" algorithm. It's not real intelligence.
- hn_throwaway_99 4y ago
  Please give some examples then. I've found the GPT-4 version to be remarkably accurate, and when it makes mistakes it's not hard to spot them. For example, I commented last week that I've found ChatGPT to be a great tool for managing my task list, and for whatever reason the "verbal" back-and-forth works much better for my brain than a simple checklist-based todo app: https://news.ycombinator.com/item?id=35390644 https://news.ycombinator.com/item?id=35390644 . But, I also pointed out how it will get the sums for my "task estimate totals by group" wrong. But it's so easy to see this mistake, and after using it for a while I have a good understanding for when it's likely to occur, that it doesn't lessen the value I get from using the tool.
  - timr 4y ago

pwinnski 4y ago

There are three types of lies, as the saying goes: lies, damned lies, and statistics. But why are statistics considered lies? Because of how they're used. If you think of AI as a source of truth, obviously you're going to run into trouble: it "lies"! But if instead of thinking of it in isolation, you think of the person+AI producing results, then you should trust that person exactly as much as you would whether or not they use AI.

RcouF1uZ4gsC 4y ago

Calculators use floating point and can have catastrophic errors if not used correctly. So yes, calculators can be confidently wrong.

onion2k 4y ago

AI is confidently wrong about a lot of things, but that doesn't mean it's useless. It means you need to verify what it generates. Doing that for code is much easier than prose. AI that produces wrong code is immediately and obviously wrong. It can't really fool you. It's easy to test. You can even ask AI to write tests for the code it produces to demonstrate it's correct.

weatherlite 4y ago
Tests help but tests can be wrong as well, if this was all so easy we wouldn't have any bugs.
lanstin 4y ago
Just because it is making obvious errors doesn’t mean it isn’t also making subtle errors.

sharemywin 4y ago

People are confidently wrong all the time and yet we still seem to get stuff done. A tool is a tool. it has good uses and not good uses. as the human you figure out where it works and where it doesn't.

carlosjobim 4y ago

Every time they are wrong, which is every time the user slips on a key.