7 ms·

Not to refute what you said, but what you describe is quite similar to what we humans call rationalization, and it has been argued (e.g. by Robert Zajonc) that

Not to refute what you said, but what you describe is quite similar to what we humans call rationalization, and it has been argued (e.g. by Robert Zajonc) that most of the time we make decisions intuitively and then seek a rationalization to explain them. Also, good luck with human explanations in the presence of bias. No human is going to say that they refused a loan due to the race or sex of the applicant.

pixl97 4y ago
Well no smart humans, but it turns out there are plenty of dumb ones.
delusional 4y ago
There's probably a lot of truth to this, but there's also a big difference. Humans are ultimately responsible and accountable for their actions. Should my actions turn out to carry a racist or sexist motive, even unbeknownst to me, i can be held to answer for that. That's a big difference. The rationalization from a human is valuable because it's delivered by the accountable party. From a machine such rationalization is at best worthless, since you can't hold the machine accountable at all.
- jvanderbot 4y ago
  But what youre saying only layers over the fact that, to avoid getting caught and convicted, a sufficiently good explanation is all that's necessary.
  - delusional 4y ago
    Not quite. What I'm saying is that the explanation is valuable because it comes from the accountable party. That you have a good explanation does not absolve you of responsibility. The short and clear way to put it is this: You don't go to jail because YOU think you did something wrong, you go to jail because WE think you did something wrong. If I'm a customer at a bank, and my loan has been denied, I don't care what some unaccountable AI system can come up with to explain that. I care what about how the accountable bankers justify putting that AI system into the process in the first place. How do they justify that AI system getting to make decisions that affect me and my life. I don't care about why the process does what it does, I care about why that is the process.
    - coldtea 4y ago
      >If I'm a customer at a bank, and my loan has been denied, I don't care what some unaccountable AI system can come up with to explain that. I care what about how the accountable bankers justify putting that AI system into the process in the first place. How do they justify that AI system getting to make decisions that affect me and my life. I don't care about why the process does what it does, I care about why that is the process. Well, a court/law just has to declare "AI" as allowed to be used in such decisions, and the whole recourse you describe vanishes though...
    - chii 4y ago
      > my loan has been denied, I don't care what some unaccountable AI system can come up with to explain that. > How do they justify that AI system getting to make decisions that affect me and my life. so you, a priori, make the assumption that your loan _should've_ been accepted? If the decision wasn't an AI, but some actuarial that calculates and computes based on a set of criteria, and the result is a denial, you could still make the same argument of "why is _this_ the process, instead of something else (that makes my loan acceptable)?".
    - robertlagrant 4y ago
      > I care what about how the accountable bankers justify putting that AI system into the process in the first place. Computers are already deciding. As to why: it's because it's their money they're lending.
      - delusional 4y ago
        I work in a bank and you're laughably wrong. Computers are at best assisting in the lending process but more realistically they're completely ignored. There is NO AI (at least recognizably, you could argue that there's some sort of very simple expert model embedded in the 10 credit limits) involved in the process at all. Out lending process is based on the judgment of a few specific individuals, with more involved clients requiring approval from more senior people. All steps of that process can be overturned by the overseeing person, and that person is accountable for their decision.
- onlyrealcuzzo 4y ago
  More importantly, to the best of our knowledge, human thinking is not simply predicting the next word based on a past model.
  - fsckboy 4y ago
    but the fact that predicting the next words based on past models does such a good job of masquerading as human thinking indicates that much of human thinking isn't much more than that. It may even simplify the task of figuring human thinking by narrowing down the last details of of the smaller set of missing pieces. an analagous result was obtained back when they mapped the small finite number of neurons in a snail brain, or the behavior of individuals in ant colonies. What looks like complex behavior turns out to be very simple under the hood. for the vast ocean of the population who... not sure how to describe them... not good students when in school, would rather spend the bulk of their time with the TV blaring, eating cheetos and swiping on tik-tok, following the lives of celebrities and fighting about it, rather than do anything long term productive with their own lives... chat gpt may have already exceeded what they do with their cranial talents. even a level up on the ladder, the types of office situations lampooned in The Office or Dilbert, are they doing much more as a percentage of time spent than chat GPT can do? "Mondays, amirite!?" then the question becomes, are the intellectual elites among us doing that much more, or just doing much more of the same thing? I think a large portion of what we do is exactly what chap GPT does. The question is what is this other piece of our brains' that intervenes to say "hmm, need to think about this part a lot harder"
    - kortilla 4y ago
      >such a good job of masquerading as human thinking indicates that much of human thinking isn't much more than that. No, that doesn’t follow. It just means it roughly looks like human thinking. Your comment is akin to saying a high resolution photo of a human has basically figured out a way to replicate humans. It looks like it in one aspect but it’s laughably wrong. Humans thought without language.
      - fsckboy 4y ago
- wilg 4y ago
  We don't know that it's true that humans are "responsible" or "accountable" for their actions in any meaningful way. This smells like a "free will" argument. IMHO, it's unlikely "free will" and "responsibility" are anything more than an illusion.
  - WA 4y ago
    It really doesn’t matter for this discussion, because society and laws are structured as if people have free will. An AI has to live up to that as well.
    - wilg 4y ago
      Maybe I'm confused about the topic, then. What does "living up to that" mean here? Legal liability or something?
      - tsimionescu 4y ago
        Yes, that was their point. Humans have accountability to one another - either legally or in less formal ways (such as being fired if you're making your co-workers uncomfortable). Current machines simply don't have that kind of accountability. Even if we wanted to, we can't punish or ostracize ChatGPT when it lies to us, or makes us uncomfortable.
        vixen99 4y ago
        Their use & application can be made accountable.
        tsimionescu 4y ago
        The point of this thread is that the explanation a human gives carries some weight because the human know they may/will be held accountable for the truth of that explanation. ChatGPT's explanations carry no such weight, since ChatGPT itself has no concept of suffering consequences for its actions. So, while both humans and ChatGPT can and do give bogus explanations for their actions, there are reasons to trust the humans' explanations more than ChatGPT's. Whether or not we hold humans using ChatGPT accountable for their use of it is irrelevant to this thread.
- smeagull 4y ago
  The only intent taking place here, is someone placing a process under automatic control. That's it. If I make a bot that posts random comments and it offends you, then I'm a bit responsible, but also you've gotten offended by a random comment.
geraneum 4y ago
Although humans are capable of rationalizing and they do it, they have other capabilities which is a key point that is missing in this argument. Maybe no one will admit to refusing a loan based on applicant’s gender, but also no real world aircraft engineer will explain why they decided to design a plane’s wing in a certain shape, purely by rationalizing an intuition without backing it by math and physics. Also, there are a group of humans elsewhere that understand those math and using the “same” principles can follow the explanation and detect mistakes or baseless rationalized explanations.
- composter 4y ago
  Using airfoil design rationalization as an example may not be the best choice in this context since there is a fairly long history of incorrect rationalizations for wing lift. "Incorrect Lift Theories": https://www.grc.nasa.gov/www/k-12/VirtualAero/BottleRocket/airplane/wrong1.html https://www.grc.nasa.gov/www/k-12/VirtualAero/BottleRocket/a... "No One Can Explain Why Planes Stay in the Air": https://www.scientificamerican.com/article/no-one-can-explain-why-planes-stay-in-the-air/ https://www.scientificamerican.com/article/no-one-can-explai...
  - SonicScrub 4y ago
    Ugh, not this clickbait again. The actual title of that article should be "explanation of flight defies a simple intuitive explanation that is both wholly correct, and understandable to the layman". Very different then "no one can explain it!"
    - composter 4y ago
      I couldn't agree more! However, I think the content of the article is informative. If you strongly disagree I'm happy to remove the link to the article
      - SonicScrub 4y ago
        No, I do think it's an informative, well written article. It's quotations come directly from some the most well-known, and respected names in the field of the aerospace. I've even referenced it before on this very site! I just bemoan it's terrible title, and am perhaps bitter from the times when relatives send me links with similar titles (usually of lesser quality) and I have to explain that yes, people in my field do know what their doing. The infamous "bees" quote being a particular bug-a-boo. I would however critic it's use as an example to prove that we have a history of rationalizing explanations where none exist (and using that to draw a parallel with AI). While the title implies this conclusion, the article itself does not. We do indeed have a very good explanation of how aerodynamic lift works. That explanation just takes the form of a set of differential equations, and isn't something one can easily tell a group of 5th grader, without simplifying to the point of spreading errors.
ElFitz 4y ago
Except for things we decide to be important enough that we should create and apply models. Now, the decisions that went into these models might have been rationalised after the fact. Or biased. But these handmade models can been reviewed by others, the logic and decisions that went into them can be challenged, and rules can be changed or added based on experience. Not so much with LLMs.
xthrown1 4y ago
The difference is that I can ask it to explain it's decision as a racist and a anti-racist and it will happily hallucinate an answer for both.
dr_dshiv 4y ago
Another term is “confabulation” — which has also been used to explain human linguistic reasoning

pixl97 4y ago

Well no smart humans, but it turns out there are plenty of dumb ones.

delusional 4y ago

There's probably a lot of truth to this, but there's also a big difference. Humans are ultimately responsible and accountable for their actions. Should my actions turn out to carry a racist or sexist motive, even unbeknownst to me, i can be held to answer for that. That's a big difference. The rationalization from a human is valuable because it's delivered by the accountable party. From a machine such rationalization is at best worthless, since you can't hold the machine accountable at all.

jvanderbot 4y ago
But what youre saying only layers over the fact that, to avoid getting caught and convicted, a sufficiently good explanation is all that's necessary.
- delusional 4y ago
  Not quite. What I'm saying is that the explanation is valuable because it comes from the accountable party. That you have a good explanation does not absolve you of responsibility. The short and clear way to put it is this: You don't go to jail because YOU think you did something wrong, you go to jail because WE think you did something wrong. If I'm a customer at a bank, and my loan has been denied, I don't care what some unaccountable AI system can come up with to explain that. I care what about how the accountable bankers justify putting that AI system into the process in the first place. How do they justify that AI system getting to make decisions that affect me and my life. I don't care about why the process does what it does, I care about why that is the process.
  - coldtea 4y ago
    >If I'm a customer at a bank, and my loan has been denied, I don't care what some unaccountable AI system can come up with to explain that. I care what about how the accountable bankers justify putting that AI system into the process in the first place. How do they justify that AI system getting to make decisions that affect me and my life. I don't care about why the process does what it does, I care about why that is the process. Well, a court/law just has to declare "AI" as allowed to be used in such decisions, and the whole recourse you describe vanishes though...
  - chii 4y ago
    > my loan has been denied, I don't care what some unaccountable AI system can come up with to explain that. > How do they justify that AI system getting to make decisions that affect me and my life. so you, a priori, make the assumption that your loan _should've_ been accepted? If the decision wasn't an AI, but some actuarial that calculates and computes based on a set of criteria, and the result is a denial, you could still make the same argument of "why is _this_ the process, instead of something else (that makes my loan acceptable)?".
  - robertlagrant 4y ago
    > I care what about how the accountable bankers justify putting that AI system into the process in the first place. Computers are already deciding. As to why: it's because it's their money they're lending.
    - delusional 4y ago
      I work in a bank and you're laughably wrong. Computers are at best assisting in the lending process but more realistically they're completely ignored. There is NO AI (at least recognizably, you could argue that there's some sort of very simple expert model embedded in the 10 credit limits) involved in the process at all. Out lending process is based on the judgment of a few specific individuals, with more involved clients requiring approval from more senior people. All steps of that process can be overturned by the overseeing person, and that person is accountable for their decision.
onlyrealcuzzo 4y ago
More importantly, to the best of our knowledge, human thinking is not simply predicting the next word based on a past model.
- fsckboy 4y ago
  but the fact that predicting the next words based on past models does such a good job of masquerading as human thinking indicates that much of human thinking isn't much more than that. It may even simplify the task of figuring human thinking by narrowing down the last details of of the smaller set of missing pieces. an analagous result was obtained back when they mapped the small finite number of neurons in a snail brain, or the behavior of individuals in ant colonies. What looks like complex behavior turns out to be very simple under the hood. for the vast ocean of the population who... not sure how to describe them... not good students when in school, would rather spend the bulk of their time with the TV blaring, eating cheetos and swiping on tik-tok, following the lives of celebrities and fighting about it, rather than do anything long term productive with their own lives... chat gpt may have already exceeded what they do with their cranial talents. even a level up on the ladder, the types of office situations lampooned in The Office or Dilbert, are they doing much more as a percentage of time spent than chat GPT can do? "Mondays, amirite!?" then the question becomes, are the intellectual elites among us doing that much more, or just doing much more of the same thing? I think a large portion of what we do is exactly what chap GPT does. The question is what is this other piece of our brains' that intervenes to say "hmm, need to think about this part a lot harder"
  - kortilla 4y ago
    >such a good job of masquerading as human thinking indicates that much of human thinking isn't much more than that. No, that doesn’t follow. It just means it roughly looks like human thinking. Your comment is akin to saying a high resolution photo of a human has basically figured out a way to replicate humans. It looks like it in one aspect but it’s laughably wrong. Humans thought without language.
    - fsckboy 4y ago
wilg 4y ago
We don't know that it's true that humans are "responsible" or "accountable" for their actions in any meaningful way. This smells like a "free will" argument. IMHO, it's unlikely "free will" and "responsibility" are anything more than an illusion.
- WA 4y ago
  It really doesn’t matter for this discussion, because society and laws are structured as if people have free will. An AI has to live up to that as well.
  - wilg 4y ago
    Maybe I'm confused about the topic, then. What does "living up to that" mean here? Legal liability or something?
    - tsimionescu 4y ago
      Yes, that was their point. Humans have accountability to one another - either legally or in less formal ways (such as being fired if you're making your co-workers uncomfortable). Current machines simply don't have that kind of accountability. Even if we wanted to, we can't punish or ostracize ChatGPT when it lies to us, or makes us uncomfortable.
      - vixen99 4y ago
        Their use & application can be made accountable.
        tsimionescu 4y ago
        The point of this thread is that the explanation a human gives carries some weight because the human know they may/will be held accountable for the truth of that explanation. ChatGPT's explanations carry no such weight, since ChatGPT itself has no concept of suffering consequences for its actions. So, while both humans and ChatGPT can and do give bogus explanations for their actions, there are reasons to trust the humans' explanations more than ChatGPT's. Whether or not we hold humans using ChatGPT accountable for their use of it is irrelevant to this thread.
smeagull 4y ago
The only intent taking place here, is someone placing a process under automatic control. That's it. If I make a bot that posts random comments and it offends you, then I'm a bit responsible, but also you've gotten offended by a random comment.

geraneum 4y ago

Although humans are capable of rationalizing and they do it, they have other capabilities which is a key point that is missing in this argument. Maybe no one will admit to refusing a loan based on applicant’s gender, but also no real world aircraft engineer will explain why they decided to design a plane’s wing in a certain shape, purely by rationalizing an intuition without backing it by math and physics. Also, there are a group of humans elsewhere that understand those math and using the “same” principles can follow the explanation and detect mistakes or baseless rationalized explanations.

composter 4y ago
Using airfoil design rationalization as an example may not be the best choice in this context since there is a fairly long history of incorrect rationalizations for wing lift. "Incorrect Lift Theories": https://www.grc.nasa.gov/www/k-12/VirtualAero/BottleRocket/airplane/wrong1.html https://www.grc.nasa.gov/www/k-12/VirtualAero/BottleRocket/a... "No One Can Explain Why Planes Stay in the Air": https://www.scientificamerican.com/article/no-one-can-explain-why-planes-stay-in-the-air/ https://www.scientificamerican.com/article/no-one-can-explai...
- SonicScrub 4y ago
  Ugh, not this clickbait again. The actual title of that article should be "explanation of flight defies a simple intuitive explanation that is both wholly correct, and understandable to the layman". Very different then "no one can explain it!"
  - composter 4y ago
    I couldn't agree more! However, I think the content of the article is informative. If you strongly disagree I'm happy to remove the link to the article
    - SonicScrub 4y ago
      No, I do think it's an informative, well written article. It's quotations come directly from some the most well-known, and respected names in the field of the aerospace. I've even referenced it before on this very site! I just bemoan it's terrible title, and am perhaps bitter from the times when relatives send me links with similar titles (usually of lesser quality) and I have to explain that yes, people in my field do know what their doing. The infamous "bees" quote being a particular bug-a-boo. I would however critic it's use as an example to prove that we have a history of rationalizing explanations where none exist (and using that to draw a parallel with AI). While the title implies this conclusion, the article itself does not. We do indeed have a very good explanation of how aerodynamic lift works. That explanation just takes the form of a set of differential equations, and isn't something one can easily tell a group of 5th grader, without simplifying to the point of spreading errors.

ElFitz 4y ago

Except for things we decide to be important enough that we should create and apply models. Now, the decisions that went into these models might have been rationalised after the fact. Or biased. But these handmade models can been reviewed by others, the logic and decisions that went into them can be challenged, and rules can be changed or added based on experience. Not so much with LLMs.

xthrown1 4y ago

The difference is that I can ask it to explain it's decision as a racist and a anti-racist and it will happily hallucinate an answer for both.

dr_dshiv 4y ago

Another term is “confabulation” — which has also been used to explain human linguistic reasoning