5 ms·

You can have ex-post narratives for what humans do, but that is not the same as an actual explanation. Actual human cognition is a black box as of now. The issu

You can have ex-post narratives for what humans do, but that is not the same as an actual explanation. Actual human cognition is a black box as of now. The issue of appeals processes is totally separate. That doesn't mean we handover everything to AI, but the idea that we understand human decisions cannot be the reason for that. For some high risk things like controlling certain complex machinery or processes AI might indeed be needed because control is beyond human understanding (other than in the abstract).

ryanklee 3y ago
That's a bit of an exaggeration. We understand high level motivations as being highly correlated to certain outcomes. Hunger, eat. Horny, sex. Wronged, vengeance. Money, directed. You don't necessarily need to know the underlying processes to make reliable predictions or valid causal explanations about human behavior.
- RandomLensman 3y ago
  There isn't even agreement on the existence of free will. For non-trivial decisions, let's say some judge's deliberation, to claim it is fully explainable is a stretch.
  - ryanklee 3y ago
    I didn't say it was fully explainable. And you don't need to settle debates about free will to offer valid causal explanations for human behaviors. Are you saying we don't know at all why anyone does anything they ever do? That every action is totally unpredictable and after the fact that there is no way for us to offer more or less plausible explanations?
    - RandomLensman 3y ago
      We are getting off track in our discussion. For non-trivial decisions we know we can have ex-post justifications from humans but at the same time we know that those aren't necessarily complete or even true at times: A credit officer not liking someone will have a justification for refusal that does not include that and we might never know - from the outside - the true explanation and that credit officer might not even know their biases themselves! AI being explainable is just going beyond what we demand of humans (we demand a justification, not a real explanation). Also, predictability isn't the same as understanding the full decision process.
      - ryanklee 3y ago
        I don't think we're off track at all. You made the claim that human cognition is a black box. That's not true. We have valid causal explanations for human cognition /and/ human behavior. Just because we don't have explanations at every level of abstraction does not prevent us from having them at some levels of abstraction. We very well may find ourselves in the same situation with regard to AI. It's not going beyond, it would be achieving parity.
        RandomLensman 3y ago
        I disagree: for anything beyond basal stuff, we have hypotheses for causal explanations of human cognition, not full theories. For example, we cannot explain the mental process how someone came up with a new chess move and check the validity in that case. We can have some ideas how it might happen and that person might also have some ideas how, but then we back to hypotheses.
        ryanklee 3y ago
        Once again, you are raising the bar artificially high by pointing to examples where we fail to have reasonable casual accounts. But you are conveniently ignoring the mountain of reasonable accounts that we employ all day long simply to navigate our human world.
        RandomLensman 3y ago
        I think this is confusing a certain predictability with having a correct explanation (again, we are beyond basal things like being hungry leading to eating). Those two things are not the same.
        troupo 3y ago
        All this is demagoguery. If a bank denies you credit, it has to explain why. Not "AI told us so". If police arrests you, they have to explain why, not "AI told us so". If your job fires you, it has to explain why, not "AI told us so". etc.
troupo 3y ago
> You can have ex-post narratives for what humans did, but that is not the same as an actual explanation. Actual human cognition is a black box as of now. The issue of appeals processes is totally separate. Demagoguery. Already right now you have police accusing innocent people because some ML system told them so, and China running the world' largest social credit system based on nothing. > For some high risk things like controlling certain complex machinery or processes AI might indeed be needed because control is beyond human understanding. Ash. I should've been more clear. That's not the high-risk systems the EU AI Act talks about. I will not re-type everything, but this article has a good overview: https://softwarecrisis.dev/letters/the-truth-about-the-eu-act/ https://softwarecrisis.dev/letters/the-truth-about-the-eu-ac...
- RandomLensman 3y ago
  You want a correct explanation, not some made up ex-post stuff. Incorrect. High-risk systems indeed include certain machinery and processes, for example, in transport or in medical devices as well as operating critical infrastructure (https://www.europarl.europa.eu/news/en/headlines/society/20230601STO93804/eu-ai-act-first-regulation-on-artificial-intelligence https://www.europarl.europa.eu/news/en/headlines/society/202...).
  - troupo 3y ago
    > You want a correct explanation, not some made up ex-post stuff. Yes, and yes. What's so difficult to understand about that?

ryanklee 3y ago

That's a bit of an exaggeration. We understand high level motivations as being highly correlated to certain outcomes. Hunger, eat. Horny, sex. Wronged, vengeance. Money, directed. You don't necessarily need to know the underlying processes to make reliable predictions or valid causal explanations about human behavior.

RandomLensman 3y ago
There isn't even agreement on the existence of free will. For non-trivial decisions, let's say some judge's deliberation, to claim it is fully explainable is a stretch.
- ryanklee 3y ago
  I didn't say it was fully explainable. And you don't need to settle debates about free will to offer valid causal explanations for human behaviors. Are you saying we don't know at all why anyone does anything they ever do? That every action is totally unpredictable and after the fact that there is no way for us to offer more or less plausible explanations?
  - RandomLensman 3y ago
    We are getting off track in our discussion. For non-trivial decisions we know we can have ex-post justifications from humans but at the same time we know that those aren't necessarily complete or even true at times: A credit officer not liking someone will have a justification for refusal that does not include that and we might never know - from the outside - the true explanation and that credit officer might not even know their biases themselves! AI being explainable is just going beyond what we demand of humans (we demand a justification, not a real explanation). Also, predictability isn't the same as understanding the full decision process.
    - ryanklee 3y ago
      I don't think we're off track at all. You made the claim that human cognition is a black box. That's not true. We have valid causal explanations for human cognition /and/ human behavior. Just because we don't have explanations at every level of abstraction does not prevent us from having them at some levels of abstraction. We very well may find ourselves in the same situation with regard to AI. It's not going beyond, it would be achieving parity.
      - RandomLensman 3y ago
        I disagree: for anything beyond basal stuff, we have hypotheses for causal explanations of human cognition, not full theories. For example, we cannot explain the mental process how someone came up with a new chess move and check the validity in that case. We can have some ideas how it might happen and that person might also have some ideas how, but then we back to hypotheses.
        ryanklee 3y ago
        Once again, you are raising the bar artificially high by pointing to examples where we fail to have reasonable casual accounts. But you are conveniently ignoring the mountain of reasonable accounts that we employ all day long simply to navigate our human world.
        RandomLensman 3y ago
        I think this is confusing a certain predictability with having a correct explanation (again, we are beyond basal things like being hungry leading to eating). Those two things are not the same.
        troupo 3y ago
        All this is demagoguery. If a bank denies you credit, it has to explain why. Not "AI told us so". If police arrests you, they have to explain why, not "AI told us so". If your job fires you, it has to explain why, not "AI told us so". etc.

troupo 3y ago

> You can have ex-post narratives for what humans did, but that is not the same as an actual explanation. Actual human cognition is a black box as of now. The issue of appeals processes is totally separate. Demagoguery. Already right now you have police accusing innocent people because some ML system told them so, and China running the world' largest social credit system based on nothing. > For some high risk things like controlling certain complex machinery or processes AI might indeed be needed because control is beyond human understanding. Ash. I should've been more clear. That's not the high-risk systems the EU AI Act talks about. I will not re-type everything, but this article has a good overview: https://softwarecrisis.dev/letters/the-truth-about-the-eu-act/ https://softwarecrisis.dev/letters/the-truth-about-the-eu-ac...

RandomLensman 3y ago
You want a correct explanation, not some made up ex-post stuff. Incorrect. High-risk systems indeed include certain machinery and processes, for example, in transport or in medical devices as well as operating critical infrastructure (https://www.europarl.europa.eu/news/en/headlines/society/20230601STO93804/eu-ai-act-first-regulation-on-artificial-intelligence https://www.europarl.europa.eu/news/en/headlines/society/202...).
- troupo 3y ago
  > You want a correct explanation, not some made up ex-post stuff. Yes, and yes. What's so difficult to understand about that?