4 ms·

AI not being allowed in high-risk applications is a bad idea, same for explicable in human terms (cannot do that now for human decisions anyway). The latter wou

AI not being allowed in high-risk applications is a bad idea, same for explicable in human terms (cannot do that now for human decisions anyway). The latter would kind of rule out any quantum computing, too, as the only real explanation there is through maths, not words.

troupo 3y ago
> AI not being allowed in high-risk applications is a bad idea wat > same for explicable in human terms (cannot do that now for human decisions anyway). What? Human decisions can be explained and appealed. Good luck proving something when a blackbox finds you guilty of anything.
- RandomLensman 3y ago
  You can have ex-post narratives for what humans do, but that is not the same as an actual explanation. Actual human cognition is a black box as of now. The issue of appeals processes is totally separate. That doesn't mean we handover everything to AI, but the idea that we understand human decisions cannot be the reason for that. For some high risk things like controlling certain complex machinery or processes AI might indeed be needed because control is beyond human understanding (other than in the abstract).
  - ryanklee 3y ago
    That's a bit of an exaggeration. We understand high level motivations as being highly correlated to certain outcomes. Hunger, eat. Horny, sex. Wronged, vengeance. Money, directed. You don't necessarily need to know the underlying processes to make reliable predictions or valid causal explanations about human behavior.
    - RandomLensman 3y ago
      There isn't even agreement on the existence of free will. For non-trivial decisions, let's say some judge's deliberation, to claim it is fully explainable is a stretch.
      - ryanklee 3y ago
        I didn't say it was fully explainable. And you don't need to settle debates about free will to offer valid causal explanations for human behaviors. Are you saying we don't know at all why anyone does anything they ever do? That every action is totally unpredictable and after the fact that there is no way for us to offer more or less plausible explanations?
        RandomLensman 3y ago
        We are getting off track in our discussion. For non-trivial decisions we know we can have ex-post justifications from humans but at the same time we know that those aren't necessarily complete or even true at times: A credit officer not liking someone will have a justification for refusal that does not include that and we might never know - from the outside - the true explanation and that credit officer might not even know their biases themselves! AI being explainable is just going beyond what we demand of humans (we demand a justification, not a real explanation). Also, predictability isn't the same as understanding the full decision process.
        ryanklee 3y ago
        I don't think we're off track at all. You made the claim that human cognition is a black box. That's not true. We have valid causal explanations for human cognition /and/ human behavior. Just because we don't have explanations at every level of abstraction does not prevent us from having them at some levels of abstraction. We very well may find ourselves in the same situation with regard to AI. It's not going beyond, it would be achieving parity.
        RandomLensman 3y ago
        I disagree: for anything beyond basal stuff, we have hypotheses for causal explanations of human cognition, not full theories. For example, we cannot explain the mental process how someone came up with a new chess move and check the validity in that case. We can have some ideas how it might happen and that person might also have some ideas how, but then we back to hypotheses.
  - troupo 3y ago
    > You can have ex-post narratives for what humans did, but that is not the same as an actual explanation. Actual human cognition is a black box as of now. The issue of appeals processes is totally separate. Demagoguery. Already right now you have police accusing innocent people because some ML system told them so, and China running the world' largest social credit system based on nothing. > For some high risk things like controlling certain complex machinery or processes AI might indeed be needed because control is beyond human understanding. Ash. I should've been more clear. That's not the high-risk systems the EU AI Act talks about. I will not re-type everything, but this article has a good overview: https://softwarecrisis.dev/letters/the-truth-about-the-eu-act/ https://softwarecrisis.dev/letters/the-truth-about-the-eu-ac...
    - RandomLensman 3y ago
      You want a correct explanation, not some made up ex-post stuff. Incorrect. High-risk systems indeed include certain machinery and processes, for example, in transport or in medical devices as well as operating critical infrastructure (https://www.europarl.europa.eu/news/en/headlines/society/20230601STO93804/eu-ai-act-first-regulation-on-artificial-intelligence https://www.europarl.europa.eu/news/en/headlines/society/202...).
      - troupo 3y ago
        > You want a correct explanation, not some made up ex-post stuff. Yes, and yes. What's so difficult to understand about that?
BlueTemplar 3y ago
The «high risk applications» mentioned here seem to be of social risk.
- RandomLensman 3y ago
  And AI fear cares no social risk? Some people are talking about internment, war, etc. to prevent certain forms of AI - not sure the right risks are in focus here.
Mordisquitos 3y ago
> The latter would kind of rule out any quantum computing, too, as the only real explanation there is through maths, not words. Maths are human terms.
- RandomLensman 3y ago
  If the level of explanation is something that the person suffering from the decision need not necessarily understand, fair enough. But then is the limit that one human on earth can understand it? They way I understand the thrust of the regulation that isn't the intention.
  - troupo 3y ago
    > But then is the limit that one human on earth can understand it? They way I understand the thrust of the regulation that isn't the intention. Yes, that is the intention when it comes to humans interacting with systems. https://news.ycombinator.com/edit?id=38113944 https://news.ycombinator.com/edit?id=38113944
    - RandomLensman 3y ago
      From my discussions with regulators, I would not necessarily subscribe to that view.

troupo 3y ago

> AI not being allowed in high-risk applications is a bad idea wat > same for explicable in human terms (cannot do that now for human decisions anyway). What? Human decisions can be explained and appealed. Good luck proving something when a blackbox finds you guilty of anything.

RandomLensman 3y ago
You can have ex-post narratives for what humans do, but that is not the same as an actual explanation. Actual human cognition is a black box as of now. The issue of appeals processes is totally separate. That doesn't mean we handover everything to AI, but the idea that we understand human decisions cannot be the reason for that. For some high risk things like controlling certain complex machinery or processes AI might indeed be needed because control is beyond human understanding (other than in the abstract).
- ryanklee 3y ago
  That's a bit of an exaggeration. We understand high level motivations as being highly correlated to certain outcomes. Hunger, eat. Horny, sex. Wronged, vengeance. Money, directed. You don't necessarily need to know the underlying processes to make reliable predictions or valid causal explanations about human behavior.
  - RandomLensman 3y ago
    There isn't even agreement on the existence of free will. For non-trivial decisions, let's say some judge's deliberation, to claim it is fully explainable is a stretch.
    - ryanklee 3y ago
      I didn't say it was fully explainable. And you don't need to settle debates about free will to offer valid causal explanations for human behaviors. Are you saying we don't know at all why anyone does anything they ever do? That every action is totally unpredictable and after the fact that there is no way for us to offer more or less plausible explanations?
      - RandomLensman 3y ago
        We are getting off track in our discussion. For non-trivial decisions we know we can have ex-post justifications from humans but at the same time we know that those aren't necessarily complete or even true at times: A credit officer not liking someone will have a justification for refusal that does not include that and we might never know - from the outside - the true explanation and that credit officer might not even know their biases themselves! AI being explainable is just going beyond what we demand of humans (we demand a justification, not a real explanation). Also, predictability isn't the same as understanding the full decision process.
        ryanklee 3y ago
        I don't think we're off track at all. You made the claim that human cognition is a black box. That's not true. We have valid causal explanations for human cognition /and/ human behavior. Just because we don't have explanations at every level of abstraction does not prevent us from having them at some levels of abstraction. We very well may find ourselves in the same situation with regard to AI. It's not going beyond, it would be achieving parity.
        RandomLensman 3y ago
        I disagree: for anything beyond basal stuff, we have hypotheses for causal explanations of human cognition, not full theories. For example, we cannot explain the mental process how someone came up with a new chess move and check the validity in that case. We can have some ideas how it might happen and that person might also have some ideas how, but then we back to hypotheses.
- troupo 3y ago
  > You can have ex-post narratives for what humans did, but that is not the same as an actual explanation. Actual human cognition is a black box as of now. The issue of appeals processes is totally separate. Demagoguery. Already right now you have police accusing innocent people because some ML system told them so, and China running the world' largest social credit system based on nothing. > For some high risk things like controlling certain complex machinery or processes AI might indeed be needed because control is beyond human understanding. Ash. I should've been more clear. That's not the high-risk systems the EU AI Act talks about. I will not re-type everything, but this article has a good overview: https://softwarecrisis.dev/letters/the-truth-about-the-eu-act/ https://softwarecrisis.dev/letters/the-truth-about-the-eu-ac...
  - RandomLensman 3y ago
    You want a correct explanation, not some made up ex-post stuff. Incorrect. High-risk systems indeed include certain machinery and processes, for example, in transport or in medical devices as well as operating critical infrastructure (https://www.europarl.europa.eu/news/en/headlines/society/20230601STO93804/eu-ai-act-first-regulation-on-artificial-intelligence https://www.europarl.europa.eu/news/en/headlines/society/202...).
    - troupo 3y ago
      > You want a correct explanation, not some made up ex-post stuff. Yes, and yes. What's so difficult to understand about that?

BlueTemplar 3y ago

The «high risk applications» mentioned here seem to be of social risk.

RandomLensman 3y ago
And AI fear cares no social risk? Some people are talking about internment, war, etc. to prevent certain forms of AI - not sure the right risks are in focus here.

Mordisquitos 3y ago

> The latter would kind of rule out any quantum computing, too, as the only real explanation there is through maths, not words. Maths are human terms.

RandomLensman 3y ago
If the level of explanation is something that the person suffering from the decision need not necessarily understand, fair enough. But then is the limit that one human on earth can understand it? They way I understand the thrust of the regulation that isn't the intention.
- troupo 3y ago
  > But then is the limit that one human on earth can understand it? They way I understand the thrust of the regulation that isn't the intention. Yes, that is the intention when it comes to humans interacting with systems. https://news.ycombinator.com/edit?id=38113944 https://news.ycombinator.com/edit?id=38113944
  - RandomLensman 3y ago
    From my discussions with regulators, I would not necessarily subscribe to that view.