3 ms·

Human explainability/interpretability is terrible -- it takes years to understand a person well enough to model their actions well in novel situations, and ther

Human explainability/interpretability is terrible -- it takes years to understand a person well enough to model their actions well in novel situations, and there's tons of literature that backs up that people consistently invent bullshit explanations when asked why they did things. Humans can only be held to account by telling them to change, which we can do with neural networks too.

rvz 3y ago
> Humans can only be held to account by telling them to change, which we can do with neural networks too. Even if humans invent bullshit explanations, in serious cases of accountability is done in the courts system which humans investigate about the whole timeline of events of a dispute which there is very little room for perverting the course of justice and making everything up. Hence this scenario, lawyers would liked to have known as to why did an AI system give hallucinated citations when it was used in a legal proceeding? It's even worse that legal experts knowingly trusted it and failed to reason with the results from this AI system; because fundamentally it cannot explain why that issue happened. [0] It even goes beyond basic citations, with autonomous cars without humans behind the wheel [1] with the company (Cruise) being unable to convince the regulators or even explain the crashes and had to pull the vehicles off the road due to this high amount of risk. So yet again, explainability in AI with neural networks is still far worse than humans, even when these systems cannot be trusted in high risk and novel situations. [0] https://www.theguardian.com/technology/2023/jun/23/two-us-lawyers-fined-submitting-fake-court-citations-chatgpt https://www.theguardian.com/technology/2023/jun/23/two-us-la... [1] https://www.theguardian.com/us-news/2023/oct/24/driverless-car-self-driving-california-cruise-gm https://www.theguardian.com/us-news/2023/oct/24/driverless-c...

rvz 3y ago

> Humans can only be held to account by telling them to change, which we can do with neural networks too. Even if humans invent bullshit explanations, in serious cases of accountability is done in the courts system which humans investigate about the whole timeline of events of a dispute which there is very little room for perverting the course of justice and making everything up. Hence this scenario, lawyers would liked to have known as to why did an AI system give hallucinated citations when it was used in a legal proceeding? It's even worse that legal experts knowingly trusted it and failed to reason with the results from this AI system; because fundamentally it cannot explain why that issue happened. [0] It even goes beyond basic citations, with autonomous cars without humans behind the wheel [1] with the company (Cruise) being unable to convince the regulators or even explain the crashes and had to pull the vehicles off the road due to this high amount of risk. So yet again, explainability in AI with neural networks is still far worse than humans, even when these systems cannot be trusted in high risk and novel situations. [0] https://www.theguardian.com/technology/2023/jun/23/two-us-lawyers-fined-submitting-fake-court-citations-chatgpt https://www.theguardian.com/technology/2023/jun/23/two-us-la... [1] https://www.theguardian.com/us-news/2023/oct/24/driverless-car-self-driving-california-cruise-gm https://www.theguardian.com/us-news/2023/oct/24/driverless-c...