3 ms·

I wouldn't be against some regulations that require safety critical AI models to be fully explainable. Or at least to some extent that "look what apparently Cha

I wouldn't be against some regulations that require safety critical AI models to be fully explainable. Or at least to some extent that "look what apparently ChatGPT can do" wouldn't qualify for.

AlotOfReading 3y ago
It's widely accepted by most people in the industry that this is in fact required by the existing functional safety standards, though they obviously weren't written with that intent originally.
osrec 3y ago
Absolutely. For me, the whole point of autonomous vehicles is the added safety, control and predictability they could bring. Relying on an unexplainable AI model leaves you open to unpredictable instabilities, which, for this particular use case, are unacceptable (at least to me).
danielmarkbruce 3y ago
Who would decide if something met the bar of "fully explainable" ? We have all kinds of drugs going into human bodies where the mechanism isn't fully explainable. Once you get a lot of variables, humans can't explain it in some nice concise fashion.
- lacrimacida 3y ago
  It’s a bit different when comparing the two You taking a drug won’t kill others but your autonomous vehicle could.
  - danielmarkbruce 3y ago
    Agree it's not exactly the same, but it's more similar than it seems on the surface. By and large, you don't get to make the decision for yourself wrt drugs. Doctors make recommendations. So, Doctors can kill others (lots and lots of them per doctor) by recommending/prescribing drugs that they can't fully explain. It is what it is. At some point with enough variables we lose the ability to explain something in a mechanistic fashion that a human can fit in their head. The nice thing about LLMs compared to drugs - an LLM is actually fully explainable. The explanation is just very long and boring. "and so we calculate this number and then that number, then adjust these numbers, and then calculate that thing and blah blah blah" for 6000 years.
  - blovescoffee 3y ago
    I recall a 60 minutes piece where people had taken too much sleeping pills the night before and the next morning were still so groggy that they got into accidents.

I wouldn't be against some regulations that require safety critical AI models to be fully explainable. Or at least to some extent that "look what apparently ChatGPT can do" wouldn't qualify for.

AlotOfReading 3y ago

It's widely accepted by most people in the industry that this is in fact required by the existing functional safety standards, though they obviously weren't written with that intent originally.

osrec 3y ago

Absolutely. For me, the whole point of autonomous vehicles is the added safety, control and predictability they could bring. Relying on an unexplainable AI model leaves you open to unpredictable instabilities, which, for this particular use case, are unacceptable (at least to me).

danielmarkbruce 3y ago

Who would decide if something met the bar of "fully explainable" ? We have all kinds of drugs going into human bodies where the mechanism isn't fully explainable. Once you get a lot of variables, humans can't explain it in some nice concise fashion.

lacrimacida 3y ago
It’s a bit different when comparing the two You taking a drug won’t kill others but your autonomous vehicle could.
- danielmarkbruce 3y ago
  Agree it's not exactly the same, but it's more similar than it seems on the surface. By and large, you don't get to make the decision for yourself wrt drugs. Doctors make recommendations. So, Doctors can kill others (lots and lots of them per doctor) by recommending/prescribing drugs that they can't fully explain. It is what it is. At some point with enough variables we lose the ability to explain something in a mechanistic fashion that a human can fit in their head. The nice thing about LLMs compared to drugs - an LLM is actually fully explainable. The explanation is just very long and boring. "and so we calculate this number and then that number, then adjust these numbers, and then calculate that thing and blah blah blah" for 6000 years.
- blovescoffee 3y ago
  I recall a 60 minutes piece where people had taken too much sleeping pills the night before and the next morning were still so groggy that they got into accidents.