4 ms·

> If we allow decisions and conclusions being drawn by the AI without a clear explanation of how it got there The funny thing is expert systems were doing that

> If we allow decisions and conclusions being drawn by the AI without a clear explanation of how it got there The funny thing is expert systems were doing that 30 years ago. Seems like this would be a perfect use of them but it's not a buzzword any more. It's crazy how many wheels get poorly reinvented in the software world.

soraki_soladead 8y ago
Explainable isn't the same as justifiable. An expert system may be able to readily explain that it determined the chance of recidivism is high due to factors x, y, and z. However, there is no guarantee that those factors correlate with recidivism at all since expert systems were simply built to emulate (human) expert decisions. While it's very easy to do machine learning _incorrectly_ it is also possible to reasonably attribute factors to outcomes based on large quantities of data. You can also look a LIME/SHAP and other factor contribution metrics. This seems like a significant improvement over expert systems.
- jlarocco 8y ago
  > Explainable isn't the same as justifiable. An expert system may be able to readily explain that it determined the chance of recidivism is high due to factors x, y, and z. However, there is no guarantee that those factors correlate with recidivism at all since expert systems were simply built to emulate (human) expert decisions. Re-reading the article now, it may very well be an expert system. The "big data" mentioned isn't training data, but pre-existing databases like bank account info, salaries, contracts, property ownership, etc. FTA: > Disciplinary officials need to help scientists train the machine with their experience and knowledge accumulated from previous cases. For instance, disciplinary officials spent many hours manually tagging unusual phenomenon in various types of data sets to teach the machine what to look for. As far as explaining the results, it's hard to draw any conclusion without knowing exactly what "not very good at explaining the process it has gone through " means.

soraki_soladead 8y ago

Explainable isn't the same as justifiable. An expert system may be able to readily explain that it determined the chance of recidivism is high due to factors x, y, and z. However, there is no guarantee that those factors correlate with recidivism at all since expert systems were simply built to emulate (human) expert decisions. While it's very easy to do machine learning _incorrectly_ it is also possible to reasonably attribute factors to outcomes based on large quantities of data. You can also look a LIME/SHAP and other factor contribution metrics. This seems like a significant improvement over expert systems.

jlarocco 8y ago
> Explainable isn't the same as justifiable. An expert system may be able to readily explain that it determined the chance of recidivism is high due to factors x, y, and z. However, there is no guarantee that those factors correlate with recidivism at all since expert systems were simply built to emulate (human) expert decisions. Re-reading the article now, it may very well be an expert system. The "big data" mentioned isn't training data, but pre-existing databases like bank account info, salaries, contracts, property ownership, etc. FTA: > Disciplinary officials need to help scientists train the machine with their experience and knowledge accumulated from previous cases. For instance, disciplinary officials spent many hours manually tagging unusual phenomenon in various types of data sets to teach the machine what to look for. As far as explaining the results, it's hard to draw any conclusion without knowing exactly what "not very good at explaining the process it has gone through " means.