4 ms·

Watson should rebrand then. You can't have a colossal failure such as the one reported by Stat News: “multiple examples of unsafe and incorrect treatment recomm

Watson should rebrand then. You can't have a colossal failure such as the one reported by Stat News: “multiple examples of unsafe and incorrect treatment recommendations.” — [Memorial Sloan Kettering Cancer Center] and expect to move on from it as if they were growing pains.

ethbro 8y ago
I call this the Netflix ML effect. If your answer is mission critical, probabilistic ML isn't advanced, explainable, or reliable enough for your problem. If your answer is of the great-to-be-right, meh-to-be-wrong sort (e.g. ranking movie recommendations), then you can and should go nuts with ML. And if someone really wants to do an ML project on the former, do everything you can to transform it into the latter.
- djakjxnanjak 8y ago
  If you can tune the specifics-sensitivity curve, you should be able to handle both cases. You have to be willing to refuse to provide an answer when you have low confidence.
  - AlotOfReading 8y ago
    The existence of adversarial attacks with high confidence on virtually all production ML systems should indicate that confidence numbers are not enough to rely on.
    - djakjxnanjak 8y ago
      Do these attacks require fine control over the input? Eg. if you are scanning a patient’s body, does it matter if the model can be fooled by editing the values of individual pixels? This implies that you have a threat model where the data coming from the sensor is being manipulated, in which case all bets are off (the image could be entirely replaced). It doesn’t seem much different from a statistical model that you can blow up by feeding in values designed to cause a divide by zero (values that wouldn’t appear in real-world data). It seems like a problem when classifying user-provided images (eg. identifying obscene images on a social network) but not so relevant when you own the sensors.
      - govg 8y ago
        You do have a point - all the adversarial attacks on ML models rely on full adversarial control on the inputs, which probably isn't the case with medical records. If there was unconstrained access to a patient's MRI scans by an adversary, then I don't think adversarial attacks on the ML diagnostic models are the biggest problems you'll face.
  - ethbro 8y ago
    Absolutely. Which I'd classify as a great way to punt. But some projects as presented don't have a clear refusal option. ML would go a long way if the Hippocratic Oath (or derivative thereof) were taken & adhered to by the industry.

ethbro 8y ago

I call this the Netflix ML effect. If your answer is mission critical, probabilistic ML isn't advanced, explainable, or reliable enough for your problem. If your answer is of the great-to-be-right, meh-to-be-wrong sort (e.g. ranking movie recommendations), then you can and should go nuts with ML. And if someone really wants to do an ML project on the former, do everything you can to transform it into the latter.

djakjxnanjak 8y ago
If you can tune the specifics-sensitivity curve, you should be able to handle both cases. You have to be willing to refuse to provide an answer when you have low confidence.
- AlotOfReading 8y ago
  The existence of adversarial attacks with high confidence on virtually all production ML systems should indicate that confidence numbers are not enough to rely on.
  - djakjxnanjak 8y ago
    Do these attacks require fine control over the input? Eg. if you are scanning a patient’s body, does it matter if the model can be fooled by editing the values of individual pixels? This implies that you have a threat model where the data coming from the sensor is being manipulated, in which case all bets are off (the image could be entirely replaced). It doesn’t seem much different from a statistical model that you can blow up by feeding in values designed to cause a divide by zero (values that wouldn’t appear in real-world data). It seems like a problem when classifying user-provided images (eg. identifying obscene images on a social network) but not so relevant when you own the sensors.
    - govg 8y ago
      You do have a point - all the adversarial attacks on ML models rely on full adversarial control on the inputs, which probably isn't the case with medical records. If there was unconstrained access to a patient's MRI scans by an adversary, then I don't think adversarial attacks on the ML diagnostic models are the biggest problems you'll face.
- ethbro 8y ago
  Absolutely. Which I'd classify as a great way to punt. But some projects as presented don't have a clear refusal option. ML would go a long way if the Hippocratic Oath (or derivative thereof) were taken & adhered to by the industry.