3 ms·

But why should I care? If you demonstrated that a model can perform more accurate diagnoses than a doctor, but also it had this strange behavior when no image

But why should I care? If you demonstrated that a model can perform more accurate diagnoses than a doctor, but also it had this strange behavior when no image was presented, why should that deter me from using the model?

swiftcoder 4mo ago
Because you don’t have any way of telling if it actually used the image presented, or based it’s conclusions on a different image it made up
- simianwords 4mo ago
  Really? You know you could just ask it.
  - swiftcoder 4mo ago
    Which would tell you what, exactly? The whole root of the problem is that the model doesn’t “know” either
    - simianwords 4mo ago
      This is untrue and probably shows lack of experience with using LLMs. In my experience, each time I get some hallucination, I can ask the llm whether it hallucinated or not and I get a correct response.
      - swiftcoder 4mo ago
        > each time I get some hallucination, I can ask the llm whether it hallucinated or not and I get a correct response. You get a hallucination of a correct response, yes, and given that it's a yes or no question, this hallucination is more likely to be correct than the response to the original more complicated question. But make no mistake that it operates under the exact same constraints
- consensus1 4mo ago
  I don't find that persuasive. This is not the error I worry about. Let's say that hypothetically the model just ignores the input image 1 in 10,000 runs. This really doesn't concern me because the output will be trivially detectable incorrect nonsense that doesn't match the symptoms at all. Such a contingency is easily handled by running the image through multiple models and distilling the output, anyway. The error I worry about is where the model uses the image and comes to an incorrect but symptom matching diagnosis. But in this hypothetical the model is less likely to do so than a doctor, so the choice is either accept the risk of the model or accept a higher risk from a doctor.

swiftcoder 4mo ago

Because you don’t have any way of telling if it actually used the image presented, or based it’s conclusions on a different image it made up

simianwords 4mo ago
Really? You know you could just ask it.
- swiftcoder 4mo ago
  Which would tell you what, exactly? The whole root of the problem is that the model doesn’t “know” either
  - simianwords 4mo ago
    This is untrue and probably shows lack of experience with using LLMs. In my experience, each time I get some hallucination, I can ask the llm whether it hallucinated or not and I get a correct response.
    - swiftcoder 4mo ago
      > each time I get some hallucination, I can ask the llm whether it hallucinated or not and I get a correct response. You get a hallucination of a correct response, yes, and given that it's a yes or no question, this hallucination is more likely to be correct than the response to the original more complicated question. But make no mistake that it operates under the exact same constraints
consensus1 4mo ago
I don't find that persuasive. This is not the error I worry about. Let's say that hypothetically the model just ignores the input image 1 in 10,000 runs. This really doesn't concern me because the output will be trivially detectable incorrect nonsense that doesn't match the symptoms at all. Such a contingency is easily handled by running the image through multiple models and distilling the output, anyway. The error I worry about is where the model uses the image and comes to an incorrect but symptom matching diagnosis. But in this hypothetical the model is less likely to do so than a doctor, so the choice is either accept the risk of the model or accept a higher risk from a doctor.