4 ms·

And people want to seek medical device from an AI that thinks eggs can melt…

bamboozled 3y ago
That sounds freaking delicious, melting eggs...
Manfred 3y ago
Who wants to do that? I assume you're talking about large language models? They are great at making word soup, but not great at diagnosis. There have been expert systems since the 90's that perform better on average than doctors when diagnosing illness in patients, and / or help doctors make better decisions.
- famouswaffles 3y ago
  >They are great at making word soup, but not great at diagnosis. No they are. >There have been expert systems since the 90's that perform better on average than doctors when diagnosing illness in patients, No there has not been. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10425828/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10425828/
  - mike741 3y ago
    66% accuracy is not "great" and definitely not the best there's ever been.
    - famouswaffles 3y ago
      If they aren't great then neither are doctors and the word becomes meaningless. Context is important. 66% accuracy on cases that stumped doctors and took extensive testing is not trivial. It's not the same as 66% accuracy for everyday diagnosis.
      - mike741 3y ago
        The study references "clinicians" rather than "doctors." Clinicians include psychologists, pharmacists, nurses, physicians, paramedics, etc.
        famouswaffles 3y ago
        Point still stands. Expert system diagnostic tools aren't that great.
LeifCarrotson 3y ago
And they want it to perform proofreading. And to generate copy for articles and blog posts and job listings and email responses. And to respond to tech support tickets. And to operate a phone tree. And, heaven help us, to write software, and - oh God no - to drive cars. The problem is that people are actually finding it useful for all of those things! I, too, would never have drawn up a functional specification for a product release which said the product might hallucinate and might be parrot incorrect facts about eggs melting if you phrase the question wrong[edit: if you insert an incorrect premise in the question], or might drive into a bus if it had a mural printed on the side. I'd have required correct performance to a lot more 9s before releasing the thing to the public than the products which have been released to the public demonstrate, or at least required a confidence metric that would allow the product to respond with an error message and escalate to a human if it thought it might fail (for tech support, not for self-driving cars, obviously). And yet here we are. Turns out people and corporations do find utility in hallucination-prone, often-wrong, easily misled, easily confused LLM chatbots. And they'll strap a weight to the wheel of their Tesla so they can stare at their phone while it drives them on Autopilot.
- elicksaur 3y ago
  Why does everyone frame the problem as “if you phrase the question wrong”? This example is a simple question. Any human who understands English understands the meaning of the question here. The fact that it gets it wrong with this phrasing indicates the model doesn’t understand the world. We shouldn’t blame users for asking questions “wrong”.
  - pipeline_peak 3y ago
    He’s not saying it’s the users fault for how they phrase the question. He’s saying the bot is limited in how it interprets questions.
- wredue 3y ago
  I’ve had people respond to my posts with “yeah well ChatGPT says …”. For starters, I don’t give a shit what ChatGPT has to say about various programming topics. Second, it makes sense that ChatGPT would get the facts of immutability utterly, childishly incorrect because fad oriented development has decided that claims absent evidence is the way to write software.

That sounds freaking delicious, melting eggs...

Who wants to do that? I assume you're talking about large language models? They are great at making word soup, but not great at diagnosis. There have been expert systems since the 90's that perform better on average than doctors when diagnosing illness in patients, and / or help doctors make better decisions.

famouswaffles 3y ago
>They are great at making word soup, but not great at diagnosis. No they are. >There have been expert systems since the 90's that perform better on average than doctors when diagnosing illness in patients, No there has not been. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10425828/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10425828/
- mike741 3y ago
  66% accuracy is not "great" and definitely not the best there's ever been.
  - famouswaffles 3y ago
    If they aren't great then neither are doctors and the word becomes meaningless. Context is important. 66% accuracy on cases that stumped doctors and took extensive testing is not trivial. It's not the same as 66% accuracy for everyday diagnosis.
    - mike741 3y ago
      The study references "clinicians" rather than "doctors." Clinicians include psychologists, pharmacists, nurses, physicians, paramedics, etc.
      - famouswaffles 3y ago
        Point still stands. Expert system diagnostic tools aren't that great.

LeifCarrotson 3y ago

And they want it to perform proofreading. And to generate copy for articles and blog posts and job listings and email responses. And to respond to tech support tickets. And to operate a phone tree. And, heaven help us, to write software, and - oh God no - to drive cars. The problem is that people are actually finding it useful for all of those things! I, too, would never have drawn up a functional specification for a product release which said the product might hallucinate and might be parrot incorrect facts about eggs melting if you phrase the question wrong[edit: if you insert an incorrect premise in the question], or might drive into a bus if it had a mural printed on the side. I'd have required correct performance to a lot more 9s before releasing the thing to the public than the products which have been released to the public demonstrate, or at least required a confidence metric that would allow the product to respond with an error message and escalate to a human if it thought it might fail (for tech support, not for self-driving cars, obviously). And yet here we are. Turns out people and corporations do find utility in hallucination-prone, often-wrong, easily misled, easily confused LLM chatbots. And they'll strap a weight to the wheel of their Tesla so they can stare at their phone while it drives them on Autopilot.

elicksaur 3y ago
Why does everyone frame the problem as “if you phrase the question wrong”? This example is a simple question. Any human who understands English understands the meaning of the question here. The fact that it gets it wrong with this phrasing indicates the model doesn’t understand the world. We shouldn’t blame users for asking questions “wrong”.
- pipeline_peak 3y ago
  He’s not saying it’s the users fault for how they phrase the question. He’s saying the bot is limited in how it interprets questions.
wredue 3y ago
I’ve had people respond to my posts with “yeah well ChatGPT says …”. For starters, I don’t give a shit what ChatGPT has to say about various programming topics. Second, it makes sense that ChatGPT would get the facts of immutability utterly, childishly incorrect because fad oriented development has decided that claims absent evidence is the way to write software.