5 ms·

If an app makes a diagnosis or a recommendation based on health data, that's Software as a Medical Device (SaMD) and it opens up a world of liability. https://

by myhf 9mo ago

If an app makes a diagnosis or a recommendation based on health data, that's Software as a Medical Device (SaMD) and it opens up a world of liability. https://www.fda.gov/medical-devices/digital-health-center-excellence/software-medical-device-samd https://www.fda.gov/medical-devices/digital-health-center-ex...

deleted 9mo ago
[deleted]
eastbound 9mo ago
How do you suggest to deal with Gemini? Extremely useful to understand whether something is worrying or not. Whether we like it or not, it’s a main participant to the discussion.
- jessetemp 9mo ago
  Ideally, hold Google liable until their AI doesn’t confabulate medical advice. Realistically, sign a EULA waiving your rights because their AI confabulates medical advice
- overfeed 9mo ago
  > How do you suggest to deal with Gemini? With robust fines based on % revenue whenever it breaks the law, would be my preference. I'm nit here to attempt solutions to Google's self-inflicted business-model challenges.
- deleted 9mo ago
  [deleted]
- arkh 9mo ago
  > How do you suggest to deal with Gemini? Don't. I do not ask my mechanic for medical advice, why would I ask a random output machine?
  - menaerus 9mo ago
    This "random output machine" is already in large use in medicine so why exactly not? Should I trust the young doctor fresh out of the Uni more by default or should I take advises from both of them with a grain of salt? I had failures and successes with both of them but lately I found Gemini to be extremely good at what it does.
    - thisislife2 9mo ago
      There's a difference between a doctor (an expert in their field) using AI (specialising in medicine) and you (a lay person) using it to diagnose and treat yourself. In the US, it takes at least 10 years of studying (and interning) to become a doctor.
      - vharish 9mo ago
        Even so, it's rather common for doctors to not be albe to diagonise correctly. It's a guessing game for them too. I don't know so much about US but it's a real problem in large parts of the world. As the comment stated, I would take anything a doctor says with a pinch of salt. Particularly so when the problem is not obvious.
        danaris 9mo ago
        These things are not equivalent. This is really not that far off from the argument that "well, people make mistakes a lot, too, so really, LLMs are just like people, and they're probably conscious too!" Yes, doctors make mistakes. Yes, some doctors make a lot of mistakes. Yes, some patients get misdiagnosed a bunch (because they have something unusual, or because they are a member of a group—like women, people of color, overweight people, or some combination—that American doctors have a tendency to disbelieve). None of that means that it's a good idea to replace those human doctors with LLMs that can make up brand-new diseases that don't exist occasionally.
      - schiffern 9mo ago
        Why stop at AI? By that same logic, we should ban non-doctors from being allowed to Google anything medical.
- derbOac 9mo ago
  Apparently we should hire the Guardian to evaluate LLM output accuracy? Why are these products being put out there for these kinds of things with no attempt to quantify accuracy? In many areas AI has become this toy that we use because it looks real enough. It sometimes works for some things in math and science because we test its output, but overall you don't go to Gemini and it says "there's a 80% chance this is correct". At least then you could evaluate that claim. There's a kind of task LLMs aren't well suited to because there's no intrinsic empirical verifiability, for lack of a better way of putting it.
  - lazide 9mo ago
    Because $$$
    - totetsu 9mo ago
      Because privatized $$$ and public !!!
- ndsipa_pomu 9mo ago
  If it's giving out medical advice without a license, it should be banned from giving medical advice and the parent company fined or forced to retire it.
- atoav 9mo ago
  As a certified electrical engineer, the amount of times googles LLM suggested a thing that would have at minimum started a fire is staggering. I have the capacity to know when it is wrong, but I teach this at university level. What worries me, are the people who are on the starting end of the Dunning-Kruger curve and needed that wrong advice to start "fixing" the spaces where this might become a danger to human life. No information is superior to wrong information presented in a convincing way.
- usefulposter 9mo ago
  No. "Whether we like it or not" is LLM inevitabilism. https://news.ycombinator.com/item?id=44567857 https://news.ycombinator.com/item?id=44567857
  - deleted 9mo ago
    [deleted]
  - protocolture 9mo ago
    Thought terminating cliche.
  - schiffern 9mo ago
    Yes. >Argument By Adding -ism To The End Of A Word Counterpoint: LLMs are inevitable. Can't put that genie back in the bottle, no matter how much the powers-that-be may wish. Such is the nature of (technological) genies. The only way to 'stop' LLMs is to invent something better.
    - deleted 9mo ago
      [deleted]
    - somewhereoutth 9mo ago
      depends if the cost of training can be recouped (with profit) from the proceeds of usage. Plenty of inventions prove themselves to be uneconomic.
    - deleted 9mo ago
      [deleted]

by myhf 9mo ago

deleted 9mo ago

[deleted]

eastbound 9mo ago

How do you suggest to deal with Gemini? Extremely useful to understand whether something is worrying or not. Whether we like it or not, it’s a main participant to the discussion.

jessetemp 9mo ago
Ideally, hold Google liable until their AI doesn’t confabulate medical advice. Realistically, sign a EULA waiving your rights because their AI confabulates medical advice
overfeed 9mo ago
> How do you suggest to deal with Gemini? With robust fines based on % revenue whenever it breaks the law, would be my preference. I'm nit here to attempt solutions to Google's self-inflicted business-model challenges.
deleted 9mo ago
[deleted]
arkh 9mo ago
> How do you suggest to deal with Gemini? Don't. I do not ask my mechanic for medical advice, why would I ask a random output machine?
- menaerus 9mo ago
  This "random output machine" is already in large use in medicine so why exactly not? Should I trust the young doctor fresh out of the Uni more by default or should I take advises from both of them with a grain of salt? I had failures and successes with both of them but lately I found Gemini to be extremely good at what it does.
  - thisislife2 9mo ago
    There's a difference between a doctor (an expert in their field) using AI (specialising in medicine) and you (a lay person) using it to diagnose and treat yourself. In the US, it takes at least 10 years of studying (and interning) to become a doctor.
    - vharish 9mo ago
      Even so, it's rather common for doctors to not be albe to diagonise correctly. It's a guessing game for them too. I don't know so much about US but it's a real problem in large parts of the world. As the comment stated, I would take anything a doctor says with a pinch of salt. Particularly so when the problem is not obvious.
      - danaris 9mo ago
        These things are not equivalent. This is really not that far off from the argument that "well, people make mistakes a lot, too, so really, LLMs are just like people, and they're probably conscious too!" Yes, doctors make mistakes. Yes, some doctors make a lot of mistakes. Yes, some patients get misdiagnosed a bunch (because they have something unusual, or because they are a member of a group—like women, people of color, overweight people, or some combination—that American doctors have a tendency to disbelieve). None of that means that it's a good idea to replace those human doctors with LLMs that can make up brand-new diseases that don't exist occasionally.
    - schiffern 9mo ago
      Why stop at AI? By that same logic, we should ban non-doctors from being allowed to Google anything medical.
derbOac 9mo ago
Apparently we should hire the Guardian to evaluate LLM output accuracy? Why are these products being put out there for these kinds of things with no attempt to quantify accuracy? In many areas AI has become this toy that we use because it looks real enough. It sometimes works for some things in math and science because we test its output, but overall you don't go to Gemini and it says "there's a 80% chance this is correct". At least then you could evaluate that claim. There's a kind of task LLMs aren't well suited to because there's no intrinsic empirical verifiability, for lack of a better way of putting it.
- lazide 9mo ago
  Because $$$
  - totetsu 9mo ago
    Because privatized $$$ and public !!!
ndsipa_pomu 9mo ago
If it's giving out medical advice without a license, it should be banned from giving medical advice and the parent company fined or forced to retire it.
atoav 9mo ago
As a certified electrical engineer, the amount of times googles LLM suggested a thing that would have at minimum started a fire is staggering. I have the capacity to know when it is wrong, but I teach this at university level. What worries me, are the people who are on the starting end of the Dunning-Kruger curve and needed that wrong advice to start "fixing" the spaces where this might become a danger to human life. No information is superior to wrong information presented in a convincing way.
usefulposter 9mo ago
No. "Whether we like it or not" is LLM inevitabilism. https://news.ycombinator.com/item?id=44567857 https://news.ycombinator.com/item?id=44567857
- deleted 9mo ago
  [deleted]
- protocolture 9mo ago
  Thought terminating cliche.
- schiffern 9mo ago
  Yes. >Argument By Adding -ism To The End Of A Word Counterpoint: LLMs are inevitable. Can't put that genie back in the bottle, no matter how much the powers-that-be may wish. Such is the nature of (technological) genies. The only way to 'stop' LLMs is to invent something better.
  - deleted 9mo ago
    [deleted]
  - somewhereoutth 9mo ago
    depends if the cost of training can be recouped (with profit) from the proceeds of usage. Plenty of inventions prove themselves to be uneconomic.
  - deleted 9mo ago
    [deleted]