4 ms·

> no one else seems to find this to be quite damning for the AI services being offered, preferring instanced to be wowed by the convenience and speed at which t

> no one else seems to find this to be quite damning for the AI services being offered, preferring instanced to be wowed by the convenience and speed at which they can be delivered unreviewed and unproven information "Be wowed by the convenience and speed", or merely "take advantage of the mere availability"? What most people find to be damning about expert advice is that they simply can't get it anywhere, at any cost that they can afford.

whatever1 4mo ago
So if you want to do a surgery but you don’t see any surgeons around you ask a grocery butcher to have his way?
- sxg 4mo ago
  In certain circumstances, the answer is yes. If an airplane's pilots are incapacitated, do you simply give up and crash the plane because there are no other pilots on board? Or would you rather have someone on the ground try to coach a passenger into at least attempting to land the plane?
  - ChrisMarshallNY 4mo ago
    As long as that passenger didn’t have the fish.
    - acheron 4mo ago
      Yes, I remember, I had lasagna.
  - close04 4mo ago
    A passenger crashing the plane while trying to avoid a certain crash doesn’t make things any worse. An incompetent doctor trying to save you from certain death can make things so much worse. It’s all about weighing the best/worst outcome compared to where you are now.
    - microgpt 4mo ago
      I hate to break it to you but death is certain for everyone. Properly emotionally processing this fact and your complete inability to do anything about it is called an "existential crisis" and if you haven't had one or several yet, you will.
      - close04 4mo ago
        I’m not sure what the “revelation” is? How is this related to what I said? Putting that aside, your philosophy sounds shallow. Death is certain, but how long you have to live and the quality of that life are not predefined. An incompetent passenger-pilot trying to save you from a crash will at worst make no difference. But an incompetent doctor can teach you that death isn’t necessarily the worst outcome.
        microgpt 4mo ago
        There are many healthy psychological ways to accept the certainty of eventual death. But the process is inevitably painful. I think the different ways people accept death explains a lot of people's psychology, like how you can guess people's attachment styles or Freudian stage fixation. For instance, billionaires who pour all their money into anti-aging research clearly are not handling it well.
        close04 4mo ago
        I thought LLMs are better than random weak sauce replies. Micro indeed.
  - jancsika 4mo ago
    You can choose a) a calm, level-headed passenger who knows they aren't a pilot, or b) a calm, level-headed passenger who almost has their pilots license but has a medical condition that prevents them from admitting when they lack certain knowledge. Who do you choose to be coached by an expert on the ground?
    - rvnx 4mo ago
      No thank you, I will ask Claude and then ask ChatGPT to challenge me, and do a couple of rounds like that. The first: Has no clue about anything and therefore no useful knowledge and cannot challenge me The second one: Is proven to willfully give wrong information and will make me do mistakes for sure. The LLMs will do their best, even if imperfect, since they summarizes what appeared in books. I prefer to be grounded on what Airbus / Boeing manuals, or on what pilots training book said, than two far more unreliable sources.
  - frereubu 4mo ago
    That's an extreme edge case, which I don't think is in the context of the concerns in this thread.
    - sxg 4mo ago
      The specific case doesn't matter--it's meant to make you think about the general question throughout this thread: when an expert isn't available, should non-experts use AI (or other tools) to help themselves? Sometimes the answer is yes because the potential benefits outweigh the potential harms (if any harms exist). But sometimes the answer is no because misleading/incorrect advice can cause a net harm.
      - frereubu 4mo ago
        But if the cases where AI use is a net positive are one in a million in medical situations? The argument is surely about the ratio, which many people here are arguing (from anecdote, would be interested to see a real study) is not in its favour, and the potential downsides - from both false positives and negatives - can be huge.
- EA-3167 4mo ago
  People, especially in medical crises, are desperate for answers that they often can't get because their clinicians don't know. The illusion of an all-knowing guru who sounds like their doctor and tells them ANYTHING is extremely alluring. If you're waiting to hear back from a doctor about test results (which these days probably showed up on your online account the moment they were completed) can be agonizing. Ok for pain in your shoulder it might not, but how about a woman with a lump in her breast waiting for the mammogram interpretation? How about someone trying to understand disturbing lab results? People are also often pushed these days to move through visits with doctors at a breakneck speed, but the AI will "hear you out" all day. Part of this is a problem with the AI, part of it a problem with our healthcare systems, and part of it is simply human nature. If you think that OpenAI, Anthropic, Google and the rest weren't aware of this going in you must have very little faith in the intelligence of their members. It's not hard to imagine the future of LLM's should involve a hell of a lot of liability on the companies running it, but for now it's the Wild West.
  - bilsbie 4mo ago
    > but how about Whatever scenario you come up with my answer is the same. As an adult I’d like to be able to choose what tools I use to learn about my condition regardless of how well it works or even if it’s likely to mislead me. There’s risk in every aspect of life and we can’t baby proof everything.
    - baconmania 4mo ago
      >choose what tools I use to learn about my condition regardless of how well it works or even if it’s likely to mislead me. Even if it "works" so poorly that you're not actually learning about your condition?
    - EA-3167 4mo ago
      If it's helping you learn about your condition then sure I agree. The issue here is that's not really the case, it's giving you the illusion that you're learning about your condition while feeding you hallucinations and half-truths at best. A recent look at medical advice from these things showed they're no better than a coin flip. So if you MUST have answers that are at most random guesses, I'd suggest saving a few bucks and asking a coin before flipping it.
- perching_aix 4mo ago
  No, people don't even go to a butcher, they do it themselves if they can. See the countless stories about farmers and their inventiveness. Example: https://www.youtube.com/watch?v=KKaJhQBusH8 https://www.youtube.com/watch?v=KKaJhQBusH8

whatever1 4mo ago

So if you want to do a surgery but you don’t see any surgeons around you ask a grocery butcher to have his way?

sxg 4mo ago
In certain circumstances, the answer is yes. If an airplane's pilots are incapacitated, do you simply give up and crash the plane because there are no other pilots on board? Or would you rather have someone on the ground try to coach a passenger into at least attempting to land the plane?
- ChrisMarshallNY 4mo ago
  As long as that passenger didn’t have the fish.
  - acheron 4mo ago
    Yes, I remember, I had lasagna.
- close04 4mo ago
  A passenger crashing the plane while trying to avoid a certain crash doesn’t make things any worse. An incompetent doctor trying to save you from certain death can make things so much worse. It’s all about weighing the best/worst outcome compared to where you are now.
  - microgpt 4mo ago
    I hate to break it to you but death is certain for everyone. Properly emotionally processing this fact and your complete inability to do anything about it is called an "existential crisis" and if you haven't had one or several yet, you will.
    - close04 4mo ago
      I’m not sure what the “revelation” is? How is this related to what I said? Putting that aside, your philosophy sounds shallow. Death is certain, but how long you have to live and the quality of that life are not predefined. An incompetent passenger-pilot trying to save you from a crash will at worst make no difference. But an incompetent doctor can teach you that death isn’t necessarily the worst outcome.
      - microgpt 4mo ago
        There are many healthy psychological ways to accept the certainty of eventual death. But the process is inevitably painful. I think the different ways people accept death explains a lot of people's psychology, like how you can guess people's attachment styles or Freudian stage fixation. For instance, billionaires who pour all their money into anti-aging research clearly are not handling it well.
        close04 4mo ago
        I thought LLMs are better than random weak sauce replies. Micro indeed.
- jancsika 4mo ago
  You can choose a) a calm, level-headed passenger who knows they aren't a pilot, or b) a calm, level-headed passenger who almost has their pilots license but has a medical condition that prevents them from admitting when they lack certain knowledge. Who do you choose to be coached by an expert on the ground?
  - rvnx 4mo ago
    No thank you, I will ask Claude and then ask ChatGPT to challenge me, and do a couple of rounds like that. The first: Has no clue about anything and therefore no useful knowledge and cannot challenge me The second one: Is proven to willfully give wrong information and will make me do mistakes for sure. The LLMs will do their best, even if imperfect, since they summarizes what appeared in books. I prefer to be grounded on what Airbus / Boeing manuals, or on what pilots training book said, than two far more unreliable sources.
- frereubu 4mo ago
  That's an extreme edge case, which I don't think is in the context of the concerns in this thread.
  - sxg 4mo ago
    The specific case doesn't matter--it's meant to make you think about the general question throughout this thread: when an expert isn't available, should non-experts use AI (or other tools) to help themselves? Sometimes the answer is yes because the potential benefits outweigh the potential harms (if any harms exist). But sometimes the answer is no because misleading/incorrect advice can cause a net harm.
    - frereubu 4mo ago
      But if the cases where AI use is a net positive are one in a million in medical situations? The argument is surely about the ratio, which many people here are arguing (from anecdote, would be interested to see a real study) is not in its favour, and the potential downsides - from both false positives and negatives - can be huge.
EA-3167 4mo ago
People, especially in medical crises, are desperate for answers that they often can't get because their clinicians don't know. The illusion of an all-knowing guru who sounds like their doctor and tells them ANYTHING is extremely alluring. If you're waiting to hear back from a doctor about test results (which these days probably showed up on your online account the moment they were completed) can be agonizing. Ok for pain in your shoulder it might not, but how about a woman with a lump in her breast waiting for the mammogram interpretation? How about someone trying to understand disturbing lab results? People are also often pushed these days to move through visits with doctors at a breakneck speed, but the AI will "hear you out" all day. Part of this is a problem with the AI, part of it a problem with our healthcare systems, and part of it is simply human nature. If you think that OpenAI, Anthropic, Google and the rest weren't aware of this going in you must have very little faith in the intelligence of their members. It's not hard to imagine the future of LLM's should involve a hell of a lot of liability on the companies running it, but for now it's the Wild West.
- bilsbie 4mo ago
  > but how about Whatever scenario you come up with my answer is the same. As an adult I’d like to be able to choose what tools I use to learn about my condition regardless of how well it works or even if it’s likely to mislead me. There’s risk in every aspect of life and we can’t baby proof everything.
  - baconmania 4mo ago
    >choose what tools I use to learn about my condition regardless of how well it works or even if it’s likely to mislead me. Even if it "works" so poorly that you're not actually learning about your condition?
  - EA-3167 4mo ago
    If it's helping you learn about your condition then sure I agree. The issue here is that's not really the case, it's giving you the illusion that you're learning about your condition while feeding you hallucinations and half-truths at best. A recent look at medical advice from these things showed they're no better than a coin flip. So if you MUST have answers that are at most random guesses, I'd suggest saving a few bucks and asking a coin before flipping it.
perching_aix 4mo ago
No, people don't even go to a butcher, they do it themselves if they can. See the countless stories about farmers and their inventiveness. Example: https://www.youtube.com/watch?v=KKaJhQBusH8 https://www.youtube.com/watch?v=KKaJhQBusH8