4 ms·

Don't leave us hanging. What happened?

dude250711 7mo ago
They stop thinking and they stop verifying output too.
camillomiller 7mo ago
A CTO sent me a message that opened with: “Here’s a friendly message that will perfectly convey what you want to say”. A double PhD friend says she has to talk to chatGPT for all sort of advice and can’t feel safe not doing it, “because you know I’m single and don’t have a companion to spitball my ideas”. She let chatGPT decide which way to take to get to a certain island, and she got stranded because the suggested service didn’t exist. I have more examples. It’s a fucking mind virus.
- sigseg1v 7mo ago
  How is the getting stranded example different than asking on a travel forum how to get somewhere, and an active and well intentioned user that isn't familiar with your area of travel answers, gives you wrong instructions, and you get lost?
  - kibwen 7mo ago
    Because the vast and overwhelmingly majority of the time, if you ask a question into the ether that nobody has a good answer to, most people will gloss over it and not bother answering, as attested by decades of relatable memes ( https://xkcd.com/979/ https://xkcd.com/979/ ). In contrast, the chatbot is trained to always attempt to give an answer, and is seemingly disincentivized via its training set to just shrug and say "I don't know, good luck fam".
    - deleted 7mo ago
      [deleted]
  - shahbaby 7mo ago
    Because they aren't probabilistic parrots? If they get it wrong, there's usually an understandable reason behind it.
  - andrewflnr 7mo ago
    The key missing step is where the traveler exercises critical thinking and checks the advice they get. Some people seem to turn that off for LLMs.
    - array_key_first 7mo ago
      It's because we spent that last 50 years training people that computers are algorithmic, cold, and don't make human mistakes. Your calculator can't tell you the meaning of life, but it will never get 2 + 2 wrong. Well, now the calculator can tell you a meaning of life, but it'll get 2 + 2 wrong 10% of the time.
  - dijksterhuis 7mo ago
    cunningham's law [0] [1] increases the likelihood that at least one other person will point out the error and correct it. chances are you'll probably get more than one person posting. LLMs don't do this. they give confident language output, not correct answers. [0]: https://meta.wikimedia.org/wiki/Cunningham%27s_Law https://meta.wikimedia.org/wiki/Cunningham%27s_Law [1]: https://xkcd.com/386/ https://xkcd.com/386/
- HK-NC 7mo ago
  I've lost count of the interactions Ive had or witnessed in which a living breathing authority on a subject is ignored in favour of using voice recognition to ask an LLM the question first, or worse, second.

They stop thinking and they stop verifying output too.

A CTO sent me a message that opened with: “Here’s a friendly message that will perfectly convey what you want to say”. A double PhD friend says she has to talk to chatGPT for all sort of advice and can’t feel safe not doing it, “because you know I’m single and don’t have a companion to spitball my ideas”. She let chatGPT decide which way to take to get to a certain island, and she got stranded because the suggested service didn’t exist. I have more examples. It’s a fucking mind virus.

sigseg1v 7mo ago
How is the getting stranded example different than asking on a travel forum how to get somewhere, and an active and well intentioned user that isn't familiar with your area of travel answers, gives you wrong instructions, and you get lost?
- kibwen 7mo ago
  Because the vast and overwhelmingly majority of the time, if you ask a question into the ether that nobody has a good answer to, most people will gloss over it and not bother answering, as attested by decades of relatable memes ( https://xkcd.com/979/ https://xkcd.com/979/ ). In contrast, the chatbot is trained to always attempt to give an answer, and is seemingly disincentivized via its training set to just shrug and say "I don't know, good luck fam".
  - deleted 7mo ago
    [deleted]
- shahbaby 7mo ago
  Because they aren't probabilistic parrots? If they get it wrong, there's usually an understandable reason behind it.
- andrewflnr 7mo ago
  The key missing step is where the traveler exercises critical thinking and checks the advice they get. Some people seem to turn that off for LLMs.
  - array_key_first 7mo ago
    It's because we spent that last 50 years training people that computers are algorithmic, cold, and don't make human mistakes. Your calculator can't tell you the meaning of life, but it will never get 2 + 2 wrong. Well, now the calculator can tell you a meaning of life, but it'll get 2 + 2 wrong 10% of the time.
- dijksterhuis 7mo ago
  cunningham's law [0] [1] increases the likelihood that at least one other person will point out the error and correct it. chances are you'll probably get more than one person posting. LLMs don't do this. they give confident language output, not correct answers. [0]: https://meta.wikimedia.org/wiki/Cunningham%27s_Law https://meta.wikimedia.org/wiki/Cunningham%27s_Law [1]: https://xkcd.com/386/ https://xkcd.com/386/
HK-NC 7mo ago
I've lost count of the interactions Ive had or witnessed in which a living breathing authority on a subject is ignored in favour of using voice recognition to ask an LLM the question first, or worse, second.