3 ms·

She's weirded out by creepy hallucinations, which is understandable! But ChatGPT is well known to hallucinate. In other words she doesn't know which of its beha

She's weirded out by creepy hallucinations, which is understandable! But ChatGPT is well known to hallucinate. In other words she doesn't know which of its behaviors are normal so she doesn't know how to react. Additionally, her particular issues are quite solvable with better prompting.

ceejayoz 1y ago
> If I do poor work with an electric drill then it's not the drill's fault. > ChatGPT's sycophancy crisis was late April. If you drill starts telling you "what a great job you're doing, keep drilling into that electrical conduit", the drill is at least partially at fault. A tool that randomly and unpredictably fails is a bad tool. How should I, as a user, account for the possibility/likelihood of another such crisis in the future?
- sandspar 1y ago
  I was in the midst of editing the comment when you replied, sorry. I didn't see your reply before I edited mine.
  - ceejayoz 1y ago
    OK, but you're still blaming the user for the tool's failings. Which even the makers of the tool agreed were failings.
    - sandspar 1y ago
      That's a valid point! The AI community still has much to improve on.
- johnfn 1y ago
  > A tool that randomly and unpredictably fails is a bad tool. But all failures are "random and unpredictable" if you have no baseline understanding of how to use the tool. "AIs hallucinate" is probably the single most obvious thing about AIs. This isn't a subtle misunderstanding that an expert could make. This is like using a drill on your face.
  - ceejayoz 1y ago
    > But all failures are "random and unpredictable" if you have no baseline understanding of how to use the tool. But the tool's behavior changed. In ways that even its creators didn't intend (example: https://openai.com/index/sycophancy-in-gpt-4o/ https://openai.com/index/sycophancy-in-gpt-4o/), and had to work to undo. If my hammer had a random week every year where it tried to smack me in the face whenever I touched it, I'd probably avoid using it.
    - deleted 1y ago
      [deleted]
    - johnfn 1y ago
      This is unrelated to sycophancy. The author is failing to understand that GPT did not make a tool call and is hallucinating. Hallucinations have always been a thing. They are not some new, surprising development.
      - ceejayoz 1y ago
        > This is unrelated to sycophancy. "It's a stunning piece. You write with an unflinching emotional clarity that's both intimate and beautifully restrained." > They are not some new, surprising development. OpenAI sure seemed surprised. https://openai.com/index/sycophancy-in-gpt-4o/ https://openai.com/index/sycophancy-in-gpt-4o/
        johnfn 1y ago
        > "It's a stunning piece. You write with an unflinching emotional clarity that's both intimate and beautifully restrained." This is a hallucination, since there is no source to refer to. The author was surprised because GPT was hallucinating, not because GPT was extra nice. Sycophancy might be related, but it's not the point of the article. If GPT had said "wow, your post is trash", the author would have been equally surprised to learn it was a hallucination.
        1y ago

ceejayoz 1y ago

> If I do poor work with an electric drill then it's not the drill's fault. > ChatGPT's sycophancy crisis was late April. If you drill starts telling you "what a great job you're doing, keep drilling into that electrical conduit", the drill is at least partially at fault. A tool that randomly and unpredictably fails is a bad tool. How should I, as a user, account for the possibility/likelihood of another such crisis in the future?

sandspar 1y ago
I was in the midst of editing the comment when you replied, sorry. I didn't see your reply before I edited mine.
- ceejayoz 1y ago
  OK, but you're still blaming the user for the tool's failings. Which even the makers of the tool agreed were failings.
  - sandspar 1y ago
    That's a valid point! The AI community still has much to improve on.
johnfn 1y ago
> A tool that randomly and unpredictably fails is a bad tool. But all failures are "random and unpredictable" if you have no baseline understanding of how to use the tool. "AIs hallucinate" is probably the single most obvious thing about AIs. This isn't a subtle misunderstanding that an expert could make. This is like using a drill on your face.
- ceejayoz 1y ago
  > But all failures are "random and unpredictable" if you have no baseline understanding of how to use the tool. But the tool's behavior changed. In ways that even its creators didn't intend (example: https://openai.com/index/sycophancy-in-gpt-4o/ https://openai.com/index/sycophancy-in-gpt-4o/), and had to work to undo. If my hammer had a random week every year where it tried to smack me in the face whenever I touched it, I'd probably avoid using it.
  - deleted 1y ago
    [deleted]
  - johnfn 1y ago
    This is unrelated to sycophancy. The author is failing to understand that GPT did not make a tool call and is hallucinating. Hallucinations have always been a thing. They are not some new, surprising development.
    - ceejayoz 1y ago
      > This is unrelated to sycophancy. "It's a stunning piece. You write with an unflinching emotional clarity that's both intimate and beautifully restrained." > They are not some new, surprising development. OpenAI sure seemed surprised. https://openai.com/index/sycophancy-in-gpt-4o/ https://openai.com/index/sycophancy-in-gpt-4o/
      - johnfn 1y ago
        > "It's a stunning piece. You write with an unflinching emotional clarity that's both intimate and beautifully restrained." This is a hallucination, since there is no source to refer to. The author was surprised because GPT was hallucinating, not because GPT was extra nice. Sycophancy might be related, but it's not the point of the article. If GPT had said "wow, your post is trash", the author would have been equally surprised to learn it was a hallucination.
        1y ago