3 ms·

> A tool that randomly and unpredictably fails is a bad tool. But all failures are "random and unpredictable" if you have no baseline understanding of how to u

by johnfn 1y ago

> A tool that randomly and unpredictably fails is a bad tool. But all failures are "random and unpredictable" if you have no baseline understanding of how to use the tool. "AIs hallucinate" is probably the single most obvious thing about AIs. This isn't a subtle misunderstanding that an expert could make. This is like using a drill on your face.

ceejayoz 1y ago
> But all failures are "random and unpredictable" if you have no baseline understanding of how to use the tool. But the tool's behavior changed. In ways that even its creators didn't intend (example: https://openai.com/index/sycophancy-in-gpt-4o/ https://openai.com/index/sycophancy-in-gpt-4o/), and had to work to undo. If my hammer had a random week every year where it tried to smack me in the face whenever I touched it, I'd probably avoid using it.
- deleted 1y ago
  [deleted]
- johnfn 1y ago
  This is unrelated to sycophancy. The author is failing to understand that GPT did not make a tool call and is hallucinating. Hallucinations have always been a thing. They are not some new, surprising development.
  - ceejayoz 1y ago
    > This is unrelated to sycophancy. "It's a stunning piece. You write with an unflinching emotional clarity that's both intimate and beautifully restrained." > They are not some new, surprising development. OpenAI sure seemed surprised. https://openai.com/index/sycophancy-in-gpt-4o/ https://openai.com/index/sycophancy-in-gpt-4o/
    - johnfn 1y ago
      > "It's a stunning piece. You write with an unflinching emotional clarity that's both intimate and beautifully restrained." This is a hallucination, since there is no source to refer to. The author was surprised because GPT was hallucinating, not because GPT was extra nice. Sycophancy might be related, but it's not the point of the article. If GPT had said "wow, your post is trash", the author would have been equally surprised to learn it was a hallucination.
      - ceejayoz 1y ago
        "Oh, yes, I read your new novel, it's great!" is precisely what a sycophant would tell you when they forgot to read it.
lukeschlather 1y ago
But in the context of this thread, I would say that using an AI to examine logical inconsistencies is the wrong way to use the tool. The problem with LLMs is that they don't have any intentionality to their worldview. They're like a wise turtle that comes to you in a dream, their dream logic is not something you should pay much attention to.

by johnfn 1y ago

ceejayoz 1y ago

> But all failures are "random and unpredictable" if you have no baseline understanding of how to use the tool. But the tool's behavior changed. In ways that even its creators didn't intend (example: https://openai.com/index/sycophancy-in-gpt-4o/ https://openai.com/index/sycophancy-in-gpt-4o/), and had to work to undo. If my hammer had a random week every year where it tried to smack me in the face whenever I touched it, I'd probably avoid using it.

deleted 1y ago
[deleted]
johnfn 1y ago
This is unrelated to sycophancy. The author is failing to understand that GPT did not make a tool call and is hallucinating. Hallucinations have always been a thing. They are not some new, surprising development.
- ceejayoz 1y ago
  > This is unrelated to sycophancy. "It's a stunning piece. You write with an unflinching emotional clarity that's both intimate and beautifully restrained." > They are not some new, surprising development. OpenAI sure seemed surprised. https://openai.com/index/sycophancy-in-gpt-4o/ https://openai.com/index/sycophancy-in-gpt-4o/
  - johnfn 1y ago
    > "It's a stunning piece. You write with an unflinching emotional clarity that's both intimate and beautifully restrained." This is a hallucination, since there is no source to refer to. The author was surprised because GPT was hallucinating, not because GPT was extra nice. Sycophancy might be related, but it's not the point of the article. If GPT had said "wow, your post is trash", the author would have been equally surprised to learn it was a hallucination.
    - ceejayoz 1y ago
      "Oh, yes, I read your new novel, it's great!" is precisely what a sycophant would tell you when they forgot to read it.

lukeschlather 1y ago

But in the context of this thread, I would say that using an AI to examine logical inconsistencies is the wrong way to use the tool. The problem with LLMs is that they don't have any intentionality to their worldview. They're like a wise turtle that comes to you in a dream, their dream logic is not something you should pay much attention to.