4 ms·

All we can do is share anecdotes here, but I have found ChatGPT to be confidently incorrect about important details in nearly every question I ask about a compl

by avalys 9mo ago

All we can do is share anecdotes here, but I have found ChatGPT to be confidently incorrect about important details in nearly every question I ask about a complex topic. Legal questions, question about AWS services, products I want to buy, the history a specific field, so many things. It gives answers that do a really good job of simulating what a person who knows the topic would say. But details are wrong everywhere, often in ways that completely change the relevant conclusion.

Jarwain 9mo ago
I don't think that LLM's do a significantly worse job than the average human professional. People get details wrong all the time, too.
DBNO 9mo ago
I definitely agree that ChatGPT can be incorrect. I’ve seen that myself. In my experience, though, it’s more often right than wrong. So when you say “in nearly every question on complex topics", I’m curious what specific examples you’re seeing. Would you be open to sharing a concrete example? Specifically: the question you asked, the part of the answer you know is wrong, and what the correct answer should be. I have a hypothesis (not a claim) that some of these failures you are seeing might be prompt-sensitive, and I’d be curious to try it as a small experiment if you’re willing.
- avalys 9mo ago
  In one example, AWS has two options for automatic deletion of objects in S3 buckets that are versioned. "Expire current versions" means that the object will be automatically deleted after some period. "Permanently delete non-current versions" means that old revisions will be permanently removed after some period. I asked ChatGPT for advice on configuring a bucket. Within a long list of other instructions, it said "Expire noncurrent versions after X days". In this case, such a setting does not exist, and the very similar "expire current versions" is exactly the wrong behavior. "Permanently delete noncurrent versions" is the option needed. The prompt I used has other information in it that I don't want to share.

by avalys 9mo ago

Jarwain 9mo ago

I don't think that LLM's do a significantly worse job than the average human professional. People get details wrong all the time, too.

DBNO 9mo ago

I definitely agree that ChatGPT can be incorrect. I’ve seen that myself. In my experience, though, it’s more often right than wrong. So when you say “in nearly every question on complex topics", I’m curious what specific examples you’re seeing. Would you be open to sharing a concrete example? Specifically: the question you asked, the part of the answer you know is wrong, and what the correct answer should be. I have a hypothesis (not a claim) that some of these failures you are seeing might be prompt-sensitive, and I’d be curious to try it as a small experiment if you’re willing.

avalys 9mo ago
In one example, AWS has two options for automatic deletion of objects in S3 buckets that are versioned. "Expire current versions" means that the object will be automatically deleted after some period. "Permanently delete non-current versions" means that old revisions will be permanently removed after some period. I asked ChatGPT for advice on configuring a bucket. Within a long list of other instructions, it said "Expire noncurrent versions after X days". In this case, such a setting does not exist, and the very similar "expire current versions" is exactly the wrong behavior. "Permanently delete noncurrent versions" is the option needed. The prompt I used has other information in it that I don't want to share.