4 ms·

Tell HN: ChatGPT has become untrustworthy

14 points by gtirloni 3y ago

This has happened for several weeks now with GPT-4: I ask it how I can do something that I know is possible and just want it to get me the right syntax. It will say "X can't be done with Y but you can use Z, here's how". I know it's possible so I just say to it "I can't believe doing X with Y is not possible" and it immediatelly says "yes, doing X with Y is possible here's now"... and now I don't know if it's just "complying" or if it's really possible or not (maybe something changed and it's not possible anymore?). So now I have to go to Google/Kagi and search it myself to confirm it's giving me the right-ish answer and not just playing nice and hallucinating. I've cancelled my ChatGPT subscription today until/if they can ever get it back to how it was several months ago. ps.: Claude 3 Opus doesn't seem to suffer from the same issues.

borissk 3y ago
It's a work in progress and it still has to be done mostly by humans. Humans make mistakes. Claude is in many ways superior to ChatGPT, reverse is also true.
smoldesu 3y ago
All AI is untrustworthy. Many people have been saying this since the start, and there's not really any evidence that AI will or can attain perfection. Set your expectations accordingly. P.S.: There is no greener pasture. If you've been following the scene since the BERT/GPT-2 days, it becomes readily apparent how "far" text generators have come. They got bigger, more expensive, slower, censored, and 10-20% better at solving human reasoning problems. There is no light at the tunnel suggesting infallible LLMs are coming. You are being sold a moving goalpost.
- gtirloni 3y ago
  Not true. Things have degraded considerably for ChatGPT. Yes, it's just a stochastic mechanism but there are degrees of accuracy and they have gone down lately.
- dbacar 3y ago
  If you set your expectations accordingly, I find ChatGPT answers fairly helpful. At least for my experience.
  - smoldesu 3y ago
    I actually tend to agree. My struggle is with people that sensationalize the untrustworthiness of AI when it has been plainly apparent since day-zero.
jamil7 3y ago
I'm not sure if I conciously noticed it but I did notice today that I just haven't been using ChatGPT or LLMs for any programming tasks for the past two weeks or so. I think it has been giving progressively worse answers or I've just hit the limits of how useful I find it. Claude isn't available in my region but maybe it's worth a try.
- gtirloni 3y ago
  You can try Claude 3 through Poe.
macksd 3y ago
I mean for anything you could have easily verified via a search engine, I would say you always should have been verifying. I've had ChatGPT give me incorrect but correct-seeming information many times, and not just in the last several weeks.
yosito 3y ago
You completely misunderstand LLMs if you think they ever were "reliable" in the first place. All it is, is a plausible text autocompleter.
pelorat 3y ago
Use an LLM that searches for answers online, like Bing Copilot or Google Gemini.
duringmath 3y ago
All LLMs are inherently untrustworthy, they can be great to get the ball rolling so to speak but you have to double check everything they generate which is its own kind of overhead.
BiteCode_dev 3y ago
In other news, Google returns spam sites and Excel parses dates incorrectly. How is this ramble of a post became front page on HN is beyond me.
sourabh03agr 3y ago
I have experienced such issues with GPT-4 returning the wrong syntax, especially when asking questions about complex operations in Polars. We are actually running performance monitoring for GPT-3.5/4 and Claude-2 to systematically track such regressions: https://demo.uptrain.ai/benchmark/ https://demo.uptrain.ai/benchmark/. We would be curious to know your thoughts and feedback.
fabiofzero 3y ago
Of course, because it was inherently trustworthy before. Goddamn, we're all lost, aren't we? Either that or you're really into believing the hype.
delichon 3y ago
I was testing ChatGPT's ability to access current events and asked it if it knows about the "bloodbath" controversy. Just that word. It briefly described the mainstream media's version of events. I asked it a few different ways if it was aware of another interpretation, it was not. Its media bubble appears to be robust. I suppose if you trust the mainstream media that's a feature.

14 points by gtirloni 3y ago

borissk 3y ago

It's a work in progress and it still has to be done mostly by humans. Humans make mistakes. Claude is in many ways superior to ChatGPT, reverse is also true.

smoldesu 3y ago

All AI is untrustworthy. Many people have been saying this since the start, and there's not really any evidence that AI will or can attain perfection. Set your expectations accordingly. P.S.: There is no greener pasture. If you've been following the scene since the BERT/GPT-2 days, it becomes readily apparent how "far" text generators have come. They got bigger, more expensive, slower, censored, and 10-20% better at solving human reasoning problems. There is no light at the tunnel suggesting infallible LLMs are coming. You are being sold a moving goalpost.

gtirloni 3y ago
Not true. Things have degraded considerably for ChatGPT. Yes, it's just a stochastic mechanism but there are degrees of accuracy and they have gone down lately.
dbacar 3y ago
If you set your expectations accordingly, I find ChatGPT answers fairly helpful. At least for my experience.
- smoldesu 3y ago
  I actually tend to agree. My struggle is with people that sensationalize the untrustworthiness of AI when it has been plainly apparent since day-zero.

jamil7 3y ago

I'm not sure if I conciously noticed it but I did notice today that I just haven't been using ChatGPT or LLMs for any programming tasks for the past two weeks or so. I think it has been giving progressively worse answers or I've just hit the limits of how useful I find it. Claude isn't available in my region but maybe it's worth a try.

gtirloni 3y ago
You can try Claude 3 through Poe.

macksd 3y ago

I mean for anything you could have easily verified via a search engine, I would say you always should have been verifying. I've had ChatGPT give me incorrect but correct-seeming information many times, and not just in the last several weeks.

yosito 3y ago

You completely misunderstand LLMs if you think they ever were "reliable" in the first place. All it is, is a plausible text autocompleter.

pelorat 3y ago

Use an LLM that searches for answers online, like Bing Copilot or Google Gemini.

duringmath 3y ago

All LLMs are inherently untrustworthy, they can be great to get the ball rolling so to speak but you have to double check everything they generate which is its own kind of overhead.

BiteCode_dev 3y ago

In other news, Google returns spam sites and Excel parses dates incorrectly. How is this ramble of a post became front page on HN is beyond me.

sourabh03agr 3y ago

I have experienced such issues with GPT-4 returning the wrong syntax, especially when asking questions about complex operations in Polars. We are actually running performance monitoring for GPT-3.5/4 and Claude-2 to systematically track such regressions: https://demo.uptrain.ai/benchmark/ https://demo.uptrain.ai/benchmark/. We would be curious to know your thoughts and feedback.

fabiofzero 3y ago

Of course, because it was inherently trustworthy before. Goddamn, we're all lost, aren't we? Either that or you're really into believing the hype.

delichon 3y ago

I was testing ChatGPT's ability to access current events and asked it if it knows about the "bloodbath" controversy. Just that word. It briefly described the mainstream media's version of events. I asked it a few different ways if it was aware of another interpretation, it was not. Its media bubble appears to be robust. I suppose if you trust the mainstream media that's a feature.