5 ms·

RE: ChatGTP... How confident are you that it's giving you the correct answers to your queries? I've noticed that it often sounds right, but is sometimes comple

RE: ChatGTP... How confident are you that it's giving you the correct answers to your queries? I've noticed that it often sounds right, but is sometimes completely wrong, in subtle ways. It always puts together words that are commonly near eachother, and it is always grammatically correct, but it is often absolutely wrong. I've observed that the more specific and technical a question I ask, the more likely it is to be wrong.

cooperaustinj 4y ago
How often are you confident that you find the right answer on Google? As with anything, you need to reason about it yourself and verify.
- ornornor 4y ago
  Parent is using chatgpt as a tutor though, not as a google search. I’d expect a tutor to give the right answer, but I wouldn’t expect google to. Chatgpt is often wrong. It’s a problem if you’re trying to learn something and using it as your tutor/truth.
- ironrabbit 4y ago
  Anecdotally, chatgpt seems much worse to me than Google for getting correct answers. Like orders of magnitude worse. Tells me the wrong timezone for a city kind of bad. No doubt it will be much better in the future, and they've definitely found PMF with the interface, but I would not trust it right now with anything even slightly important to me.
- JW_00000 4y ago
  But how can you verify ChatGPT's answers if you don't know what its sources were? E.g. if I google a technical question about HTML5, I can see whether a result is the HTML5 spec, MDN, W3Schools, or a random medium blog. If I google a medical question, I can see if I'm on a hospital's website or on Men's Health.
hammeiam 4y ago
You can actually ask ChatGPT to cite its sources. Anecdotally when I tried, it provided me with a valid url to a census.gov site that contained the answer to my question. That doesn't make it better than google, but it does mean that some verification is possible.
- masukomi 4y ago
  it will, very frequently, make up citations with real looking but completely bogus urls. it is NOT a research tool. It is a creative text generation tool. It is not programmed to provide accurate data. It's programmed to provide accurate sounding text.
  - O_H_E 4y ago
    Actually just today I asked if for the density of silicon nitrite and to cite a source. It cave a citation that seemed correct (reference book on materials) but with completely made up authors.

cooperaustinj 4y ago

How often are you confident that you find the right answer on Google? As with anything, you need to reason about it yourself and verify.

ornornor 4y ago
Parent is using chatgpt as a tutor though, not as a google search. I’d expect a tutor to give the right answer, but I wouldn’t expect google to. Chatgpt is often wrong. It’s a problem if you’re trying to learn something and using it as your tutor/truth.
ironrabbit 4y ago
Anecdotally, chatgpt seems much worse to me than Google for getting correct answers. Like orders of magnitude worse. Tells me the wrong timezone for a city kind of bad. No doubt it will be much better in the future, and they've definitely found PMF with the interface, but I would not trust it right now with anything even slightly important to me.
JW_00000 4y ago
But how can you verify ChatGPT's answers if you don't know what its sources were? E.g. if I google a technical question about HTML5, I can see whether a result is the HTML5 spec, MDN, W3Schools, or a random medium blog. If I google a medical question, I can see if I'm on a hospital's website or on Men's Health.

hammeiam 4y ago

You can actually ask ChatGPT to cite its sources. Anecdotally when I tried, it provided me with a valid url to a census.gov site that contained the answer to my question. That doesn't make it better than google, but it does mean that some verification is possible.

masukomi 4y ago
it will, very frequently, make up citations with real looking but completely bogus urls. it is NOT a research tool. It is a creative text generation tool. It is not programmed to provide accurate data. It's programmed to provide accurate sounding text.
- O_H_E 4y ago
  Actually just today I asked if for the density of silicon nitrite and to cite a source. It cave a citation that seemed correct (reference book on materials) but with completely made up authors.