4 ms·

How do you assess whether the information from ChatGPT is correct? It's downfall is that it very convincingly produces false information. At least in a Google s

How do you assess whether the information from ChatGPT is correct? It's downfall is that it very convincingly produces false information. At least in a Google search you can compare multiple sources, and blogspam is usually easy to filter out once you're good at identifying it. In ChatGPT, good and bad results look equally plausible.

trees101 4y ago
https://www.perplexity.ai https://www.perplexity.ai This augments ChatGPT results with Bing results to provide references you can look up.
heliophobicdude 4y ago
I started asking in a follow up for it to write a set of test cases. I could tweak and run them to my liking. I think for knowledge based things, you are better off using Google. But if it can help you narrow down your search, you can use that for Google. For example, I queried, what projects are good for practicing data-intensive design. It gave me a list of projects with descriptions. Perfect for me to Google more information for. It's very good for narrowing down search and addressing follow ups. It's like the perfect Google Search companion.
- oomrazoom 4y ago
  you.com Offers a chat feature that pairs the gpt-3 response with search results
CSMastermind 4y ago
Lots of time when I'm searching I'm relying on recognition and not recall. I know the right answer but I don't have it memorized. If I search for python for loop syntax or metallica bob seger cover or tuck rule game year, etc. I know the right answer if you show it to me and given enough time to think I probably could recall it but I don't have the information at the top of my mind. If chatgpt returns something that's wrong I'll know it's wrong and then maybe go check google.
WalterBright 4y ago
Some people have tried asking ChatGPT to do some D coding. The results looked like plausible D code, but were subtly wrong.
0x6c6f6c 4y ago
> it very convincingly produces false information This is exactly the scenario I encountered. I have a teammate who is using ChatGPT to ask questions as opposed to searching documentation and it gave incorrect info all around on all the following: 1. Where the property would be found in the event schema 2. What structure the value would be 3. The fact that it's not included at all for AWS AppSync specifically in question, even if the previous two had been answered correctly regarding API Gateway event schema. All of this was available in the first doc returned in the search results of both API Gateway and AppSync respectively. The fact it convincingly tells you otherwise causes additional friction since they're pointing at their screen telling me I'm wrong cause look at the response from ChatGPT.
- deleted 4y ago
  [deleted]
- nprateem 4y ago
  I think solving this inaccuracy will be openai's biggest challenge. The web is safe for now. Once they do fix it it's basically game over.
  - ilaksh 4y ago
    You can turn the temperature down on the OpenAI API and it is much more accurate. They also have a code-specific model which I don't think ChatGPT is using.
- moritonal 4y ago
  But, you have to admit that if ChatGPT had a process whereby it'd internally list the sites most likely relevant to your question (essentially Google the keywords) then included those sites as context it'd likely ace your question? Basically, if Google owned ChatGPT the result would be very different.
- spaceman_2020 4y ago
  Whenever chatGPT gives me a function and I try it out and it doesn’t work, I just tell it “hey this isn’t working” and it will correct itself
lumost 4y ago
How do you do this with google search when half the results are spammed copies of the same content? For many cases you can get enough information out of chatGPT to retrieve a specific corroborating reference. For coding tasks, you can just try the code, ask it for tests, or give it code review like feedback. I’d suggest asking it about deep technical topics that you are familiar with, then work out it’s limitations. It’s effectively equivalent to a universal teaching assistant, sometimes wrong - usually points in the right direction.
- vel0city 4y ago
  The answer is still the same, I evaluate the source. Which is near impossible to truly know with bots like ChatGPT.
drusepth 4y ago
I just treat ChatGPT as another source. It's become a go-to for questions (according to my log, this week I've asked it about naturopathic medicine, game development, algorithms, FSAs, vocabulary, and general trivia). It's waaaaay faster to get an immediate answer -- and be able to ask pointed, follow-up questions, which you couldn't otherwise do with a traditional site. And it's a lot easier to just fact-check statements online than to wade headfirst into something you know nothing about in random sites of varying qualities.
rerdavies 4y ago
Just ask for links. The example below is contrived. But when you're looking for very specific information, GptChat can save you from wading through pages of Google Search results. Providing a list of 30,000 pages that are only tangentially related to what you want is not a virtue. ----- > Give me a link to page that gives an introduction to quantum computing. Here is a link to a page that provides an introduction to quantum computing: https://www.ibm.com/quantum-computing/learn/what-is-quantum-computing/ https://www.ibm.com/quantum-computing/learn/what-is-quantum-... This page gives a brief overview of what quantum computing is, how it works, and some of the potential applications. It also includes links to more in-depth resources for those who want to learn more. I hope this helps! Let me know if you have any other questions. > Give me another link. Here is another link that provides an introduction to quantum computing: https://www.dwavesys.com/tutorials/getting-started-with-quantum-computing https://www.dwavesys.com/tutorials/getting-started-with-quan... This page gives a high-level overview of quantum computing, including the basics of quantum mechanics and how it is used in computing. It also discusses the potential applications of quantum computing and some of the challenges and limitations of the technology. I hope this helps! Let me know if you have any other questions. ...

trees101 4y ago

https://www.perplexity.ai https://www.perplexity.ai This augments ChatGPT results with Bing results to provide references you can look up.

heliophobicdude 4y ago

I started asking in a follow up for it to write a set of test cases. I could tweak and run them to my liking. I think for knowledge based things, you are better off using Google. But if it can help you narrow down your search, you can use that for Google. For example, I queried, what projects are good for practicing data-intensive design. It gave me a list of projects with descriptions. Perfect for me to Google more information for. It's very good for narrowing down search and addressing follow ups. It's like the perfect Google Search companion.

oomrazoom 4y ago
you.com Offers a chat feature that pairs the gpt-3 response with search results

CSMastermind 4y ago

Lots of time when I'm searching I'm relying on recognition and not recall. I know the right answer but I don't have it memorized. If I search for python for loop syntax or metallica bob seger cover or tuck rule game year, etc. I know the right answer if you show it to me and given enough time to think I probably could recall it but I don't have the information at the top of my mind. If chatgpt returns something that's wrong I'll know it's wrong and then maybe go check google.

WalterBright 4y ago

Some people have tried asking ChatGPT to do some D coding. The results looked like plausible D code, but were subtly wrong.

0x6c6f6c 4y ago

> it very convincingly produces false information This is exactly the scenario I encountered. I have a teammate who is using ChatGPT to ask questions as opposed to searching documentation and it gave incorrect info all around on all the following: 1. Where the property would be found in the event schema 2. What structure the value would be 3. The fact that it's not included at all for AWS AppSync specifically in question, even if the previous two had been answered correctly regarding API Gateway event schema. All of this was available in the first doc returned in the search results of both API Gateway and AppSync respectively. The fact it convincingly tells you otherwise causes additional friction since they're pointing at their screen telling me I'm wrong cause look at the response from ChatGPT.

deleted 4y ago
[deleted]
nprateem 4y ago
I think solving this inaccuracy will be openai's biggest challenge. The web is safe for now. Once they do fix it it's basically game over.
- ilaksh 4y ago
  You can turn the temperature down on the OpenAI API and it is much more accurate. They also have a code-specific model which I don't think ChatGPT is using.
moritonal 4y ago
But, you have to admit that if ChatGPT had a process whereby it'd internally list the sites most likely relevant to your question (essentially Google the keywords) then included those sites as context it'd likely ace your question? Basically, if Google owned ChatGPT the result would be very different.
spaceman_2020 4y ago
Whenever chatGPT gives me a function and I try it out and it doesn’t work, I just tell it “hey this isn’t working” and it will correct itself

lumost 4y ago

How do you do this with google search when half the results are spammed copies of the same content? For many cases you can get enough information out of chatGPT to retrieve a specific corroborating reference. For coding tasks, you can just try the code, ask it for tests, or give it code review like feedback. I’d suggest asking it about deep technical topics that you are familiar with, then work out it’s limitations. It’s effectively equivalent to a universal teaching assistant, sometimes wrong - usually points in the right direction.

vel0city 4y ago
The answer is still the same, I evaluate the source. Which is near impossible to truly know with bots like ChatGPT.

drusepth 4y ago

I just treat ChatGPT as another source. It's become a go-to for questions (according to my log, this week I've asked it about naturopathic medicine, game development, algorithms, FSAs, vocabulary, and general trivia). It's waaaaay faster to get an immediate answer -- and be able to ask pointed, follow-up questions, which you couldn't otherwise do with a traditional site. And it's a lot easier to just fact-check statements online than to wade headfirst into something you know nothing about in random sites of varying qualities.

rerdavies 4y ago

Just ask for links. The example below is contrived. But when you're looking for very specific information, GptChat can save you from wading through pages of Google Search results. Providing a list of 30,000 pages that are only tangentially related to what you want is not a virtue. ----- > Give me a link to page that gives an introduction to quantum computing. Here is a link to a page that provides an introduction to quantum computing: https://www.ibm.com/quantum-computing/learn/what-is-quantum-computing/ https://www.ibm.com/quantum-computing/learn/what-is-quantum-... This page gives a brief overview of what quantum computing is, how it works, and some of the potential applications. It also includes links to more in-depth resources for those who want to learn more. I hope this helps! Let me know if you have any other questions. > Give me another link. Here is another link that provides an introduction to quantum computing: https://www.dwavesys.com/tutorials/getting-started-with-quantum-computing https://www.dwavesys.com/tutorials/getting-started-with-quan... This page gives a high-level overview of quantum computing, including the basics of quantum mechanics and how it is used in computing. It also discusses the potential applications of quantum computing and some of the challenges and limitations of the technology. I hope this helps! Let me know if you have any other questions. ...