12 ms·

I've already regularly replaced many search queries with ChatGPT. The spam on G is just too painful to wade through. It's far from perfect and I wish I could

I've already regularly replaced many search queries with ChatGPT. The spam on G is just too painful to wade through. It's far from perfect and I wish I could 'tune' it to be less 'mealymouth' where the answers feel almost deliberately nerf'd to be qualified and generic but if you could tune it to offer strong opinions on these - not even politics but even product queries like, "best place to buy X" or "best value robot vacuum" that'd be interesting as these are super spammed on traditional search and I think an AI with strong opinions that could have a conversation about the product would be categorially better. Like I need to replace my water heater and I want a reasonable assortment of options for my situation from a dispassionate AI that isn't trying to sell me anything.

Thorentis 4y ago
How do you assess whether the information from ChatGPT is correct? It's downfall is that it very convincingly produces false information. At least in a Google search you can compare multiple sources, and blogspam is usually easy to filter out once you're good at identifying it. In ChatGPT, good and bad results look equally plausible.
- trees101 4y ago
  https://www.perplexity.ai https://www.perplexity.ai This augments ChatGPT results with Bing results to provide references you can look up.
- heliophobicdude 4y ago
  I started asking in a follow up for it to write a set of test cases. I could tweak and run them to my liking. I think for knowledge based things, you are better off using Google. But if it can help you narrow down your search, you can use that for Google. For example, I queried, what projects are good for practicing data-intensive design. It gave me a list of projects with descriptions. Perfect for me to Google more information for. It's very good for narrowing down search and addressing follow ups. It's like the perfect Google Search companion.
  - oomrazoom 4y ago
    you.com Offers a chat feature that pairs the gpt-3 response with search results
- CSMastermind 4y ago
  Lots of time when I'm searching I'm relying on recognition and not recall. I know the right answer but I don't have it memorized. If I search for python for loop syntax or metallica bob seger cover or tuck rule game year, etc. I know the right answer if you show it to me and given enough time to think I probably could recall it but I don't have the information at the top of my mind. If chatgpt returns something that's wrong I'll know it's wrong and then maybe go check google.
- WalterBright 4y ago
  Some people have tried asking ChatGPT to do some D coding. The results looked like plausible D code, but were subtly wrong.
- 0x6c6f6c 4y ago
  > it very convincingly produces false information This is exactly the scenario I encountered. I have a teammate who is using ChatGPT to ask questions as opposed to searching documentation and it gave incorrect info all around on all the following: 1. Where the property would be found in the event schema 2. What structure the value would be 3. The fact that it's not included at all for AWS AppSync specifically in question, even if the previous two had been answered correctly regarding API Gateway event schema. All of this was available in the first doc returned in the search results of both API Gateway and AppSync respectively. The fact it convincingly tells you otherwise causes additional friction since they're pointing at their screen telling me I'm wrong cause look at the response from ChatGPT.
  - deleted 4y ago
    [deleted]
  - nprateem 4y ago
    I think solving this inaccuracy will be openai's biggest challenge. The web is safe for now. Once they do fix it it's basically game over.
    - ilaksh 4y ago
      You can turn the temperature down on the OpenAI API and it is much more accurate. They also have a code-specific model which I don't think ChatGPT is using.
  - moritonal 4y ago
    But, you have to admit that if ChatGPT had a process whereby it'd internally list the sites most likely relevant to your question (essentially Google the keywords) then included those sites as context it'd likely ace your question? Basically, if Google owned ChatGPT the result would be very different.
  - spaceman_2020 4y ago
    Whenever chatGPT gives me a function and I try it out and it doesn’t work, I just tell it “hey this isn’t working” and it will correct itself
- lumost 4y ago
  How do you do this with google search when half the results are spammed copies of the same content? For many cases you can get enough information out of chatGPT to retrieve a specific corroborating reference. For coding tasks, you can just try the code, ask it for tests, or give it code review like feedback. I’d suggest asking it about deep technical topics that you are familiar with, then work out it’s limitations. It’s effectively equivalent to a universal teaching assistant, sometimes wrong - usually points in the right direction.
  - vel0city 4y ago
    The answer is still the same, I evaluate the source. Which is near impossible to truly know with bots like ChatGPT.
- drusepth 4y ago
  I just treat ChatGPT as another source. It's become a go-to for questions (according to my log, this week I've asked it about naturopathic medicine, game development, algorithms, FSAs, vocabulary, and general trivia). It's waaaaay faster to get an immediate answer -- and be able to ask pointed, follow-up questions, which you couldn't otherwise do with a traditional site. And it's a lot easier to just fact-check statements online than to wade headfirst into something you know nothing about in random sites of varying qualities.
- rerdavies 4y ago
  Just ask for links. The example below is contrived. But when you're looking for very specific information, GptChat can save you from wading through pages of Google Search results. Providing a list of 30,000 pages that are only tangentially related to what you want is not a virtue. ----- > Give me a link to page that gives an introduction to quantum computing. Here is a link to a page that provides an introduction to quantum computing: https://www.ibm.com/quantum-computing/learn/what-is-quantum-computing/ https://www.ibm.com/quantum-computing/learn/what-is-quantum-... This page gives a brief overview of what quantum computing is, how it works, and some of the potential applications. It also includes links to more in-depth resources for those who want to learn more. I hope this helps! Let me know if you have any other questions. > Give me another link. Here is another link that provides an introduction to quantum computing: https://www.dwavesys.com/tutorials/getting-started-with-quantum-computing https://www.dwavesys.com/tutorials/getting-started-with-quan... This page gives a high-level overview of quantum computing, including the basics of quantum mechanics and how it is used in computing. It also discusses the potential applications of quantum computing and some of the challenges and limitations of the technology. I hope this helps! Let me know if you have any other questions. ...
hooande 4y ago
The biggest problem with ChatGPT is that the training data is frozen in 2021. I would imagine that there have been robot vacuums released in the last year. I wouldn't search for this using ChatGPT because I know that the answers will be out of date. It's the equivalent of limiting a google search to have an end date in Dec 2021. If I'm buying something, I want to know what the recent options are
- Izkata 4y ago
  > I would imagine that there have been robot vacuums released in the last year. A really specific example that has an affirmative, not just in a basic release but a leap forward: The beginning of 2022 is when Roborock released their robot vacuum dock that empties, refills, and cleans the mop on their top robot vacuum/mop combo.
Aicy 4y ago
You are making a mistake. ChatGPT does not care about truth, but just giving an answer that sounds like what you'd like to hear the most. So replacing SEO blog spam with ChatGPT is not an improvement. Blog spam optimises for SEO rules, ChatGPT optimises to sound like legitimate (but wrong) blog spam.
- Enginerrrd 4y ago
  Honestly, I agree with what you're saying, but there are alarm bells in my head going off saying that this sounds suspiciously like what the school librarians and teachers used to say against wikipedia. And despite its flaws, Wikipedia is usually the first place I stop for knowledge.
  - yosame 4y ago
    The reason why Wikipedia is such a great first stop for knowledge is because it has citations you can follow. With ChatGTP you have to take the information on pure trust, and the information it gives is not at a level you can trust yet.
    - drewcon 4y ago
      Why couldn’t it also offer citations then. That sounds ideal actually.
      - pixl97 4y ago
        Are we sure the citations were not written by Chatgpt too?
        statheros 4y ago
        >> Explain briefly what the Fourier Transform is. Provide sources with more information about the topic. >> The Fourier transform is a mathematical technique that decomposes a function or signal into its constituent frequencies. It allows us to represent a function in terms of the frequencies that make it up, rather than in terms of its time or spatial behavior. The Fourier transform has many applications in a variety of fields, including signal and image processing, engineering, and physics. It is a fundamental tool in the analysis of periodic phenomena and is widely used in the study of waves and oscillations. For more information about the Fourier transform, you may find the following resources helpful: Wikipedia: https://en.wikipedia.org/wiki/Fourier_transform https://en.wikipedia.org/wiki/Fourier_transform Khan Academy: https://www.khanacademy.org/math/differential-equations/laplace-transform/fourier-series-diff-eq/v/fourier-series https://www.khanacademy.org/math/differential-equations/lapl... Wolfram MathWorld: http://mathworld.wolfram.com/FourierTransform.html http://mathworld.wolfram.com/FourierTransform.html
vikp 4y ago
I've had the exact same problem, and have experimented with Kagi, DDG, etc, to try to find better results. ChatGPT, in my opinion, is great for "how do I code X" type questions, but isn't so good at the types of queries you mentioned, due to the lack of a search engine. My weekend project was an open source combination of Google + GPT that returns pretty good results for these types of queries. You can check it out here - https://github.com/VikParuchuri/researcher https://github.com/VikParuchuri/researcher Example - the response to "what are the best current smartphones" is: `...According to Search Result [2], the best phones have been thoroughly reviewed and tested, and include the Apple iPhone 14 and 14 Pro, the Pixel 7 Pro and the Samsung Galaxy S22 Ultra. Search Result [5] also states that there are strong options available at all price levels, so you don't have to spend a lot to get something great...`
morsecodist 4y ago
What kinds of things do you query? I am genuinely curious. I have seen people say things like this about ChatGPT and it makes me wonder if we are using the same product. I have developed a hobby application with GPT-3 completion and I have ChatGPT more or less daily a week or so mostly for programing related queries. I have found the error rate to be extremely high, like > 20%. Not just sub-optimal but producing code that does not work, in python, which is a very common language. It is really tempting to use it for things like regexes or time format strings (the latter it does do well IMO) because often you want something too specific to google. When it works it's great but my trust in it has been shaken and I don't see myself using it long term.
- scarface74 4y ago
  I had a JSON file with the following format: [{“Key”:”Company”},{“Value”:”${company}”},…] I asked it to write a Python script that replaces any word surrounded by ${} with the value in its corresponding environment variable and accept the path of the file as a command line argument —json-file using argparse. It worked perfectly. Then I started asking it to write a script to successively do the following Given the same json file, write a snippet of YML that looks like sample CF templates parameter section that I gave it. https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui... It worked perfectly. Then I told it to accept an optional argument that generated the corresponding meta data section. It worked flawlessly. I gave it sample expected output https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudformation-interface.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui... Then I told it to output the format needed to pass those parameters to a nested stack and gave it an example; Parameters: Company: !Ref company … It worked again. Finally, I needed it to generate the Python code to generate the CodeBuild Environment section and I gave it an example. https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-codebuild-project-environmentvariable.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...
rish1_2 4y ago
I already Q&A site like StackOverflow to correct misinformation or just to share: https://www.gptoverflow.link/ https://www.gptoverflow.link/
pleb_nz 4y ago
I use kagi. I reluctantly paid the monthly fee, but by Jebs it's worth it to get pure accurate results with no ads and filters.
ilaksh 4y ago
Get an OpenAI API key. Then `npm i -g askleo` and set env var OPENAI_API_KEY `> askleo how large is the average dog` `> runleo list files in reverse date order`
more_corn 4y ago
The answer is probably AI middleware (something to stand between you and the LLM and help tune your prompts). We did this for a hackathon recently and it worked quite well. We also open-sourced OpenPlayground so you can test prompt tuning against multiple LLMs.

Thorentis 4y ago

How do you assess whether the information from ChatGPT is correct? It's downfall is that it very convincingly produces false information. At least in a Google search you can compare multiple sources, and blogspam is usually easy to filter out once you're good at identifying it. In ChatGPT, good and bad results look equally plausible.

trees101 4y ago
https://www.perplexity.ai https://www.perplexity.ai This augments ChatGPT results with Bing results to provide references you can look up.
heliophobicdude 4y ago
I started asking in a follow up for it to write a set of test cases. I could tweak and run them to my liking. I think for knowledge based things, you are better off using Google. But if it can help you narrow down your search, you can use that for Google. For example, I queried, what projects are good for practicing data-intensive design. It gave me a list of projects with descriptions. Perfect for me to Google more information for. It's very good for narrowing down search and addressing follow ups. It's like the perfect Google Search companion.
- oomrazoom 4y ago
  you.com Offers a chat feature that pairs the gpt-3 response with search results
CSMastermind 4y ago
Lots of time when I'm searching I'm relying on recognition and not recall. I know the right answer but I don't have it memorized. If I search for python for loop syntax or metallica bob seger cover or tuck rule game year, etc. I know the right answer if you show it to me and given enough time to think I probably could recall it but I don't have the information at the top of my mind. If chatgpt returns something that's wrong I'll know it's wrong and then maybe go check google.
WalterBright 4y ago
Some people have tried asking ChatGPT to do some D coding. The results looked like plausible D code, but were subtly wrong.
0x6c6f6c 4y ago
> it very convincingly produces false information This is exactly the scenario I encountered. I have a teammate who is using ChatGPT to ask questions as opposed to searching documentation and it gave incorrect info all around on all the following: 1. Where the property would be found in the event schema 2. What structure the value would be 3. The fact that it's not included at all for AWS AppSync specifically in question, even if the previous two had been answered correctly regarding API Gateway event schema. All of this was available in the first doc returned in the search results of both API Gateway and AppSync respectively. The fact it convincingly tells you otherwise causes additional friction since they're pointing at their screen telling me I'm wrong cause look at the response from ChatGPT.
- deleted 4y ago
  [deleted]
- nprateem 4y ago
  I think solving this inaccuracy will be openai's biggest challenge. The web is safe for now. Once they do fix it it's basically game over.
  - ilaksh 4y ago
    You can turn the temperature down on the OpenAI API and it is much more accurate. They also have a code-specific model which I don't think ChatGPT is using.
- moritonal 4y ago
  But, you have to admit that if ChatGPT had a process whereby it'd internally list the sites most likely relevant to your question (essentially Google the keywords) then included those sites as context it'd likely ace your question? Basically, if Google owned ChatGPT the result would be very different.
- spaceman_2020 4y ago
  Whenever chatGPT gives me a function and I try it out and it doesn’t work, I just tell it “hey this isn’t working” and it will correct itself
lumost 4y ago
How do you do this with google search when half the results are spammed copies of the same content? For many cases you can get enough information out of chatGPT to retrieve a specific corroborating reference. For coding tasks, you can just try the code, ask it for tests, or give it code review like feedback. I’d suggest asking it about deep technical topics that you are familiar with, then work out it’s limitations. It’s effectively equivalent to a universal teaching assistant, sometimes wrong - usually points in the right direction.
- vel0city 4y ago
  The answer is still the same, I evaluate the source. Which is near impossible to truly know with bots like ChatGPT.
drusepth 4y ago
I just treat ChatGPT as another source. It's become a go-to for questions (according to my log, this week I've asked it about naturopathic medicine, game development, algorithms, FSAs, vocabulary, and general trivia). It's waaaaay faster to get an immediate answer -- and be able to ask pointed, follow-up questions, which you couldn't otherwise do with a traditional site. And it's a lot easier to just fact-check statements online than to wade headfirst into something you know nothing about in random sites of varying qualities.
rerdavies 4y ago
Just ask for links. The example below is contrived. But when you're looking for very specific information, GptChat can save you from wading through pages of Google Search results. Providing a list of 30,000 pages that are only tangentially related to what you want is not a virtue. ----- > Give me a link to page that gives an introduction to quantum computing. Here is a link to a page that provides an introduction to quantum computing: https://www.ibm.com/quantum-computing/learn/what-is-quantum-computing/ https://www.ibm.com/quantum-computing/learn/what-is-quantum-... This page gives a brief overview of what quantum computing is, how it works, and some of the potential applications. It also includes links to more in-depth resources for those who want to learn more. I hope this helps! Let me know if you have any other questions. > Give me another link. Here is another link that provides an introduction to quantum computing: https://www.dwavesys.com/tutorials/getting-started-with-quantum-computing https://www.dwavesys.com/tutorials/getting-started-with-quan... This page gives a high-level overview of quantum computing, including the basics of quantum mechanics and how it is used in computing. It also discusses the potential applications of quantum computing and some of the challenges and limitations of the technology. I hope this helps! Let me know if you have any other questions. ...

hooande 4y ago

The biggest problem with ChatGPT is that the training data is frozen in 2021. I would imagine that there have been robot vacuums released in the last year. I wouldn't search for this using ChatGPT because I know that the answers will be out of date. It's the equivalent of limiting a google search to have an end date in Dec 2021. If I'm buying something, I want to know what the recent options are

Izkata 4y ago
> I would imagine that there have been robot vacuums released in the last year. A really specific example that has an affirmative, not just in a basic release but a leap forward: The beginning of 2022 is when Roborock released their robot vacuum dock that empties, refills, and cleans the mop on their top robot vacuum/mop combo.

Aicy 4y ago

You are making a mistake. ChatGPT does not care about truth, but just giving an answer that sounds like what you'd like to hear the most. So replacing SEO blog spam with ChatGPT is not an improvement. Blog spam optimises for SEO rules, ChatGPT optimises to sound like legitimate (but wrong) blog spam.

Enginerrrd 4y ago
Honestly, I agree with what you're saying, but there are alarm bells in my head going off saying that this sounds suspiciously like what the school librarians and teachers used to say against wikipedia. And despite its flaws, Wikipedia is usually the first place I stop for knowledge.
- yosame 4y ago
  The reason why Wikipedia is such a great first stop for knowledge is because it has citations you can follow. With ChatGTP you have to take the information on pure trust, and the information it gives is not at a level you can trust yet.
  - drewcon 4y ago
    Why couldn’t it also offer citations then. That sounds ideal actually.
    - pixl97 4y ago
      Are we sure the citations were not written by Chatgpt too?
      - statheros 4y ago
        >> Explain briefly what the Fourier Transform is. Provide sources with more information about the topic. >> The Fourier transform is a mathematical technique that decomposes a function or signal into its constituent frequencies. It allows us to represent a function in terms of the frequencies that make it up, rather than in terms of its time or spatial behavior. The Fourier transform has many applications in a variety of fields, including signal and image processing, engineering, and physics. It is a fundamental tool in the analysis of periodic phenomena and is widely used in the study of waves and oscillations. For more information about the Fourier transform, you may find the following resources helpful: Wikipedia: https://en.wikipedia.org/wiki/Fourier_transform https://en.wikipedia.org/wiki/Fourier_transform Khan Academy: https://www.khanacademy.org/math/differential-equations/laplace-transform/fourier-series-diff-eq/v/fourier-series https://www.khanacademy.org/math/differential-equations/lapl... Wolfram MathWorld: http://mathworld.wolfram.com/FourierTransform.html http://mathworld.wolfram.com/FourierTransform.html

vikp 4y ago

I've had the exact same problem, and have experimented with Kagi, DDG, etc, to try to find better results. ChatGPT, in my opinion, is great for "how do I code X" type questions, but isn't so good at the types of queries you mentioned, due to the lack of a search engine. My weekend project was an open source combination of Google + GPT that returns pretty good results for these types of queries. You can check it out here - https://github.com/VikParuchuri/researcher https://github.com/VikParuchuri/researcher Example - the response to "what are the best current smartphones" is: `...According to Search Result [2], the best phones have been thoroughly reviewed and tested, and include the Apple iPhone 14 and 14 Pro, the Pixel 7 Pro and the Samsung Galaxy S22 Ultra. Search Result [5] also states that there are strong options available at all price levels, so you don't have to spend a lot to get something great...`

morsecodist 4y ago

What kinds of things do you query? I am genuinely curious. I have seen people say things like this about ChatGPT and it makes me wonder if we are using the same product. I have developed a hobby application with GPT-3 completion and I have ChatGPT more or less daily a week or so mostly for programing related queries. I have found the error rate to be extremely high, like > 20%. Not just sub-optimal but producing code that does not work, in python, which is a very common language. It is really tempting to use it for things like regexes or time format strings (the latter it does do well IMO) because often you want something too specific to google. When it works it's great but my trust in it has been shaken and I don't see myself using it long term.

scarface74 4y ago
I had a JSON file with the following format: [{“Key”:”Company”},{“Value”:”${company}”},…] I asked it to write a Python script that replaces any word surrounded by ${} with the value in its corresponding environment variable and accept the path of the file as a command line argument —json-file using argparse. It worked perfectly. Then I started asking it to write a script to successively do the following Given the same json file, write a snippet of YML that looks like sample CF templates parameter section that I gave it. https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui... It worked perfectly. Then I told it to accept an optional argument that generated the corresponding meta data section. It worked flawlessly. I gave it sample expected output https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudformation-interface.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui... Then I told it to output the format needed to pass those parameters to a nested stack and gave it an example; Parameters: Company: !Ref company … It worked again. Finally, I needed it to generate the Python code to generate the CodeBuild Environment section and I gave it an example. https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-codebuild-project-environmentvariable.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...

rish1_2 4y ago

I already Q&A site like StackOverflow to correct misinformation or just to share: https://www.gptoverflow.link/ https://www.gptoverflow.link/

pleb_nz 4y ago

I use kagi. I reluctantly paid the monthly fee, but by Jebs it's worth it to get pure accurate results with no ads and filters.

ilaksh 4y ago

Get an OpenAI API key. Then `npm i -g askleo` and set env var OPENAI_API_KEY `> askleo how large is the average dog` `> runleo list files in reverse date order`

more_corn 4y ago

The answer is probably AI middleware (something to stand between you and the LLM and help tune your prompts). We did this for a hackathon recently and it worked quite well. We also open-sourced OpenPlayground so you can test prompt tuning against multiple LLMs.