7 ms·

Is AI the solution to search's problem?

21 points by lucaserb 4y ago

lucaserb 4y ago
Using OpenAI to research and learn faster online
- Tagbert 4y ago
  How reliable is the information that it returns?
  - peakay 4y ago
    sometimes I ask it if it's sure it completely changes its mind. you have to really scrutinize the response. this is to say it's not very reliable.
    - lardolan 4y ago
      From my experience same like wiki - good place to start your research path, but not best to end it.
swiftpoll 4y ago
Very interesting! And the bear-themed solution is cute! I would love to use such a tool to summarize and to ELI5 things like employment contracts or fine prints on a bank statement on the web. This tool might save folks a lot of time and energy.
tracerbulletx 4y ago
I think the next major leap will be when it accurately recommends real world books, videos, excerpts, papers, and products that naturally flow from your interaction.
baxtr 4y ago
Maybe I’m an outliner. But I’m not buying it. To me, it misses to analyze and address a fundamental problem of why google “got so bad”. I believe this is mainly a problem of the top 5%. There is reason why google presents their results like they do. It’s not like they’re a bunch of idiots not knowing their craft. Yes, for those of use around the internet since 1996, the results got worse. But since then the entire population got online. They’re just fine with the top results. They’re good enough. Google’s machine knows and sees this in the data. I am sure there is value in summarizing and querying the text. But this will increase the convenience not necessarily the quality of results.
- drpixie 4y ago
  Agree. But not sure what you mean by the "top 5%". Google have deliberately stopped trying to provide an index of the web. They now focus solely on revenue (advertising) maximization. A search will show paid sites, then popular sites, up to a few pages, and that's it. For example, search for "brioche buns" - you can't convince me that there are only 420 pages in the entire www that refer to "brioche buns" :(
  - charcircuit 4y ago
    >About 11,300,000 results I get a lot more than 400.
    - deleted 4y ago
      [deleted]
    - throwaway1851 4y ago
      Those results aren’t real. If you actually try to retrieve them, you will find out that Google stops delivering unique results after a few pages. (Other search engines are the same in this respect.)
      - charcircuit 4y ago
        Yes, they are real. It will stop ranking them after a few pages because the results become worse and it becomes too expensive to rank.
sovnwnt 4y ago
First the LLM needs to know what it doesn't know, and it looks like except for the explicit limitations embedded by the developers it isn't capable of doing that. When I asked ChatGPT about its own model variants, it gave me wild answers for things that made sense but didn't exist. """ There are several variants of GPT-3 model, each fine-tuned for specific tasks or industries. Here are a few examples: - GPT-3 "davinci" is fine-tuned for creative and imaginative writing tasks, such as poetry, short stories, and song lyrics. - GPT-3 "curie" is fine-tuned for conversational and dialog generation. - GPT-3 "babbage" is fine-tuned for code generation and language understanding. - GPT-3 "einstein" is fine-tuned for question answering tasks. - GPT-3 "jules" is fine-tuned for summarization. - GPT-3 "parliament" is fine-tuned for legal text generation. - GPT-3 "bronte" is fine-tuned for creative writing tasks like storytelling and fiction-writing. """ I'd almost assume "parliament" it's a leak of some sort if any of the other examples (except for davinci) were accurate.
- corndoge 4y ago
  Another possibility is that these code names do exist internally within OpenAI and chatgpt is aware of them. Ah, i guess that's what you meant by leak. I think some of the others are accurate - babbage in particular is very accurate!
acadapter 4y ago
Google search from 2010-2015 was a great tool. Google search today is a sad shadow of its former self.
friend_and_foe 4y ago
I think this line of reasoning is probably correct: these language models aren't intelligence, they're just the next iteration on information lookup systems.
danbulant 4y ago
This solves the problem of pages full of ads and keywords and irrelevant content. It doesn't solve the problem that's talked about in the article - there are other solutions, usually alternative search engines (kagi[0], or if more AI=more better, the kagi labs with contextual answers with sources[1]. Not sponsored, just like using them. Similar service offered by Hey.com, but I didn't try them much). [0]: https://kagi.com https://kagi.com [1]: https://labs.kagi.com/ai/contextai https://labs.kagi.com/ai/contextai
looseyesterday 4y ago
AI is not the solution. The root issue here is Goodheart's law. SEO marketers are hacking rankings with mediocre content, this leads to lower returns on time invested for generating valuable content, so writers / creators divert their attention elsewhere. Kagi.ai is a good example of how google could be fixed. Generative models dont know what they dont know, they will always struggle on new topics and worse still they wont provide feedback to the user to adjust their query as they will simply hallucinate details
AussieWog93 4y ago
Anecdotally, I've been using ChatGPT to do the legwork of copying-and-pasting from Stack Overflow, and it's honestly really quite good. Have asked it to write maybe a dozen programs, even asked it why a particular line of code won't compile, and it does a really good job of explaining how things work.
Barrin92 4y ago
No. Generative AI at least won't. ChatGPT is Borges Library of Babel, it produces any reasonably sounding sentence in existence if you ask it to, of course useful information by definition is somehow in there but so is everything else. If your problem is that too much crap is on the internet, well that crap is in ChatGPT, and then some. Even if you only get fiction 20% of the time that's still absolute poison for a search engine that sounds completely authoritative. It's like having sand on your toothbrush. The solution to abundance of bad info on the internet are two things. One, rudimentary search engine skills, i.e. filtering by source, keyword search and some basic literacy because all the good sources are still there and growing. Secondly, trusted sources curated by people that you have confidence in to produce high quality output. The discourse increasingly reminds me of Alexa/"smart" home assistants which people thought would replace everything because the natural language interaction seems futuristic.
fedux 4y ago
Ads will not go away. Wait until the free-tier model is fine-tuned to promote certain products in its output.
flohofwoe 4y ago
I totally read that as "Is AI a solution in search of a problem?" at first, but I guess that summarises the article pretty well too.
- whiplash451 4y ago
  Radioactivity was also in search of a problem when we found it.
  - flohofwoe 4y ago
    Which led to trailblazing innovations like radioactive toothpaste ;) https://en.wikipedia.org/wiki/Doramad_Radioactive_Toothpaste https://en.wikipedia.org/wiki/Doramad_Radioactive_Toothpaste
isaacfrond 4y ago
Honestly, 'google has gotten so bad', it is just a meme. It is just not true. Use any of the competing search engines for even a day and you'll come running back to Google. Believe me, I tried it. I really wanted to switch to Bing or the Duck, but they just don't compare. If the search results of Google are not to your liking it is because search is hard. Also login, if you are a super user, Google will catch on to this as well.
- to-too-two 4y ago
  I think the key is appending the word 'reddit' at the end of every search. Come to think of it, this feels like the precursor to ChatGPT since ChatGPT often feels like it responds with a Reddit comment that has been awarded and voted to the top anyway.
  - wildrhythms 4y ago
    Right, but the value in "site:reddit.com" is the human nature of the content: we trust that a human wrote the Reddit comment in good faith, and we trust that it was upvoted by people who are knowledgeable in that area of expertise. I fear we are losing that important element with ChatGPT; or worse, it will create a feedback loop where Reddit comments are "enhanced" with ChatGPT snippets and fed back into the system, and we're back at square one trying to find a way to cut through the noise.
    - dpkirchner 4y ago
      Adding site:reddit.com to searches improves the chances you'll get a result that is not totally wrong or spam because reddit has a built-in moderation system (upvotes/downvotes). The community also has little tolerance for SEO clone and other junk sites. It's not perfect -- there are definitely gamed topics and bullshitters -- but it's light years better than default Google. I don't think I've ever seen a clone of StackOverflow appear on an upvoted reddit post or comment while I see those sites every day on Google. I agree with you on the risks of that feedback loop. I'm hopeful that humans will still be around to moderate information on forums.
- 4y ago
LastTrain 4y ago
This isn't a solution to the search problem but a good option would be to add the ability to search by original index date. Google and DDG have operators for searching by date, but they are limited and seem to be looking for dates declared in the indexed documents, not on an internal date. One advantage to searching by original index date is that it cannot be manipulated by SEO's retroactively, so searches can be crafted to get results from before a time when SEOs started manipulating the topic you are interested in.

21 points by lucaserb 4y ago

lucaserb 4y ago

Using OpenAI to research and learn faster online

Tagbert 4y ago
How reliable is the information that it returns?
- peakay 4y ago
  sometimes I ask it if it's sure it completely changes its mind. you have to really scrutinize the response. this is to say it's not very reliable.
  - lardolan 4y ago
    From my experience same like wiki - good place to start your research path, but not best to end it.

swiftpoll 4y ago

Very interesting! And the bear-themed solution is cute! I would love to use such a tool to summarize and to ELI5 things like employment contracts or fine prints on a bank statement on the web. This tool might save folks a lot of time and energy.

tracerbulletx 4y ago

I think the next major leap will be when it accurately recommends real world books, videos, excerpts, papers, and products that naturally flow from your interaction.

baxtr 4y ago

Maybe I’m an outliner. But I’m not buying it. To me, it misses to analyze and address a fundamental problem of why google “got so bad”. I believe this is mainly a problem of the top 5%. There is reason why google presents their results like they do. It’s not like they’re a bunch of idiots not knowing their craft. Yes, for those of use around the internet since 1996, the results got worse. But since then the entire population got online. They’re just fine with the top results. They’re good enough. Google’s machine knows and sees this in the data. I am sure there is value in summarizing and querying the text. But this will increase the convenience not necessarily the quality of results.

drpixie 4y ago
Agree. But not sure what you mean by the "top 5%". Google have deliberately stopped trying to provide an index of the web. They now focus solely on revenue (advertising) maximization. A search will show paid sites, then popular sites, up to a few pages, and that's it. For example, search for "brioche buns" - you can't convince me that there are only 420 pages in the entire www that refer to "brioche buns" :(
- charcircuit 4y ago
  >About 11,300,000 results I get a lot more than 400.
  - deleted 4y ago
    [deleted]
  - throwaway1851 4y ago
    Those results aren’t real. If you actually try to retrieve them, you will find out that Google stops delivering unique results after a few pages. (Other search engines are the same in this respect.)
    - charcircuit 4y ago
      Yes, they are real. It will stop ranking them after a few pages because the results become worse and it becomes too expensive to rank.

sovnwnt 4y ago

First the LLM needs to know what it doesn't know, and it looks like except for the explicit limitations embedded by the developers it isn't capable of doing that. When I asked ChatGPT about its own model variants, it gave me wild answers for things that made sense but didn't exist. """ There are several variants of GPT-3 model, each fine-tuned for specific tasks or industries. Here are a few examples: - GPT-3 "davinci" is fine-tuned for creative and imaginative writing tasks, such as poetry, short stories, and song lyrics. - GPT-3 "curie" is fine-tuned for conversational and dialog generation. - GPT-3 "babbage" is fine-tuned for code generation and language understanding. - GPT-3 "einstein" is fine-tuned for question answering tasks. - GPT-3 "jules" is fine-tuned for summarization. - GPT-3 "parliament" is fine-tuned for legal text generation. - GPT-3 "bronte" is fine-tuned for creative writing tasks like storytelling and fiction-writing. """ I'd almost assume "parliament" it's a leak of some sort if any of the other examples (except for davinci) were accurate.

corndoge 4y ago
Another possibility is that these code names do exist internally within OpenAI and chatgpt is aware of them. Ah, i guess that's what you meant by leak. I think some of the others are accurate - babbage in particular is very accurate!

acadapter 4y ago

Google search from 2010-2015 was a great tool. Google search today is a sad shadow of its former self.

friend_and_foe 4y ago

I think this line of reasoning is probably correct: these language models aren't intelligence, they're just the next iteration on information lookup systems.

danbulant 4y ago

This solves the problem of pages full of ads and keywords and irrelevant content. It doesn't solve the problem that's talked about in the article - there are other solutions, usually alternative search engines (kagi[0], or if more AI=more better, the kagi labs with contextual answers with sources[1]. Not sponsored, just like using them. Similar service offered by Hey.com, but I didn't try them much). [0]: https://kagi.com https://kagi.com [1]: https://labs.kagi.com/ai/contextai https://labs.kagi.com/ai/contextai

looseyesterday 4y ago

AI is not the solution. The root issue here is Goodheart's law. SEO marketers are hacking rankings with mediocre content, this leads to lower returns on time invested for generating valuable content, so writers / creators divert their attention elsewhere. Kagi.ai is a good example of how google could be fixed. Generative models dont know what they dont know, they will always struggle on new topics and worse still they wont provide feedback to the user to adjust their query as they will simply hallucinate details

AussieWog93 4y ago

Anecdotally, I've been using ChatGPT to do the legwork of copying-and-pasting from Stack Overflow, and it's honestly really quite good. Have asked it to write maybe a dozen programs, even asked it why a particular line of code won't compile, and it does a really good job of explaining how things work.

Barrin92 4y ago

No. Generative AI at least won't. ChatGPT is Borges Library of Babel, it produces any reasonably sounding sentence in existence if you ask it to, of course useful information by definition is somehow in there but so is everything else. If your problem is that too much crap is on the internet, well that crap is in ChatGPT, and then some. Even if you only get fiction 20% of the time that's still absolute poison for a search engine that sounds completely authoritative. It's like having sand on your toothbrush. The solution to abundance of bad info on the internet are two things. One, rudimentary search engine skills, i.e. filtering by source, keyword search and some basic literacy because all the good sources are still there and growing. Secondly, trusted sources curated by people that you have confidence in to produce high quality output. The discourse increasingly reminds me of Alexa/"smart" home assistants which people thought would replace everything because the natural language interaction seems futuristic.

fedux 4y ago

Ads will not go away. Wait until the free-tier model is fine-tuned to promote certain products in its output.

flohofwoe 4y ago

I totally read that as "Is AI a solution in search of a problem?" at first, but I guess that summarises the article pretty well too.

whiplash451 4y ago
Radioactivity was also in search of a problem when we found it.
- flohofwoe 4y ago
  Which led to trailblazing innovations like radioactive toothpaste ;) https://en.wikipedia.org/wiki/Doramad_Radioactive_Toothpaste https://en.wikipedia.org/wiki/Doramad_Radioactive_Toothpaste

isaacfrond 4y ago

Honestly, 'google has gotten so bad', it is just a meme. It is just not true. Use any of the competing search engines for even a day and you'll come running back to Google. Believe me, I tried it. I really wanted to switch to Bing or the Duck, but they just don't compare. If the search results of Google are not to your liking it is because search is hard. Also login, if you are a super user, Google will catch on to this as well.

to-too-two 4y ago
I think the key is appending the word 'reddit' at the end of every search. Come to think of it, this feels like the precursor to ChatGPT since ChatGPT often feels like it responds with a Reddit comment that has been awarded and voted to the top anyway.
- wildrhythms 4y ago
  Right, but the value in "site:reddit.com" is the human nature of the content: we trust that a human wrote the Reddit comment in good faith, and we trust that it was upvoted by people who are knowledgeable in that area of expertise. I fear we are losing that important element with ChatGPT; or worse, it will create a feedback loop where Reddit comments are "enhanced" with ChatGPT snippets and fed back into the system, and we're back at square one trying to find a way to cut through the noise.
  - dpkirchner 4y ago
    Adding site:reddit.com to searches improves the chances you'll get a result that is not totally wrong or spam because reddit has a built-in moderation system (upvotes/downvotes). The community also has little tolerance for SEO clone and other junk sites. It's not perfect -- there are definitely gamed topics and bullshitters -- but it's light years better than default Google. I don't think I've ever seen a clone of StackOverflow appear on an upvoted reddit post or comment while I see those sites every day on Google. I agree with you on the risks of that feedback loop. I'm hopeful that humans will still be around to moderate information on forums.
4y ago

LastTrain 4y ago

This isn't a solution to the search problem but a good option would be to add the ability to search by original index date. Google and DDG have operators for searching by date, but they are limited and seem to be looking for dates declared in the indexed documents, not on an internal date. One advantage to searching by original index date is that it cannot be manipulated by SEO's retroactively, so searches can be crafted to get results from before a time when SEOs started manipulating the topic you are interested in.