4 ms·

I hope ChatGPT in its current form will not be used for search. As my friend says it, ChatGPT is not intelligent, it’s just capable of creating responses like i

I hope ChatGPT in its current form will not be used for search. As my friend says it, ChatGPT is not intelligent, it’s just capable of creating responses like it’s knows everything. The things it hallucinates is likely going to spread misinformation and make it harder for the masses to search for true, factual information. The other part is webtraffic: Google in theory could have created an interactive, conversational style search engine (with it without LLMs) if they wanted to, but a lot of websites would have complained about Google taking away traffic from them. I believe the same happened when Google started showing it’s own reviews instead of redirecting to Yelp. I wonder how openAI or any LLM powered search is going to deal with it. They don’t have to worry about it anytime soon, they still have a lot of time to get to a stage where they come anywhere close to the number of queries Google handles in a day, but it’ll be interesting to see how things go.

nathan_compton 4y ago
I agree that I'd still rather use a search engine over a small set of sites than ChatGPT for exactly the reasons you suggest and others. But I don't see ChatGPT as having a lot of utility beyond functioning as a search interface for credulous dummies. I mean if I were literally developing a chatbot then clearly its a pretty interesting technology (assuming its problems can be tamed or censored somehow), but beyond that I don't really get it.
- solidasparagus 4y ago
  The problem you are running into is that you are overindexing on the fact that LLMs will sometimes be wrong and you are used to using technology that is basically always right. But we are in the early stages of LLM adoption - correctness will improve (see for example citation driven LLM-search) but more importantly, the set of LLM-driven applications that can be probabilistically correct and still wildly useful will grow. LLMs like ChatGPT are just so damn cheap for the power they provide, it's inevitable
TeMPOraL 4y ago
Thing is, ChatGPT is already incredibly useful for searching random things you know enough about you can evaluate responses critically. The alternative here is doing a regular search, and wading through SEO-bloated, ad-laden content marketing "articles". The quality and reliability of information is about the same (or even favoring ChatGPT), but without 90% of the text that's just filler, without bullshit, ads, upsells, tracking scripts, etc. I tried it a few times and it's a much better experience than the web. I'm gonna be using it for as long as it lasts.
- nathan_compton 4y ago
  Yeah, but its not as reliable as just restricting your search to Wikipedia or the appropriate academic journals or even chatting with a librarian!
  - TeMPOraL 4y ago
    Sure, when the topic matters or I need to study it in depth, I can still go to Wikipedia or PubMed or Arxiv. But there are plenty of searches one does that are trivial, or serve to illuminate the problem space, and cover topics that in which I can rely on common sense to correct wrong advice. And the issue with non-technical topics, the kind applicable to mass audience - like e.g. cooking or parenting or hygiene - are very hard to search about online, because all results are bullshit pseudo articles written to drive traffic and deliver ads. So it's not that ChatGPT is so good, but more that Internet for normal people is complete trash, and ChatGPT nicely cuts straight through it.
    - b3morales 4y ago
      But if so this isn't because of its nature (the fact that it's an LLM), but because of its inputs. An LLM fed the same bullshit pseudo articles you refer to would likewise spit out more bullshit. If ChatGPT works it's because its sources have been carefully curated.
      - TeMPOraL 4y ago
albystein 4y ago
The problem of hallucination in LLMs is a well-known and studied problem and solutions have been proposed to counter it. The most promising one is augmenting LLMs with a retrieval system. This involves sourcing a large database of factual information, say journal articles, over which the LLM uses an information retrieval system(search engine) to extract information on which its generated output is conditioned. Recent job postings from OpenAI suggest that’s their next step of development for these LLMs. I think critics of these LLMs are missing the point about the excitement around them. People are excited because of the rate of progress/improvement from just two years or a year ago. These systems have come a long way, and if you extrapolate that progress into the future, I predict majority of these shortcomings getting resolved

nathan_compton 4y ago

I agree that I'd still rather use a search engine over a small set of sites than ChatGPT for exactly the reasons you suggest and others. But I don't see ChatGPT as having a lot of utility beyond functioning as a search interface for credulous dummies. I mean if I were literally developing a chatbot then clearly its a pretty interesting technology (assuming its problems can be tamed or censored somehow), but beyond that I don't really get it.

solidasparagus 4y ago
The problem you are running into is that you are overindexing on the fact that LLMs will sometimes be wrong and you are used to using technology that is basically always right. But we are in the early stages of LLM adoption - correctness will improve (see for example citation driven LLM-search) but more importantly, the set of LLM-driven applications that can be probabilistically correct and still wildly useful will grow. LLMs like ChatGPT are just so damn cheap for the power they provide, it's inevitable

TeMPOraL 4y ago

Thing is, ChatGPT is already incredibly useful for searching random things you know enough about you can evaluate responses critically. The alternative here is doing a regular search, and wading through SEO-bloated, ad-laden content marketing "articles". The quality and reliability of information is about the same (or even favoring ChatGPT), but without 90% of the text that's just filler, without bullshit, ads, upsells, tracking scripts, etc. I tried it a few times and it's a much better experience than the web. I'm gonna be using it for as long as it lasts.

nathan_compton 4y ago
Yeah, but its not as reliable as just restricting your search to Wikipedia or the appropriate academic journals or even chatting with a librarian!
- TeMPOraL 4y ago
  Sure, when the topic matters or I need to study it in depth, I can still go to Wikipedia or PubMed or Arxiv. But there are plenty of searches one does that are trivial, or serve to illuminate the problem space, and cover topics that in which I can rely on common sense to correct wrong advice. And the issue with non-technical topics, the kind applicable to mass audience - like e.g. cooking or parenting or hygiene - are very hard to search about online, because all results are bullshit pseudo articles written to drive traffic and deliver ads. So it's not that ChatGPT is so good, but more that Internet for normal people is complete trash, and ChatGPT nicely cuts straight through it.
  - b3morales 4y ago
    But if so this isn't because of its nature (the fact that it's an LLM), but because of its inputs. An LLM fed the same bullshit pseudo articles you refer to would likewise spit out more bullshit. If ChatGPT works it's because its sources have been carefully curated.
    - TeMPOraL 4y ago

albystein 4y ago

The problem of hallucination in LLMs is a well-known and studied problem and solutions have been proposed to counter it. The most promising one is augmenting LLMs with a retrieval system. This involves sourcing a large database of factual information, say journal articles, over which the LLM uses an information retrieval system(search engine) to extract information on which its generated output is conditioned. Recent job postings from OpenAI suggest that’s their next step of development for these LLMs. I think critics of these LLMs are missing the point about the excitement around them. People are excited because of the rate of progress/improvement from just two years or a year ago. These systems have come a long way, and if you extrapolate that progress into the future, I predict majority of these shortcomings getting resolved