5 ms·

Search without LLM/GPT is just dead in the water now, isn't it?

_fat_santa 4y ago
I can't decide if LLM/GPT is a seismic shift in the "search industry" or if it's just a gimmick. For now I'm considering it more of a gimmick that everyone is just very very excited about.
- aatd86 4y ago
  Definitely not a gimmick. A lot of questions I would have googled, I now ask ChatGPT. It's not always reliable of course but it's often quicker and somewhat more extensive and specific when I need code snippets for example. I think it's complimentary technologies. It will definitely eat up search market share. It's almost as if yesteryear encyclopedias were having their revenge.
- _pjee 4y ago
  If it's useful it isn't a gimmick.
- ChicagoBoy11 4y ago
  I think the problem is the popular media coverage makes it hard to allow people to actually focus on the value this actually brings. I work in ED and its astounding how many people fundamentally think basically the school is going to shutdown imminently (clearer heads are prevailing... for now). But the tool clearly has some value, which may even reveal itself to be quite significant. I think a useful metric is to look at what happened with Copilot, which is a domain where there has been a lot less media frenzy about, and in which, arguably, this kind of model could've have much more readily made a tremendous impact. I think even in the dev community we went through a small period of folks thinking this was going to be earthshattering, followed by a natural cooldown, followed by a probably much saner interpretation that it is a tool, and in the right contexts might actually be useful.
  - _fat_santa 4y ago
    yeah I think you're spot on. I don't think GPT is earth shattering but it's also hard to tell with all the media frenzy about it.
- CSSer 4y ago
  I've always wondered: Is LLM/GPT already how Google is generating answers to "People also ask" questions or are they using a prior, or unknown, ML approach for this? I ask this because the quality of answers linked to these queries is often so astoundingly awful that I can't believe this is in production. The question is often very relevant. I'll readily admit that I have a high engagement with that accordion feature, but I can't believe how often I open it to find a disappointing text selection or even page. And if it is, I wonder where that leaves JSON-LD schema in all of this. Schema is the perfect signal for something like this, but I'm afraid, and I believe I can speak representatively about this from an agency perspective, the trust is kind of broken for that model. Too many people, myself included, are uneasy about how much information to give to Google since they have an insidious aim to use it to get information to users faster regardless of how much impact it has on a business' ability to remain competitive. Yet, on the other hand, I sympathize with the idea that the more Google reverses course on this and leans into embracing SEO industry-driven control they risk compromising the product. It makes me think that Google has reached a theoretically maximum level of product optimization.
  - gardenhedge 4y ago
    I think Google has built up a knowledge base that they query for those answers. And yes, it's embarrassingly wrong so much of the time
_pjee 4y ago
It depends on your goal. If the focus of your search engine is question answering (like Google), ChatGPT is serious competition. Browsing the web to discover online communities and niche websites is an entirely different activity that ChatGPT doesn't compete with, and frankly Google doesn't do well either.
- vczf 4y ago
  I asked ChatGPT for webcomic recommendations and it gave me imaginary ones. No trace of them anywhere on the web.
- marginalia_nu 4y ago
  Yeah discovery is in such a messy state right now. I don't think anyone is doing it well. "Give me facts about X" there are several decent options for. "Give me quality reading material about topic X", there's like nobody even trying. Not that an answer doesn't exist for the query, just nobody is able to produce it. This goes well beyond websites, discovery sucks for streaming video, for shopping, almost everything.
  - _pjee 4y ago
    I think one issue is broadness. When a user issues a query there is often a choice of 1) low quality material that directly answers the specific question, or 2) high quality material about the subject more broadly. If someone asks "how do I fix water in camera iphone 3", if someone has set up a content farm for that exact question Google will probably send the user there, rather than to a broader repair page from apple.com. There's tradeoffs in search ranking, which is why I think it's possible to make a search engine that takes the opposite side of the tradeoffs that Google has. Nice job on marginalia.nu btw. Your site is a good reminder that there is a whole internet out there that just isn't able to rank on Google for various reasons.
randomsolutions 4y ago
Search is one of the least interesting applications of LLMs. Most of the complaints about search are self imposed, not technical problems. Why are we still talking about a 20 year old problem that is basically solved?
- etrautmann 4y ago
  Because it’s not solved for users?

I can't decide if LLM/GPT is a seismic shift in the "search industry" or if it's just a gimmick. For now I'm considering it more of a gimmick that everyone is just very very excited about.

aatd86 4y ago
Definitely not a gimmick. A lot of questions I would have googled, I now ask ChatGPT. It's not always reliable of course but it's often quicker and somewhat more extensive and specific when I need code snippets for example. I think it's complimentary technologies. It will definitely eat up search market share. It's almost as if yesteryear encyclopedias were having their revenge.
_pjee 4y ago
If it's useful it isn't a gimmick.
ChicagoBoy11 4y ago
I think the problem is the popular media coverage makes it hard to allow people to actually focus on the value this actually brings. I work in ED and its astounding how many people fundamentally think basically the school is going to shutdown imminently (clearer heads are prevailing... for now). But the tool clearly has some value, which may even reveal itself to be quite significant. I think a useful metric is to look at what happened with Copilot, which is a domain where there has been a lot less media frenzy about, and in which, arguably, this kind of model could've have much more readily made a tremendous impact. I think even in the dev community we went through a small period of folks thinking this was going to be earthshattering, followed by a natural cooldown, followed by a probably much saner interpretation that it is a tool, and in the right contexts might actually be useful.
- _fat_santa 4y ago
  yeah I think you're spot on. I don't think GPT is earth shattering but it's also hard to tell with all the media frenzy about it.
CSSer 4y ago
I've always wondered: Is LLM/GPT already how Google is generating answers to "People also ask" questions or are they using a prior, or unknown, ML approach for this? I ask this because the quality of answers linked to these queries is often so astoundingly awful that I can't believe this is in production. The question is often very relevant. I'll readily admit that I have a high engagement with that accordion feature, but I can't believe how often I open it to find a disappointing text selection or even page. And if it is, I wonder where that leaves JSON-LD schema in all of this. Schema is the perfect signal for something like this, but I'm afraid, and I believe I can speak representatively about this from an agency perspective, the trust is kind of broken for that model. Too many people, myself included, are uneasy about how much information to give to Google since they have an insidious aim to use it to get information to users faster regardless of how much impact it has on a business' ability to remain competitive. Yet, on the other hand, I sympathize with the idea that the more Google reverses course on this and leans into embracing SEO industry-driven control they risk compromising the product. It makes me think that Google has reached a theoretically maximum level of product optimization.
- gardenhedge 4y ago
  I think Google has built up a knowledge base that they query for those answers. And yes, it's embarrassingly wrong so much of the time

_pjee 4y ago

It depends on your goal. If the focus of your search engine is question answering (like Google), ChatGPT is serious competition. Browsing the web to discover online communities and niche websites is an entirely different activity that ChatGPT doesn't compete with, and frankly Google doesn't do well either.

vczf 4y ago
I asked ChatGPT for webcomic recommendations and it gave me imaginary ones. No trace of them anywhere on the web.
marginalia_nu 4y ago
Yeah discovery is in such a messy state right now. I don't think anyone is doing it well. "Give me facts about X" there are several decent options for. "Give me quality reading material about topic X", there's like nobody even trying. Not that an answer doesn't exist for the query, just nobody is able to produce it. This goes well beyond websites, discovery sucks for streaming video, for shopping, almost everything.
- _pjee 4y ago
  I think one issue is broadness. When a user issues a query there is often a choice of 1) low quality material that directly answers the specific question, or 2) high quality material about the subject more broadly. If someone asks "how do I fix water in camera iphone 3", if someone has set up a content farm for that exact question Google will probably send the user there, rather than to a broader repair page from apple.com. There's tradeoffs in search ranking, which is why I think it's possible to make a search engine that takes the opposite side of the tradeoffs that Google has. Nice job on marginalia.nu btw. Your site is a good reminder that there is a whole internet out there that just isn't able to rank on Google for various reasons.

randomsolutions 4y ago

Search is one of the least interesting applications of LLMs. Most of the complaints about search are self imposed, not technical problems. Why are we still talking about a 20 year old problem that is basically solved?

etrautmann 4y ago
Because it’s not solved for users?