3 ms·

There seem to be two different directions for innovation here. The first is a little more mundane: LLM embeddings. OpenAI currently offers an API that turns se

There seem to be two different directions for innovation here. The first is a little more mundane: LLM embeddings. OpenAI currently offers an API that turns sentences into coordinates for a point in some 1536-dimensional conceptual space such that two points are close together if they are conceptually close together. This is insanely powerful. For example, you can generate captions for a bunch of images and store the embeddings for them. Then, you can look for a "picture of a rabbit eating a carrot" by turning that phrase into a 1536-dimensional point and looking for the nearest points around it. Basically, it blows open search technology for everyone. You no longer have to deal with synonyms, idiomatic phrases that mean similar things, misspellings etc - the problems you'd run into when trying to implement simple text search using traditional techniques. It all gets simplified to generating coordinates in some hyperspace and looking for nearest neighbors. This is a total game changer. The second direction is ChatGPT. Sure, if you want to read a detailed analysis of the demographic situation in China, you'd prefer an article written by an expert. You would still use a search engine, pick a search result and do things the way you do them today. However, there's an entire collection of things that can be answered directly by ChatGPT. For example "how many mins should I hard boil an egg" or "Can I take NyQuil when I'm stoned" or anything else where you really just want a single sentence answer. Today, you launch a browser, search for what you want, skip past the first 10 advertisements, look for a site that seems reasonably reputable, click through all the GDPR warnings, scroll past the banner ads and the SEO optimizing bullshit text to find that one sentence that you wanted all along. Or, you could ask ChatGPT and get an answer instantly. (assuming chatGPT is good enough eventually). It's hard to predict which of these two technologies will disrupt the current status quo in search. Neither might. But we haven't ever been this close to a level playing field in search since the 1990s. The excitement is hard to resist.

karpierz 4y ago
> The first is a little more mundane: LLM embeddings. You know Google has been doing this for years now? > However, there's an entire collection of things that can be answered directly by ChatGPT. For example "how many mins should I hard boil an egg" or "Can I take NyQuil when I'm stoned" or anything else where you really just want a single sentence answer. Google has been doing this for years via search cards, which are AI generated summaries of website information.
- dilippkumar 4y ago
  > You know Google has been doing this for years now? Of course. What’s changed is that before now, they were the only ones who could do it, vs now, everyone can do it. So this technology only got deployed where someone could get a promo out of it whereas now, every to-do list app, dating app, and even a reasonably sophisticated nigerian prince can find a place to deploy it. Think of what gcc did to software tool chains, Apache to servers, Linux to operating systems and to a lesser degree, blockchains to distributed databases. > Google has been doing this for years via search cards, which are AI generated summaries of website information. Yes. But now, someone else can do it too. And if that someone else does a good job, Google just lost the opportunity to show advertisements to all of those “searches”. Doesn’t mean that anyone is about to beat Google in terms of sheer talent and experience with this stuff. But a very hungry and determined community of entrepreneurs just got hands on something that’s about as good as Google’s secret sauce and they’re about to run wild.
danShumway 4y ago
So LLM embedding are an actual useful thing that could actually improve search. Categorization is a real problem that AI could help solve (particularly with search queries). But that's not a new category of search, it's just a question of whether the current LLMs would be better than whatever Google is currently using to make the same inferences. And the direction Bard and Bing seem to be going with these giant models is the conversational direction, and where that's concerned: > there's an entire collection of things that can be answered directly by ChatGPT. [...] assuming chatGPT is good enough eventually I am a lot less impressed with this. And I know I'm an outlier and plenty of people are shocked at how good GPT is at this kind of problem, so I am constantly second-guessing myself and thinking to myself, "are we using the same product?" Because I think ChatGPT produces really bad quality information. It's cool, it's wildly impressive, it's a massive achievement and an incredible milestone for AI, but 'cool' is different from 'useful.' Leaving aside the problem that answering simple questions is a very small subset of what search is used for, and isn't on its own probably a big enough category of questions to make me change search engines, the bigger problem is that the current state of ChatGPT seems to be wildly inconsistent about what it knows and what it doesn't know and I don't have a way to pre-predict what categories of information it's safe to ask about. And the only way for me to verify the answers it gives me are to... double check its work with a real search. I would not advise anyone to ask ChatGPT for advice about what drugs are safe to take while high, that seems profoundly unwise to me. So it's a bit like Instant Answers. Google has been trying to auto-answer questions for ages, and in practice the only time it's ever been useful for me is when it's extremely predictable and when I know that a category of question will only ever have its answer pulled from one site and where I know what the format of that answer will be. Unpredictability is generally a quality that I try to avoid any time that I am using a computer. One of the primary strengths of a computer to me is specificity and predictability. And so the bar here is really high. The question I ask myself is, "would I want to replace a search engine with a human assistant?" And I think the answer is no, I feel like that would be missing the point of what a search engine is. And ChatGPT gives worse answers than a human assistant would, and its sources/knowledge is just as unpredictable as a human's would be if not worse. So, I also don't want to replace my search engine with ChatGPT. It could get more accurate in the future, and if it does then maybe my opinion will change then, but... it's hard for me to get excited about using a worse product today on the promise that it might get better in the future. And I guess it's accurate enough that a bunch of people keep telling me that they're saving time when they use it, so maybe I don't understand what I'm talking about. But I just don't see how people are reaching that conclusion unless they're either asking questions where they don't actually care about the accuracy or unless they're just rolling the dice and trusting that ChatGPT won't accidentally poison them when they ask what drug combinations they can take.

karpierz 4y ago

> The first is a little more mundane: LLM embeddings. You know Google has been doing this for years now? > However, there's an entire collection of things that can be answered directly by ChatGPT. For example "how many mins should I hard boil an egg" or "Can I take NyQuil when I'm stoned" or anything else where you really just want a single sentence answer. Google has been doing this for years via search cards, which are AI generated summaries of website information.

dilippkumar 4y ago
> You know Google has been doing this for years now? Of course. What’s changed is that before now, they were the only ones who could do it, vs now, everyone can do it. So this technology only got deployed where someone could get a promo out of it whereas now, every to-do list app, dating app, and even a reasonably sophisticated nigerian prince can find a place to deploy it. Think of what gcc did to software tool chains, Apache to servers, Linux to operating systems and to a lesser degree, blockchains to distributed databases. > Google has been doing this for years via search cards, which are AI generated summaries of website information. Yes. But now, someone else can do it too. And if that someone else does a good job, Google just lost the opportunity to show advertisements to all of those “searches”. Doesn’t mean that anyone is about to beat Google in terms of sheer talent and experience with this stuff. But a very hungry and determined community of entrepreneurs just got hands on something that’s about as good as Google’s secret sauce and they’re about to run wild.

danShumway 4y ago

So LLM embedding are an actual useful thing that could actually improve search. Categorization is a real problem that AI could help solve (particularly with search queries). But that's not a new category of search, it's just a question of whether the current LLMs would be better than whatever Google is currently using to make the same inferences. And the direction Bard and Bing seem to be going with these giant models is the conversational direction, and where that's concerned: > there's an entire collection of things that can be answered directly by ChatGPT. [...] assuming chatGPT is good enough eventually I am a lot less impressed with this. And I know I'm an outlier and plenty of people are shocked at how good GPT is at this kind of problem, so I am constantly second-guessing myself and thinking to myself, "are we using the same product?" Because I think ChatGPT produces really bad quality information. It's cool, it's wildly impressive, it's a massive achievement and an incredible milestone for AI, but 'cool' is different from 'useful.' Leaving aside the problem that answering simple questions is a very small subset of what search is used for, and isn't on its own probably a big enough category of questions to make me change search engines, the bigger problem is that the current state of ChatGPT seems to be wildly inconsistent about what it knows and what it doesn't know and I don't have a way to pre-predict what categories of information it's safe to ask about. And the only way for me to verify the answers it gives me are to... double check its work with a real search. I would not advise anyone to ask ChatGPT for advice about what drugs are safe to take while high, that seems profoundly unwise to me. So it's a bit like Instant Answers. Google has been trying to auto-answer questions for ages, and in practice the only time it's ever been useful for me is when it's extremely predictable and when I know that a category of question will only ever have its answer pulled from one site and where I know what the format of that answer will be. Unpredictability is generally a quality that I try to avoid any time that I am using a computer. One of the primary strengths of a computer to me is specificity and predictability. And so the bar here is really high. The question I ask myself is, "would I want to replace a search engine with a human assistant?" And I think the answer is no, I feel like that would be missing the point of what a search engine is. And ChatGPT gives worse answers than a human assistant would, and its sources/knowledge is just as unpredictable as a human's would be if not worse. So, I also don't want to replace my search engine with ChatGPT. It could get more accurate in the future, and if it does then maybe my opinion will change then, but... it's hard for me to get excited about using a worse product today on the promise that it might get better in the future. And I guess it's accurate enough that a bunch of people keep telling me that they're saving time when they use it, so maybe I don't understand what I'm talking about. But I just don't see how people are reaching that conclusion unless they're either asking questions where they don't actually care about the accuracy or unless they're just rolling the dice and trusting that ChatGPT won't accidentally poison them when they ask what drug combinations they can take.