Well, I think it comes down to how you define search. People are amazed and how good GPT-3 is at generating text, but looking at it from a search lens, you coul

Well, I think it comes down to how you define search. People are amazed and how good GPT-3 is at generating text, but looking at it from a search lens, you could argue that rather than "generating" it searches for the most relevant pieces to the input, combines them, and then enforces correct grammatical rules on the output. A recent example I saw on Twitter was someone asking GPT-3 questions about human evolution, and the model answers with a bunch of book references and links - that's something you would have done in Google before. The fact that the Transformer attention mechanisms works based on matching queries and keys/values is probably why the model is so good at finding relevant information to your input - that's kind of what it was trained to do.