3 ms·
We don't talk about Google "not having a moat" since their patents on PageRank expired. All the talk of OpenAI's moats (or lack thereof) since the memo, seems
by ben_w 3y ago
We don't talk about Google "not having a moat" since their patents on PageRank expired.
All the talk of OpenAI's moats (or lack thereof) since the memo, seems like humans being stochastic parrots.
- ldjkfkdsjnv 3y agoGoogle has a data moat on traditional search. Its really hard to train a traditional search system without google scale data. LLMs effectively side stepped that whole issue
- ben_w 3y agoGoogle's special sauce, is the data of all user's responses to the things Google's algorithm shows them. This description also works for ChatGPT, what it has which others do not (at anything close to the same scale). Google's search engine is a well understood bit of matrix multiplication on a scale many others can easily afford to replicate. This description also works for GPT-3.
- ldjkfkdsjnv 3y agoNo becuause ChatGPT was trained on the whole internet using a new algorithm. You needed perfect tagged data for the old algorithm (page rank). LLMs are a fundamental leap forward
- ben_w 3y agoThe GPT algorithm isn't a secret. The RLFH data — users giving a thumbs up or down, or regenerating a response, or even getting detectably angry in their subsequent messages — is. PageRank isn't a secret. Google's version of RLFH — which link does a user click on, do they ignore the results without clicking any, do they write several related queries before clicking a link, do they return to the search results soon after clicking a link — are also secret. That the Transformer model is a breakthrough doesn't make it a moat; that the Transformer model is public doesn't mean people using it don't have a moat. Hence why I'm criticising the use of "moat" as mere parroting.
- rchaud 3y agoGoogle's acquisitions are its moat. Google Search is only one small part of that Google wouldn't be the company it is if it didn't also control Youtube, Gmail, Android and Chrome.