3 ms·

Pretty much this. If Google wanted - they could've easily released ChatGPT version that is 100x better than current one considering they invented the technolog

• 0XYZxyzXYZ0 4y ago
  This doesn't make sense. They did invest in PaLM, which is ~2.5 larger than GPT-3, but no where near 100x better. I do agree that the limiting factor is the data, not scale though. What data has Google that OpenAI (or any other LLM company) can't scrape from the web as well? Using Gmail data without consent would be definitely a move towards a mountain of law suits.
  • CuriouslyC 4y ago
    Google has a lot of scanned/ocr'd books that aren't available in public data sets. I wouldn't be surprised if they used anonymized gmail data though. Even if they removed all proper nouns it'd probably provide a lot of extra tokens for training.
  • flangola7 4y ago
    PaLM isn't public, there's no way to know if it is better or if so how much better it is.
• epistemer 4y ago
  That is a huge assumption. The simplest explanation to me is that while they have larger language models they don't have a better product than chatGPT to release. I would think building that product is what this $400m represents. The impressive thing with chatGPT to me is how well it understand what you want even with very sparse input. Even if it gives wrong answers it still feels like you are both on the same page. That seems like the secret sauce even if a larger language model would give more correct output. I wouldn't be shocked at all if Google doesn't have anything currently that feels the way chatGPT does when it comes to interaction and now they are racing to build exactly that.