3 ms·
This doesn't make sense. They did invest in PaLM, which is ~2.5 larger than GPT-3, but no where near 100x better. I do agree that the limiting factor is the dat
by 0XYZxyzXYZ0 4y ago
This doesn't make sense. They did invest in PaLM, which is ~2.5 larger than GPT-3, but no where near 100x better. I do agree that the limiting factor is the data, not scale though. What data has Google that OpenAI (or any other LLM company) can't scrape from the web as well? Using Gmail data without consent would be definitely a move towards a mountain of law suits.
- CuriouslyC 4y agoGoogle has a lot of scanned/ocr'd books that aren't available in public data sets. I wouldn't be surprised if they used anonymized gmail data though. Even if they removed all proper nouns it'd probably provide a lot of extra tokens for training.
- flangola7 4y agoPaLM isn't public, there's no way to know if it is better or if so how much better it is.