4 ms·

What are the chances that their copy is more useful than CommonCrawl? It’s been the target of SEO for decades - are they even capable of separating the good con

• sitkack 3y ago
  Data pollution isn't the problem it is made out to be. You can even use a model trained on garbage data to relabel the original set and retrain. Google of all companies is sitting on exaflops of TPU capacity, they can retrain models larger than anyone else, faster than anyone else.
  • visarga 3y ago
    I agree, but it would probably require to use LLMs to filter or augment the raw text. They can afford it.
• chaxor 3y ago
  Their data may not be that much better; however Deepmind is still one of the best institutions that study AI fundamentals. OpenAI works hard and focused towards products, but the academic research is far better from Deepmind. So since Google hasn't been trying to put any AI into products really at all for the past 10 years, they're going to have to push a little bit to make the public facing interfaces and put polish on things, but the underlying models they have made are far better than OpenAI (just not the ones made publicly accessible to anyone).
  • visarga 3y ago
    They have no model on par with GPT-4.
    • chaxor 3y ago
      I would heavily disagree with that - the only reason that you think that is that GPT-4 is available to the public for cheap, whereas Deepminds models have not been released publicly.
• fomine3 3y ago
  For websites that blocks all robots.txt-compliant bots except Google, Bing and some, Googlebots can crawl but CommonCrawl bots can't