7 ms·

The moat is there until someone releases (or leaks) comparable training data.

• droopyEyelids 3y ago
  And that gets more difficult every day, as previously accesible sources of data turn off their api. though google may have something up its sleeve with the corpus of google books! I have been wondering if openAI secretly pulled in scihub or zlibrary to neutralize that potential advantage.
  • londons_explore 3y ago
    Are there any stats on how many words are in google books, vs how many words are on the open web? My feeling is that the web has a lot more on it than the total of all libraries - simply because anyone can start a blog, but publishing a book requires quite some commitment.
    • droopyEyelids 3y ago
      I think you're right but I also think the text in published books would be at least an order of magnitude more valuable than the same length of text from the web