3 ms·

Wow - this explains a lot about why Chinese LLMs and AI struggle so much to get data, despite the supposed near-infinite databases: a lot of it is just gone, fa

by gwern 2y ago

Wow - this explains a lot about why Chinese LLMs and AI struggle so much to get data, despite the supposed near-infinite databases: a lot of it is just gone, far more than anyone had ever proposed might be the case, because of neglect, censorship, walled gardens/apps killing everything trapped within, and chilling effects. You can't scrape what is no longer there, or was never written to begin with...

rfoo 2y ago
> supposed near-infinite databases: a lot of it is just gone Pretty sure megacorps still have their near-infinite databases around, in walled garden as always, but that's another topic. Censored contents are almost certainly soft-deleted. The problem is more about independent-ish sites, for example Tianya [1]. Think ancient BBS. Or newsgroups. If nearly all archive of these are lost it would indeed be very sad. [1] https://en.wikipedia.org/wiki/Tianya_Club#cite_note-4 https://en.wikipedia.org/wiki/Tianya_Club#cite_note-4
- telotortium 2y ago
  > Pretty sure megacorps still have their near-infinite databases around Not necessarily. It's quite possible that, if no one accesses them, they got moved to backups with only one copy, and the backups may not actually be restorable in the case of disaster (it happens somewhat frequently even for backups of data you might actually care about, so imagine the effects of neglect on that).
  - gwern 2y ago
    There's also serious coordination problems internally. How easy is it to get access? Did you make buddies with the little emperors in charge of the historical archives? Is the archive in your company large enough? Are the chances of getting access to another competing company's dataset high enough to not require multiple decimal digits to express? (This I think is part of why Chinese human genomics has been such a disappointment: yes, there's collectively a lot of data, but there ain't nothing like UKBB pulling it all together. Just thousands of fiefdoms.)

by gwern 2y ago

rfoo 2y ago

> supposed near-infinite databases: a lot of it is just gone Pretty sure megacorps still have their near-infinite databases around, in walled garden as always, but that's another topic. Censored contents are almost certainly soft-deleted. The problem is more about independent-ish sites, for example Tianya [1]. Think ancient BBS. Or newsgroups. If nearly all archive of these are lost it would indeed be very sad. [1] https://en.wikipedia.org/wiki/Tianya_Club#cite_note-4 https://en.wikipedia.org/wiki/Tianya_Club#cite_note-4

telotortium 2y ago
> Pretty sure megacorps still have their near-infinite databases around Not necessarily. It's quite possible that, if no one accesses them, they got moved to backups with only one copy, and the backups may not actually be restorable in the case of disaster (it happens somewhat frequently even for backups of data you might actually care about, so imagine the effects of neglect on that).
- gwern 2y ago
  There's also serious coordination problems internally. How easy is it to get access? Did you make buddies with the little emperors in charge of the historical archives? Is the archive in your company large enough? Are the chances of getting access to another competing company's dataset high enough to not require multiple decimal digits to express? (This I think is part of why Chinese human genomics has been such a disappointment: yes, there's collectively a lot of data, but there ain't nothing like UKBB pulling it all together. Just thousands of fiefdoms.)