3 ms·
The text of every article is easily worth archiving. If we take average article size to be 500-800 words, then 2KB each is plenty. You want to hang on to 10 b
by Dylan16807 2y ago
The text of every article is easily worth archiving. If we take average article size to be 500-800 words, then 2KB each is plenty. You want to hang on to 10 billion articles? That will cost you 1 (ONE) hard drive. Remember to make backups.
If we allocate 200KB per article for pretty strongly compressed images, then it's still 100 million on a single hard drive. Why not archive the whole lot and let God or future historians sort it out?
You only need to start getting picky when video is involved, but on the other hand when the alternative is total obliteration you can crush video down to 1MB per minute and have a tolerable VHS-like experience. And even 8MB Shrek gets the point across.
- robertlagrant 2y ago> when the alternative is total obliteration you can crush video down to 1MB per minute and have a tolerable VHS-like experience That's still pushing a terabyte a day[0] on YouTube alone. What about TikTok as well? [0] https://www.statista.com/statistics/259477/hours-of-video-uploaded-to-youtube-every-minute https://www.statista.com/statistics/259477/hours-of-video-up...
- estebank 2y agoSo, a single laptop hard drive a day? 50 tape drives a year? For the biggest repository of video there is? That's literally peanuts. There's some logistics in handling backup tapes, but that's a well understood process.
- Dylan16807 2y agoI think you undershot that by a factor of 60. But when I said "get picky" I meant things like not archiving all of youtube. And for articles specifically, archiving every video article is a lot smaller. https://routenote.com/blog/wp-content/uploads/2020/08/pex-youtube-analysis-2019-total-videos-tiers-700x475.png https://routenote.com/blog/wp-content/uploads/2020/08/pex-yo... And going by this, restricting to a reasonable view count will cut the space you need by a factor of 10 to 25. If we round that to a petabyte per year, it's not in the range of a typical personal archive, but it's a reasonable thing to picture several big libraries doing. (Though a single person could handle that much if they really wanted to, spending $10 a day on data tapes.)