3 ms·
That content has extreme temporal properties. An image that's uploaded is increasingly less likely to be accessed over time. You can probably store it on very c
by staticassertion 4y ago
That content has extreme temporal properties. An image that's uploaded is increasingly less likely to be accessed over time. You can probably store it on very cheap hardware with pretty serious compression.
Not to mention that it's mostly text and images that get uploaded to these sites. Videos are the worst case. Let's say there are 1 billion facebook accounts, each one uploads 1GB of data (seems huge), and let's assume that compression just cancels out replication.
That's 1,000 petabytes. On S3 that's in the low 6 figures per year, obviously ignoring the exfil/access costs.
That's not that much. Obviously you want to keep some of it "hot" - profile pictures, recently uploaded pictures, etc. Hand waving, assuming 1% of the data needs to stay hot (seems high), that's 10PB of data. Certainly you're in "big data" land but it's not like there aren't databases that'll handle it.
Not to mention FB invests in tech like zstd.
Cold storage is definitely still getting much cheaper. Dropbox SMR drives are pretty crazy:
https://dropbox.tech/infrastructure/smr-what-we-learned-in-our-first-year https://dropbox.tech/infrastructure/smr-what-we-learned-in-o...