3 ms·
1. Shrink your data. That's just an absurd amount of data for a start-up. Even large organizations can't quickly work around too much data. Resource growth dire
by throwaway823882 5y ago
1. Shrink your data. That's just an absurd amount of data for a start-up. Even large organizations can't quickly work around too much data. Resource growth directly affects system performance and complexity and limits what you will be able to practically do with the data. You already have a million problems as a start-up, don't make another one for yourself by trying to find a clever solution when you can just get rid of the problem.
2. As a general-purpose alternative, I would use Backblaze. It's cheap and they know what they're doing. Here is a comparison of (non-personal) cloud vendor storage prices: https://gist.github.com/peterwwillis/83a4636476f01852dc2b6703b78941e9 https://gist.github.com/peterwwillis/83a4636476f01852dc2b670...
3. You need to know how the architecture impacts the storage costs. There are costs for incoming traffic, outgoing traffic, intra-zone traffic, storage costs, archive costs, 'access' costs (cost per GET | POST | etc). You may end up paying $500K a month just to serve files smaller than 1KB.
4. You need to match up availability and performance requirements against providers' guarantees, and then measure a real-world performance test over a month. Some providers enforce rate limits, with others you might be in a shared pool of rate limits.
5. You need to verify the logistics for backup and restore. For 10PB you're gonna need an option to mail physical drives/tapes. Ensure that process works if you want to keep the data around.
6. Don't become your own storage provider. Unless you have a ton of time and money and engineering talent to waste and don't want to ship a reliable product soon.