Not being snarky, but at that scale why would you still be using Digital Ocean? Are you processing this dataset with just a single droplet? Are you going to ke

Not being snarky, but at that scale why would you still be using Digital Ocean? Are you processing this dataset with just a single droplet? Are you going to keep each months 10 TiB forever? Something like Nearline is going to be a much better fit even if you process the data and have to pay the $.01/GB retrieval fee. Disclaimer: I work on compute engine (so of course I want your money), but I'm honestly curious.