4 ms·
512GB is probably good enough for 99%. 4TB is probably good enough for 99.9%. Context for all this: I lead tech at Remind, where we support 30M MAU with around
by phamilton 4y ago
512GB is probably good enough for 99%. 4TB is probably good enough for 99.9%.
Context for all this: I lead tech at Remind, where we support 30M MAU with around 1 engineer per million MAU. Years ago we focused heavily on horizontally scaling, including our datastores. We stored a few hundred GB in clustered redis, dozens of TB in dynamoDB, a half dozen postgres clusters, etc.
The past year we've reversed course and doubled down on AWS Aurora. Every time we've moved data into Aurora our stability has improved, our devs can move faster and our costs go down.
We've got an order of magnitude headroom in Aurora and frankly our code is far from well optimized. There's so much to be gained from simplicity.