4 ms·
I'll bite despite not being the most qualified person around to answer the question :) First of all. It's well considered to think about alternatives to NoSQL.
by brandur 10y ago
I'll bite despite not being the most qualified person around to answer the question :)
First of all. It's well considered to think about alternatives to NoSQL. You lose _so much_ correctness and resilience by moving to systems like Mongo and Cassandra that throw ACID out the window. At some point of scale you're going to need to loosen your data constraints to allow further growth, but most people aren't actually going to get to that point, and of those left over, practicing some basic pragmatic data austerity may allow you to avoid "webscale" databases altogether.
Regarding Dynamo: I've used it on fairly large projects at two companies now. It _mostly_ works as advertised, but its nature means that you're constantly fighting the system if you ever need to access data in a way that you hadn't originally considered. You can scan relatively cheaply along an index, but if you have any other predicate condition that needs to be added, you'll have to fall back to seq scanning the table. This is obviously slow, but also costly because it'll have a huge impact on your provisioned read throughput.
On both projects we eventually had to think about moving off on Dynamo because of serious troubles with ingestion bottlenecks. I wasn't directly involved in either project so I have a little trouble remembering the exact details, but both problems were essentially related to the partitioning balance of incoming data. You can provision as much write throughput as you want, but if you're data isn't balanced perfectly by partition key as it comes in, single shards can become exhausted and put severe pressure on the entire system.
Regarding Citus: I still haven't used it directly, but Postgres is obviously a great system, and Citus seems to be a nice augmentation for its functionality. The fact that it can guarantee atomicity, consistency, and isolation within the bounds of a single shard is potentially huge for allowing you to maintain incredible robustness even at scale.
I had a casual conversation with a CloudFlare engineer once wherein I asked them about the dirt, but they didn't give me any and seemed to be fairly content with the product.
Regarding Cassandra: again, no direct experience. I guess this is where you eventually need to go if you scale out far enough, but it seems like an incredible pain to setup, build with, and manage.