4 ms·
One scaling paradigm that's occurred to me for SQLite would be to run lots of small replicated instances in a system like Nomad or Kubernetes and even allow man
by api 4y ago
One scaling paradigm that's occurred to me for SQLite would be to run lots of small replicated instances in a system like Nomad or Kubernetes and even allow many running on the same physical node. So you're replicating multiple times on the same box which wastes some space but storage is cheap.
Wouldn't be good for huge data sets but these days huge means like a terabyte or more. You don't really have to start thinking about sharding or smart fractional replication until you hit many terabytes.
Even many successful SaaS companies don't have databases that big. Several hundred gigs these days is not too big for a brute force replicate everywhere approach and it comes with some benefits like less locking and stronger fault tolerances. If you have 50 replicas the odds of them all being screwed at once gets very low.