5 ms·
The title of this piece is great: very catchy. But I don't think the content supports the title - by the end of it I wasn't at all clear /why/ the database ruin
by simonw 5y ago
The title of this piece is great: very catchy. But I don't think the content supports the title - by the end of it I wasn't at all clear /why/ the database ruins all good ideas.
A few other points. First, horizontally scaling database reads is actually reasonably straight-forward these days: one leader, multiple replicas, load balance reads to the replicas and use a mechanism such that users that have just performed a write have their read traffic sent to the leader for the next 10 seconds or so.
Not trivial, but also not impossibly difficult - plenty of places implement this without too much trouble.
Scaling writes is a lot harder - but a well specc'd relational database server will handle tens of thousands of writes per second, so the vast majority of projects will never have to solve this.
When you do need to solve this, patterns for horizontally sharding your data exist. They're not at all easy to implement, but it's not an impossible problem either.
The article talks briefly about mocking your database: definitely never do this. How your database behaves should be considered part of your application code under test. Running a temporary database for your tests is a solved problem for most development frameworks these days (Django supports this out of the box).
Overall, my experience is that the database /enables/ all good ideas. Building stateful applications without a relational database in the mix is usually the wrong choice.
- tremon 5y agohorizontally scaling database reads is actually reasonably straight-forward these days: one leader, multiple replicas There's one feature that I'd like to see in that area: partitioned writable replicas. In the same vein that you can partition the table storage across an index, I'd like it to be possible to assign different writers to different parts of a table/database. Of course, you'd still need a single primary replica to handle the transactions that transcend the configured partitions, but we already have routing engines that can transparently redirect an incoming query to any available replica, so it's partly there. There's probably corner cases lurking that I can't even think of, but in my mind it's the only thing missing from building a truly web-scale (multiple zones, multiple datacenters) ACID-preserving relational database.
- taffer 5y agoThis is essentially the idea of VoltDB [1]: All tables are partitioned and each partition is assigned to a single thread. As long as each query only touches a single partition, you get linear horizontal scalability and serializable isolation. The drawback is that if a query touches multiple partitions, only that one query can be executed on the entire cluster, so this architecture only works for very specific OLTP workloads. [1] https://www.voltdb.com/ https://www.voltdb.com/
- klodolph 5y ago> There's one feature that I'd like to see in that area: partitioned writable replicas. This is getting more common these days. Google’s Spanner database does this. The database is divided into “tablets” which are each given their own Paxos state. Spanner moves tablets between server instances to do load balancing. There’s no need for you to do the partitioning manually, although if you run into performance problems, you may need to understand how tablets work. Spanner’s definitely not unique in doing this, but Spanner does have a nice writeup of how it works: https://static.googleusercontent.com/media/research.google.com/en//archive/spanner-osdi2012.pdf https://static.googleusercontent.com/media/research.google.c... (PDF)
- Andys 5y agoCheckout TiDB. It sounds too good to be true, and it is, because transactions must be co-ordinated across multiple nodes on the network. Thus the write throughput can be very high but the write latency is also very high.
- derekperkins 5y agoVitess does just that, letting you seamlessly scale writes in the same way you would scale reads today, scaling linearly with additional nodes.
- Negitivefrags 5y agoI’ve manually implemented this approach before and I would be a little scared of doing it automatically. When you do it manually you are very aware of queries that need to go to multiple shards and really do whatever it takes to avoid them. We sharded by account id and all an individual users data would be on that shard. Basically the only queries that need to transcend the shard are when you need to enumerate accounts by some non-ID value like an email address. You have to ask all the shards in that case. Things like creating an account are tricky if you want to make sure that each has a unique email address. You need to use two-phase commit across all the databases to do it reliably.
- AmericanChopper 5y agoI think I better title would be “reality reveals that ideas were actually bad”. The issues described in this blog are about the challenges of persisting a consistent set of data, when you need to support a lot of read and write access. That is just a fundamentally difficult problem. Databases are the most sophisticated tools we have for doing that, but really they just tend to be a set of (hopefully) well implemented trade offs that attempt to nicely balance all of those conflicting requirements. If a person can’t find a way to adequately persist their data, then their idea is probably bad, or they just don’t have the competence to implement it. An equally suitable title for this post could have been “why doesn’t magic exist”.
- twhitmore 5y agoExactly. Consistently and reliably storing business data is something that actually turns out to be important in the real world. If you can't actually retain your business data, it really doesn't matter how many "good ideas" or fancy deployment strategies for the presentation tier you have.
- DougN7 5y ago> The article talks briefly about mocking your database: definitely never do this I honestly thought you meant to never ‘ridicule’ your database, and I was thinking “that might be a healthy level of respect” :)
- mikeyjk 5y agoBut what if the data setup is convoluted? How does one avoid a horrific domino stack of fixtured data?
- simonw 5y agoI find using libraries like factory_boy and pytest's fixtures mechanism makes that horrific domino stack pretty manageable.
- sonthonax 5y agoMan I fucking hate factory boy, by default it fuzzes the input meaning your tests are non deterministic. Honesty, I don’t understand why people use it; how many models do people have where meta programming factory classes makes sense?
- yowlingcat 5y agoI think it comes down to whether you want to maintain a fixture or a factory. In my experience, it can be easy to set up a fixture but over time it increasingly becomes a pain -- sometimes enough that setting up a factory is worth it.
- noisy_boy 5y ago> The article talks briefly about mocking your database: definitely never do this. How your database behaves should be considered part of your application code under test. Running a temporary database for your tests is a solved problem for most development frameworks these days (Django supports this out of the box). Spring Boot for Java does this too (generally using H2 in memory database). I have found that for CRUD services, doing this vs. pure unit tests that mock the db, yields far better benefit (provided you don't involve heavy volumes in the tests to avoid test data maintenance issues). With this approach, I have found database side edge cases/constraint violations that wouldn't be caught by mocking the database. Also, it is a much more comprehensive test if you take the approach of a) loading an initial state, b) doing business logic mutations and c) compare the final state with the expected state; e.g. you might find side effects that you hadn't considered testing for and so on.
- heurisko 5y agoI always go further and use the same database for my tests e.g. use postgres rather than H2. It is easy enough to setup and tear down using docker.
- danielheath 5y agoAfter a bug slipped through because of it, I switched CI to run the same point release of the database, and validate that the schema is identical to production.
- noisy_boy 5y agoThat is a good idea. Though H2 supports specifying the database dialect so it is a pretty close and usually quicker approximation.
- heurisko 5y agoIf speed is one of the things holding you back from testing with the native database, running the database data on a tmpfs mountpoint is worth looking at. https://vladmihalcea.com/how-to-run-integration-tests-at-warp-speed-with-docker-and-tmpfs/ https://vladmihalcea.com/how-to-run-integration-tests-at-war...
- cornel_io 5y agoScaling writes is theoretically really easy via sharding, and I've done it several times. Unfortunately in typical web stacks these days ORMs absolutely do not support sharding in any reasonable way, even if your data model supports it easily (users don't collaborate so you can 100% segregate by user ID and never need cross-shard joins except to static tables). Agreed re: db mocks, database bugs and quirks should be captured and addressed in tests, not discovered only in production. I don't care how mature the DB you're using is, there's always going to be some edge case behavior that you can't predict until you actually see it happen, even just simple stuff like exactly how an error filters back to the application when you send data that violates a constraint. I've seen way too many tests that make assumptions about these things that turn out to be incorrect, leading to horrible behavior like writes getting through on prod that should have been blocked because of pre-checks that didn't turn out to be enforced the way the test writer assumed they would be.
- MakersF 5y ago> The article talks briefly about mocking your database: definitely never do this. I definitely recommend doing this for same tests: it's the only way to check how your application behaves in case of failures. The best thing would be to have a DB where you can inject failures, but I'm not aware of any. So test - the happy path on the real db - the sad path on the mock/fake (which might be a light wrapper around a real db, but with the ability of injecting failures)
- simonw 5y agoThat's true: the one place where it's a good idea to mock the database is when you want to simulate what would happen if a specific database error occurred that can't be triggered another way.