3 ms·
If we could wave a medium-sized magic wand to make the database fast for tests (as least as fast as the proposed alternative), would we still use a single share
by sa46 5y ago
If we could wave a medium-sized magic wand to make the database fast for tests (as least as fast as the proposed alternative), would we still use a single shared database for all tests? I would use a separate database per test since I don't find the arguments about realistic data shape convincing.
Back the real world: since the database isn't fast, we can either make the test infrastructure faster or hoist the problem into the application domain by using tricks like the OP. I'm a platform engineer at heart so I lean towards making the infra faster in order to accelerate app development.
My hot take is that developers are too comfortable with slow test databases and we're due for an "esbuild moment". With appropriate elbow grease, some databases, like Postgres, can be quite speedy (~300ms to start a cluster for a suite and ~40ms per test). The main tricks:
- Avoid Docker. Postgres is cross platform and Docker on macOS is sloooow.
- Create a single database cluster (Postgres lingo). Initialize a fresh database in the cluster with the seed data. Use that database as a template for all future tests. Creating from a template was 12x faster than creating from scratch in a benchmark I ran.
- Use a ram-backed file system. Using /tmp is probably good enough since its backed by tmpfs which should use RAM for small files.
- Disable fsync for the database.
I wrote up a more detailed take for Postgres here: https://news.ycombinator.com/item?id=28472062 https://news.ycombinator.com/item?id=28472062.
- dataflow 5y agoI'm not sure what distro you use but /tmp isn't generally tmpfs (like on Ubuntu for example), so you might want to double check before relying on that.
- sa46 5y agoNice catch, that was a neat rabbit hole [1]! I run an Ubuntu derivative so you're correct that /tmp is not mounted using tmpfs. Looks like Arch and Fedora use tmpfs and the Debian flavors do not. [1]: https://lwn.net/Articles/499410/ https://lwn.net/Articles/499410/