4 ms·

> by emulating the Postgres server in an x86 emulator > because performance is not usually a concern in tests I hate to be a downer, especially because it sou

> by emulating the Postgres server in an x86 emulator > because performance is not usually a concern in tests I hate to be a downer, especially because it sounds like a lot of complex work has gone into this, but I can't disagree more with test performance. I think for many projects test performance is more important than runtime performance – a 100ms request time is unlikely to cause issues for most applications, but a 10 minute test run can significantly hamper engineering productivity. I don't understand why, as others have suggested, running Postgres in a RAM disk isn't an option. We did this on my previous team, tuned Postgres for better performance in tests, and ran all unit tests against it, the performance was excellent. Setup was trivial, still containerised like this project aims for. Why run Postgres on an X86 emulator, in a WASM VM, in a Docker container... when one could just run Postgres in a Docker container and have all the same advantages? Someone commented that you could run it in a browser... but Postgres doesn't run in the browser normally so why would you need that for tests?

thibaut_barrere 3y ago
If you use languages that compile to WASM (such as Gleam https://gleam.run https://gleam.run), and can also run Postgres via WASM, then it opens very interesting offline scenarios with codebases which are similar on both the client and the server, for instance.
- danpalmer 3y ago
  To be clear, this is if the client is a browser. If the "client" is a Linux machine you can just run a Docker container, or a full VM, or a Firecracker VM, no need for special languages or running software in a WASM VM. If it's a desktop/laptop you can just run a regular Postgres binary in an app (like Postgres.app does). If it's a phone you definitely don't want to be running the extra overhead of a persistent database process emulated on WASM, or something as heavyweight as Postgres. I've heard many people talk about "interesting offline scenarios", but I've yet to see an application do them properly. Some embed SQLite to good effect, but I've not yet seen anything running something like an embedded Postgres, and other than it being a clever trick that might be fun to implement, I'm struggling to think of a use-case.
pmarreck 3y ago
Your tests, or at least your unit tests, should run extremely quickly (on the order of a maximum of a few seconds). If you have a minute+ long test suite, you have already failed IMHO. The utility of a test suite goes down quite steadily IMHO as the run-time goes up. The road to a slow test suite is reached in very small increments over time, so you have to be fastidious about all of it in order to stave that off as long as possible. A simple optimization that I've seen double some test suite speeds is to swap out slow password hash algorithms on login with a no-op, but just for the test environment. If running your DB slower significantly slows down your test suite then it's arguable that you are touching the DB too much already and repeatedly testing the same things over and over again (like creating users in the database for each test instead of using something in-memory)
- danpalmer 3y ago
  You're right that minutes matter here, and so does correctness. Most applications with a database depend strongly on that database layer to provide various guarantees and correctness. Personally, I think having a database and then tuning it for performance is the best option, followed by not having a database and being faster. Having the database but then running it in an emulator on a VM on your infra (which is probably itself a VM), as in this case, seems like a bad call.
deleted 3y ago
[deleted]

thibaut_barrere 3y ago

If you use languages that compile to WASM (such as Gleam https://gleam.run https://gleam.run), and can also run Postgres via WASM, then it opens very interesting offline scenarios with codebases which are similar on both the client and the server, for instance.

danpalmer 3y ago
To be clear, this is if the client is a browser. If the "client" is a Linux machine you can just run a Docker container, or a full VM, or a Firecracker VM, no need for special languages or running software in a WASM VM. If it's a desktop/laptop you can just run a regular Postgres binary in an app (like Postgres.app does). If it's a phone you definitely don't want to be running the extra overhead of a persistent database process emulated on WASM, or something as heavyweight as Postgres. I've heard many people talk about "interesting offline scenarios", but I've yet to see an application do them properly. Some embed SQLite to good effect, but I've not yet seen anything running something like an embedded Postgres, and other than it being a clever trick that might be fun to implement, I'm struggling to think of a use-case.

pmarreck 3y ago

Your tests, or at least your unit tests, should run extremely quickly (on the order of a maximum of a few seconds). If you have a minute+ long test suite, you have already failed IMHO. The utility of a test suite goes down quite steadily IMHO as the run-time goes up. The road to a slow test suite is reached in very small increments over time, so you have to be fastidious about all of it in order to stave that off as long as possible. A simple optimization that I've seen double some test suite speeds is to swap out slow password hash algorithms on login with a no-op, but just for the test environment. If running your DB slower significantly slows down your test suite then it's arguable that you are touching the DB too much already and repeatedly testing the same things over and over again (like creating users in the database for each test instead of using something in-memory)

danpalmer 3y ago
You're right that minutes matter here, and so does correctness. Most applications with a database depend strongly on that database layer to provide various guarantees and correctness. Personally, I think having a database and then tuning it for performance is the best option, followed by not having a database and being faster. Having the database but then running it in an emulator on a VM on your infra (which is probably itself a VM), as in this case, seems like a bad call.

deleted 3y ago

[deleted]