4 ms·

Pgtemp: The easiest way to write tests with Postgres in Rust, without Docker

54 points by ben_s 3y ago

peter_l_downs 3y ago
Interesting idea to implement a proxy that does something on each new connection. I’ll have to think about that but very cool concept that lets this work for tests written in any language. Will each testcase need to run migrations after connecting for the first time? Also, I’m used to using clients which lazily pool connections, do they still work with this model? Op, I’d be curious what you think of my project pgtestdb, which takes a different approach: use a single server, but create new databases each time. Maybe there is some middle ground where a proxy creates a new cloned db after each connection? https://github.com/peterldowns/pgtestdb https://github.com/peterldowns/pgtestdb
- thom 3y ago
  I assume the idea here is similar to other embedded Postgres libraries - you create a template database that is very fast to copy on startup (or you manually copy the correct files which is even faster) so you don’t need to run migrations. In our tests on one of the JVM equivalents the overhead is about 20ms per test which is entirely tolerable. Making this language agnostic is really interesting though.
  - peter_l_downs 3y ago
    Yeah, the parent project mentions that you can use pg_dump and pg_restore, but the examples don't seem to show that (or I'm misreading them.) My project uses template databases to do exactly what you said. Definitely fast and definitely nice.
deleted 3y ago
[deleted]
cpursley 3y ago
Cool, much appreciated (esp no Docker). Added to test section in my "Postgres Is Enough gist: https://gist.github.com/cpursley/c8fb81fe8a7e5df038158bdfe0f06dbb#unit-tests https://gist.github.com/cpursley/c8fb81fe8a7e5df038158bdfe0f...
pmkelly4444 3y ago
extremely useful
rodorgas 3y ago
Ideally, integration tests should not touch a real database. That would be more of an end-to-end test, that could be actually monitoring. But mocking things are hard on Rust because of its strong type system, so this project can be useful.
dkpk 3y ago
Interesting. Is there something similar for Node.js?
mnahkies 3y ago
I made a similar (less features) docker based thing https://github.com/mnahkies/ephemeral-postgres https://github.com/mnahkies/ephemeral-postgres - in my instance I was just sick of creating users/databases with the right names when hopping between projects. In general I expect test suites to put things into the expected starting state, though I can see the appeal of giving each suite it's own isolated state since that makes parallelism easier

54 points by ben_s 3y ago

Interesting idea to implement a proxy that does something on each new connection. I’ll have to think about that but very cool concept that lets this work for tests written in any language. Will each testcase need to run migrations after connecting for the first time? Also, I’m used to using clients which lazily pool connections, do they still work with this model? Op, I’d be curious what you think of my project pgtestdb, which takes a different approach: use a single server, but create new databases each time. Maybe there is some middle ground where a proxy creates a new cloned db after each connection? https://github.com/peterldowns/pgtestdb https://github.com/peterldowns/pgtestdb

thom 3y ago
I assume the idea here is similar to other embedded Postgres libraries - you create a template database that is very fast to copy on startup (or you manually copy the correct files which is even faster) so you don’t need to run migrations. In our tests on one of the JVM equivalents the overhead is about 20ms per test which is entirely tolerable. Making this language agnostic is really interesting though.
- peter_l_downs 3y ago
  Yeah, the parent project mentions that you can use pg_dump and pg_restore, but the examples don't seem to show that (or I'm misreading them.) My project uses template databases to do exactly what you said. Definitely fast and definitely nice.

deleted 3y ago

[deleted]

cpursley 3y ago

Cool, much appreciated (esp no Docker). Added to test section in my "Postgres Is Enough gist: https://gist.github.com/cpursley/c8fb81fe8a7e5df038158bdfe0f06dbb#unit-tests https://gist.github.com/cpursley/c8fb81fe8a7e5df038158bdfe0f...

pmkelly4444 3y ago

extremely useful

rodorgas 3y ago

Ideally, integration tests should not touch a real database. That would be more of an end-to-end test, that could be actually monitoring. But mocking things are hard on Rust because of its strong type system, so this project can be useful.

dkpk 3y ago

Interesting. Is there something similar for Node.js?

mnahkies 3y ago

I made a similar (less features) docker based thing https://github.com/mnahkies/ephemeral-postgres https://github.com/mnahkies/ephemeral-postgres - in my instance I was just sick of creating users/databases with the right names when hopping between projects. In general I expect test suites to put things into the expected starting state, though I can see the appeal of giving each suite it's own isolated state since that makes parallelism easier