4 ms·

> This seems to be quite a contradiction. If it's so easy to just write from scratch, then why would it be scary to depend on? It's easy to write an implementa

> This seems to be quite a contradiction. If it's so easy to just write from scratch, then why would it be scary to depend on? It's easy to write an implementation specific to your existing project's development environment and workflow correctly. It's hard to write a generic version that works in any environment. It's hard to write a version that lets you build a company and make money. It's scary to depend on this generic version because it's too generic, and it's built by a for-profit company now who wants to upsell you to some "testcontainer cloud" crap, which doesn't exactly incentivize them to make the OSS version perfect. For example, we were already using bazel, so writing a version using bazel to create a correct bare rootfs for a dependency + using runc to execute it in tests resulted in something with a roughly 3ms warm startup time that fulfilled all our needs, and cached correctly (since bazel has good caching). A junior engineer, starry-eyed at test-containers, switched some code over to it, and the warm startup time went up by 1000x from 3ms to 3s, as did the flake-rate, and docker's caching is far worse than bazel's so the several minute cold-starts also happened even more often. > Testcontainers is exactly the sort of thing open source is great for; it's something where everyone gets to benefit from the wisdom and battle-testing of everyone else. You get to have a mismatched mess of solutions to everyone's problems, including solution's to problems you don't have. You get code of the quality of the average OSS programmer which, while higher than the average programmer, is still pretty crap. Free Software is great when it's run by a small group of smart opinionated people. As soon as you try to build a company around that OSS and, god forbid, hire some enterprise sales people and product managers, it quickly becomes worse at the actual small developer problem than what even a mediocre developer could hack out in an hour. Depending on testcontainers is fine, but if you know what you're doing, writing something purpose-built is fine too, and probably gives you something much nicer in the end.

jchw 3y ago
To be honest, it doesn't sound like you really had a good use case for Testcontainers anyways. Where it excels the most is in just pulling in some external containers, especially databases, e.g. PostgreSQL and Redis, directly in your test harness. In those cases, it works well.
- TheDong 3y ago
  Our use-case was redis servers, one of your supposedly good use-cases for testcontainers. I didn't mention, but the testcontainer code also preferred to add that network io to the actual test runtime, which made measuring the test's performance harder, and meant we couldn't as safely cache the test's results. The bazel version made it easy to build the test's dependency as part of the test compilation process (as it should be) so the test runtime didn't have to do external network IO. "Excels" is also a stretch; we had a few hundred tests launching dedicated redis servers, and with hand-rolled code, that worked fine with zero flakes. With testcontainers, it regularly flaked with some opaque docker network error or sometimes just plain a timeout because apparently launching 100s of containers in parallel is a hard problem for docker or something. I'm sure it works well for some people, but if those people wanted to build out their own version without docker and testcontainers, specific to their development tooling and environment, it would probably work better in most cases I think.
  - jchw 3y ago
    Yeah, I couldn't tell you: I didn't have serious issues with performance or reliability when I used this. It didn't really seem like it was doing anything special, it seemed like it is just starting a Docker container like you'd expect, at least on Linux. I sincerely doubt I would've had any better experience handrolling it since my solution to handroll it would also be to use the Docker API; the reason for this is for portability.

jchw 3y ago

To be honest, it doesn't sound like you really had a good use case for Testcontainers anyways. Where it excels the most is in just pulling in some external containers, especially databases, e.g. PostgreSQL and Redis, directly in your test harness. In those cases, it works well.

TheDong 3y ago
Our use-case was redis servers, one of your supposedly good use-cases for testcontainers. I didn't mention, but the testcontainer code also preferred to add that network io to the actual test runtime, which made measuring the test's performance harder, and meant we couldn't as safely cache the test's results. The bazel version made it easy to build the test's dependency as part of the test compilation process (as it should be) so the test runtime didn't have to do external network IO. "Excels" is also a stretch; we had a few hundred tests launching dedicated redis servers, and with hand-rolled code, that worked fine with zero flakes. With testcontainers, it regularly flaked with some opaque docker network error or sometimes just plain a timeout because apparently launching 100s of containers in parallel is a hard problem for docker or something. I'm sure it works well for some people, but if those people wanted to build out their own version without docker and testcontainers, specific to their development tooling and environment, it would probably work better in most cases I think.
- jchw 3y ago
  Yeah, I couldn't tell you: I didn't have serious issues with performance or reliability when I used this. It didn't really seem like it was doing anything special, it seemed like it is just starting a Docker container like you'd expect, at least on Linux. I sincerely doubt I would've had any better experience handrolling it since my solution to handroll it would also be to use the Docker API; the reason for this is for portability.