5 ms·

FYI Docker already has a RESTful API, and programming container start/stop is trivial to do in any language. I haven't used Testcontainers before, and can kinda

by paxys 3y ago

FYI Docker already has a RESTful API, and programming container start/stop is trivial to do in any language. I haven't used Testcontainers before, and can kinda see the utility, but IMO it really isn't worth it in the long term to take on a new external dependency for a bit of code that (1) is a critical part of the team's development and release process and (2) can be written in-house in maybe an hour.

andrewxdiamond 3y ago
Conveniently as of a few days ago, you don’t need to be worried about this anymore. TestContainers is no longer a third-party! https://www.docker.com/blog/docker-whale-comes-atomicjar-maker-of-testcontainers/ https://www.docker.com/blog/docker-whale-comes-atomicjar-mak...
- comprev 3y ago
  If only Docker used a Sperm Whale as their mascot….
jchw 3y ago
> it really isn't worth it in the long term to take on a new external dependency for a bit of code that (1) is a critical part of the team's development and release process and (2) can be written in-house in maybe an hour. This seems to be quite a contradiction. If it's so easy to just write from scratch, then why would it be scary to depend on? Of course, it's not that easy to write from scratch. You could make a proof-of-concept in maybe an hour... Maybe. But they already took the proof of concept to a complete phase. Made it work with Podman. Added tons of integration code to make it easy to use with many common services. Ported it to several different languages. And, built a community around it. If you do this from scratch, you have to go through most of the effort and problems they already did, except if you write your own solution, you have to maintain it right from the git-go, whereas if you choose Testcontainers, you'll only wind up having to maintain it if the project is left for dead and starts to bitrot. The Docker API is pretty stable though, so honestly, this doesn't seem likely to be a huge issue. Testcontainers is exactly the sort of thing open source is great for; it's something where everyone gets to benefit from the wisdom and battle-testing of everyone else. For most of the problems you might run into, there is a pretty decent chance someone already did, so there's a pretty decent chance it's already been fixed. Most people have GitHub and Dockerhub dependencies in their critical dependency path for builds and deployment. Services go down, change their policies, deprecate APIs, and go under, but code continues to work if you replicate the environment it originally worked in. The biggest risk with code dependencies (for non-production code like test code) is usually that it blocks you from updating some other software. The biggest risk with services is that they completely disappear and you are completely blocked until you fully remove the dependency. I think people depending on Testcontainers are fine and doing very well with their risk analysis.
- TheDong 3y ago
  > This seems to be quite a contradiction. If it's so easy to just write from scratch, then why would it be scary to depend on? It's easy to write an implementation specific to your existing project's development environment and workflow correctly. It's hard to write a generic version that works in any environment. It's hard to write a version that lets you build a company and make money. It's scary to depend on this generic version because it's too generic, and it's built by a for-profit company now who wants to upsell you to some "testcontainer cloud" crap, which doesn't exactly incentivize them to make the OSS version perfect. For example, we were already using bazel, so writing a version using bazel to create a correct bare rootfs for a dependency + using runc to execute it in tests resulted in something with a roughly 3ms warm startup time that fulfilled all our needs, and cached correctly (since bazel has good caching). A junior engineer, starry-eyed at test-containers, switched some code over to it, and the warm startup time went up by 1000x from 3ms to 3s, as did the flake-rate, and docker's caching is far worse than bazel's so the several minute cold-starts also happened even more often. > Testcontainers is exactly the sort of thing open source is great for; it's something where everyone gets to benefit from the wisdom and battle-testing of everyone else. You get to have a mismatched mess of solutions to everyone's problems, including solution's to problems you don't have. You get code of the quality of the average OSS programmer which, while higher than the average programmer, is still pretty crap. Free Software is great when it's run by a small group of smart opinionated people. As soon as you try to build a company around that OSS and, god forbid, hire some enterprise sales people and product managers, it quickly becomes worse at the actual small developer problem than what even a mediocre developer could hack out in an hour. Depending on testcontainers is fine, but if you know what you're doing, writing something purpose-built is fine too, and probably gives you something much nicer in the end.
  - jchw 3y ago
    To be honest, it doesn't sound like you really had a good use case for Testcontainers anyways. Where it excels the most is in just pulling in some external containers, especially databases, e.g. PostgreSQL and Redis, directly in your test harness. In those cases, it works well.

by paxys 3y ago

andrewxdiamond 3y ago

Conveniently as of a few days ago, you don’t need to be worried about this anymore. TestContainers is no longer a third-party! https://www.docker.com/blog/docker-whale-comes-atomicjar-maker-of-testcontainers/ https://www.docker.com/blog/docker-whale-comes-atomicjar-mak...

comprev 3y ago
If only Docker used a Sperm Whale as their mascot….

jchw 3y ago

> it really isn't worth it in the long term to take on a new external dependency for a bit of code that (1) is a critical part of the team's development and release process and (2) can be written in-house in maybe an hour. This seems to be quite a contradiction. If it's so easy to just write from scratch, then why would it be scary to depend on? Of course, it's not that easy to write from scratch. You could make a proof-of-concept in maybe an hour... Maybe. But they already took the proof of concept to a complete phase. Made it work with Podman. Added tons of integration code to make it easy to use with many common services. Ported it to several different languages. And, built a community around it. If you do this from scratch, you have to go through most of the effort and problems they already did, except if you write your own solution, you have to maintain it right from the git-go, whereas if you choose Testcontainers, you'll only wind up having to maintain it if the project is left for dead and starts to bitrot. The Docker API is pretty stable though, so honestly, this doesn't seem likely to be a huge issue. Testcontainers is exactly the sort of thing open source is great for; it's something where everyone gets to benefit from the wisdom and battle-testing of everyone else. For most of the problems you might run into, there is a pretty decent chance someone already did, so there's a pretty decent chance it's already been fixed. Most people have GitHub and Dockerhub dependencies in their critical dependency path for builds and deployment. Services go down, change their policies, deprecate APIs, and go under, but code continues to work if you replicate the environment it originally worked in. The biggest risk with code dependencies (for non-production code like test code) is usually that it blocks you from updating some other software. The biggest risk with services is that they completely disappear and you are completely blocked until you fully remove the dependency. I think people depending on Testcontainers are fine and doing very well with their risk analysis.

TheDong 3y ago
> This seems to be quite a contradiction. If it's so easy to just write from scratch, then why would it be scary to depend on? It's easy to write an implementation specific to your existing project's development environment and workflow correctly. It's hard to write a generic version that works in any environment. It's hard to write a version that lets you build a company and make money. It's scary to depend on this generic version because it's too generic, and it's built by a for-profit company now who wants to upsell you to some "testcontainer cloud" crap, which doesn't exactly incentivize them to make the OSS version perfect. For example, we were already using bazel, so writing a version using bazel to create a correct bare rootfs for a dependency + using runc to execute it in tests resulted in something with a roughly 3ms warm startup time that fulfilled all our needs, and cached correctly (since bazel has good caching). A junior engineer, starry-eyed at test-containers, switched some code over to it, and the warm startup time went up by 1000x from 3ms to 3s, as did the flake-rate, and docker's caching is far worse than bazel's so the several minute cold-starts also happened even more often. > Testcontainers is exactly the sort of thing open source is great for; it's something where everyone gets to benefit from the wisdom and battle-testing of everyone else. You get to have a mismatched mess of solutions to everyone's problems, including solution's to problems you don't have. You get code of the quality of the average OSS programmer which, while higher than the average programmer, is still pretty crap. Free Software is great when it's run by a small group of smart opinionated people. As soon as you try to build a company around that OSS and, god forbid, hire some enterprise sales people and product managers, it quickly becomes worse at the actual small developer problem than what even a mediocre developer could hack out in an hour. Depending on testcontainers is fine, but if you know what you're doing, writing something purpose-built is fine too, and probably gives you something much nicer in the end.
- jchw 3y ago
  To be honest, it doesn't sound like you really had a good use case for Testcontainers anyways. Where it excels the most is in just pulling in some external containers, especially databases, e.g. PostgreSQL and Redis, directly in your test harness. In those cases, it works well.