13 ms·

Testcontainers

719 points by floriangosse 3y ago

globular-toast 3y ago
I'm surprised this is getting so much attention. I thought this just standard practice at this point? If you use things like Gitlab CI then you get this via the `services` in your pipeline. The CI job itself runs in a container too. I use a very similar thing via pytest-docker: https://github.com/avast/pytest-docker https://github.com/avast/pytest-docker The only difference seems to be you declare your containers via a docker-compose file which I prefer because it's a standard thing you can use elsewhere.
whalesalad 3y ago
Integrated this in an afternoon. It was surprisingly simple and works great locally and also inside GitHub actions to do psql integration tests of our core app.
srid 3y ago
If your project uses Nix, checkout services-flake for running services via Nix. https://github.com/juspay/services-flake https://github.com/juspay/services-flake We actually do this in Nammayatri, an OSS project providing "Uber" for autos in India. https://github.com/nammayatri/nammayatri https://github.com/nammayatri/nammayatri There is a services-flake module allowing you to spin the entire nammayatri stack (including postgres, redis, etc.) using a flake app. Similarly, there's one for running load test, which is also run in Jenkins CI.
nym3r0s 3y ago
Something that improved developer experience by far and also sped up our builds is starting the container dependencies via docker-compose and connect to it for integration testing. This allows reuse of containers, you can connect to it after/during an integration test to debug without having to keep searching for ports constantly. With TestContainers - I've perceived that running integration tests / a single test repeatedly locally is extremely slow as the containers are shut down when the java process is killed. This approach allows for this while also allowing to keep it consistent - example, just mount the migrations folder in the start volume of your DB container and you have a like-for-like schema of your prod DB ready for integration tests. I've found the https://github.com/avast/gradle-docker-compose-plugin/ https://github.com/avast/gradle-docker-compose-plugin/ very useful for this.
mrklol 3y ago
I am using https://github.com/ory/dockertest https://github.com/ory/dockertest for tests, specifically for databases. Is there any advantage to use Testcontainers?
puradawid 3y ago
Interesting. I would say this should not be any close to the "unit test", but I see it doesn't matter anymore.
Hack0133 3y ago
[flagged]
redact207 3y ago
I didn't quite understand why this was made. We create our local test environments using docker-compose, and so I read: > Creating reliable and fully-initialized service dependencies using raw Docker commands or using Docker Compose requires good knowledge of Docker internals and how to best run specific technologies in a container This sounds like a <your programming language> abstraction over docker-compose, which lets you define your docker environment without learning the syntax of docker-compose itself. But then > port conflicts, containers not being fully initialized or ready for interactions when the tests start, etc. means you'd still need a good understanding of docker networking, dependencies, healthchecks to know if your test environment is ready to be used. Am I missing something? Is this basically change what's starting your docker test containers?
- AlfeG 3y ago
  We create own DB env for each set of test fixtures to run them parallel. There is no way I can achieve this with this little amount of frictions.
- stonecolddevin 3y ago
  Testcontainers is great. It's got seamless junit integration and really Just Works. I've never once had to even think about any of the docker aspects of it. There's really not much to it.
- mleo 3y ago
  It’s not coming across in your comment, but Testcontainers can work with unit tests to start a container, run the unit tests and shutdown. For example, to verify database operations against the actual database, the unit test can start an instance of Postgres run tests and then shut it down. If running tests in parallel, each test can start its own container and shutdown at the end.
  - DanHulton 3y ago
    Wouldn't that just massively, _massively_ slow down your tests, if each test was spinning up its own Postgres container? I ask because I really like this and would love to use it, but I'm concerned that that would add just an insane amount of overhead to the point where the convenience isn't worth the immense amount of extra time it would take.
senorrib 3y ago
Pulling up infra to run unit tests is an anti-pattern. This is a great tool for integration tests, though.
- deleted 3y ago
  [deleted]
- marginalia_nu 3y ago
  What if you are unit testing something that is dependent on infra?
  - perlclutcher 3y ago
    [dead]
  - devthane 3y ago
    Typically you mock them in unit tests.
    - globular-toast 3y ago
      If the "thing" is a database, for example, then the way to mock it is to bring up a database container and load it with dummy data.
    - nomel 3y ago
      I've rarely found this to be worth it, for the effort required for a proper mock, in a complex system. I've seen most people mock in ways that are so superficial that it's basically a no-op.
      - CuriouslyC 3y ago
        Mocks are a contentious topic as you've probably guessed. In my opinion they're a sign of coupled code, you should be able to hit very high coverage without a single mock, but if you're a dev in an org that tracks code coverage you'll probably end up writing a fair number of them since the odds are high you'll be consuming coupled code.
alifaziz 3y ago
Unit test suppose to be fast. Especially during coding. I wonder how this is necessary & not affecting the test feedback speed
- marginalia_nu 3y ago
  Testcontainers are surprisingly fast. Like seconds of runtime for a suite.
- alemanek 3y ago
  This is for integration tests.
  - thomasfromcdnjs 3y ago
    Homepage hero says "Unit tests with real dependencies"
    - alemanek 3y ago
      A unit test with real dependencies is by definition an integration test isn’t it? I guess the unit in “Unit Test” is a bit subjective but every place I have worked we wrote both unit and integration tests that lived side by side. Integration tests used Test Containers and Unit Tests were isolated with mocks. So, only functional difference was the “unit” under test being more, or less, isolated.
dm03514 3y ago
Test containers is such a game changer for integration testing, they have language specific docker apis that make it trivial to bring up containers and verify that they are fully initialized and ready to accept connections. Pretty much every project I create now has testcontainers for integration testing :) I setup CI so it lints, builds, unit tests then integration tests (using testcontainers) https://github.com/turbolytics/latte/blob/main/.github/workflows/ci.yml#L22 https://github.com/turbolytics/latte/blob/main/.github/workf... Their language bindings provide nice helper functions for common database operations (like generating a connection uri from a container user) https://github.com/turbolytics/latte/blob/main/internal/source/metric/mongodb/integration_test.go#L16 https://github.com/turbolytics/latte/blob/main/internal/sour... I use them in $day job use them in side projects use them everywhere :)
- weq 3y ago
  Can you explain more in more detail why this is a game changer if i already have an inhouse framework that is similiar in using docker for integration tests? Does it start docker up faster then you could do normally? Is it just the out of the box apis it provides? I dont know why integration testing like this is considered a gamechanger. the testing pyramid is a testing pyramid for a reason and its always considered them important. Sometimes starting with integration tests in your project is right because your dont waste time doing manual point and clicks. Instead you design your system around being able to integration test, this includes when you choose dependancies. You think to yourself "how easily will that be able to be stood up on its own from a command?" If the answer is "not very good" then you move on.
  - sverhagen 3y ago
    If you have an existing in-house framework for anything, maybe it's not worth switching over. It does help though when a best practice bubbles to the top and makes this in reach for those who don't have an existing in-house framework and who wouldn't know how to get started on one. It also helps for more people to have a shared understanding about a subject like this thanks to a popular implementation. Meanwhile, Testcontainers is done quite well. It's not perfect, but it's sure better than the in-house stuff I built in the past (for the same basic concept). No, it does not start faster than other Docker containers. I do challenge the testing pyramid, though. At the risk of repeating my other comment on a different branch of the discussion: the value of integration tests is high, as the cost of integration tests has decreased, it makes sense to do more integration testing, at the expense of unit testing. The cost has decreased exactly due to Docker and mature application frameworks (like in Java: Spring). (See: Testing Trophy.)
simonw 3y ago
I was intrigued to see that they have PyPI packages for a ton of different things - like https://pypi.org/project/testcontainers-postgres https://pypi.org/project/testcontainers-postgres That wheel file is only 2.9KB, so I grabbed a copy to see how it works. I've put the contents in a Gist here: https://gist.github.com/simonw/c53f80a525d573533a730f5f28858f84 https://gist.github.com/simonw/c53f80a525d573533a730f5f28858... It's pretty neat - it depends on testcontainers-core, sqlalchemy and psycopg2-binary and then defines a PostgresContainer class which fires up a "postgres:latest" container and provides a helper function for getting the right connection URL.
- gv83 3y ago
  This is also its downfall as my organization uses asyncpg and compatibility with it is still absent iirc :(
mrAssHat 3y ago
Testcontainers aren't even compatible with kubernetes, that's a tool from the past.
- the_jeremy 3y ago
  We use [kubedock](https://github.com/joyrex2001/kubedock https://github.com/joyrex2001/kubedock) to run testcontainers in kubernetes clusters. As long as you're only pulling the images, not building or loading them (explicitly not supported by kubedock), it works pretty well.
- marginalia_nu 3y ago
  Why'd you run them in kubernetes? Seems like extreme overkill for launching a short lived container for an integration test. What could kubernetes possibly add to that?
  - mrAssHat 3y ago
    Because we are a big company and would like to utilize resources better. We also want homogeneity in tech when possible (we already heavily use kubernetes, we don't want to keep docker hosts anymore). Teams of testers need to be accounted in terms of resource quotas and RBAC. What exactly do you see as an overkill in wanting to run short-lived containers in kubernetes rather than in docker (if we already have kubernetes and "cook" it ourselves)?
    - politelemon 3y ago
      That reasoning seems more like one from policy/cargo cult rather than reasoning specific to your org. For something short lived and meant to be isolated I wouldn't want to subject them to even more infrastructural dependencies outside their control.
      - mrAssHat 3y ago
        Better resources utilization definitely sounds like cargo cult, riiight.
simonw 3y ago
Not sure how I hadn't encountered this before, I LOVE this pattern. I find integration tests that exercise actual databases/Elasticsearch/Redis/Varnish etc to be massively more valuable than traditional unit tests. In the past I've gone to pretty deep lengths to do things like spin up a new Elasticsearch index for the duration of a test suite and spin it down again at the end. It looks like Testcontainers does all of that work for me. My testing strategy is to have as much of my application's functionality covered by proper end-to-end integration-style tests as possible - think tests that simulate an incoming HTTP request and then run assertions against the response (and increasingly Playwright-powered browser automation tests for anything with heavy JavaScript). I'll use unit tests sparingly, just for the bits of my code that have very clear input/output pairs that afford unit testing. I only use mocks for things that I don't have any chance of controlling - calls to external APIs for example, where I can't control if the API provider will be flaky or not.
- trevor-e 3y ago
  Do you find this results in less overall test code to maintain since you likely have fewer but higher quality/signal tests?
  - simonw 3y ago
    Yeah - I find that sticking to tests like this means I don't have hundreds of tiny unit tests that rely on mocks, and it's still very supportive of refactoring - I can make some pretty big changes and be confident that I've not broken anything because a given request continues to return the expected response.
    - gorjusborg 3y ago
      The choice isn't unit tests vs . end-to-end tests, its between testing things you don't really care about and those you do. You care about real use cases and verifying design constraints are met. You don't care about internal implementation details. The nuance is that there are often things one cares about at multiple levels.
  - 3y ago
ants_everywhere 3y ago
> No more need for mocks or complicated environment configurations. Define your test dependencies as code, then simply run your tests and containers will be created and then deleted. Wait what? They think you don't need unit tests because you can run integration tests with containers? It's trivial to set up a docker container with one of your dependencies, but starting containers is painful and slow.
- tomnipotent 3y ago
  No mocks doesn't mean no tests. It means running tests against the full code path which includes requests to running instances of the services you might otherwise mock. For many apps and use cases, the overhead in managing container state is worth it.
  - ants_everywhere 3y ago
    > It means running tests against the full code path which includes requests to running instances of the services you might otherwise mock. Yeah, those are called end to end tests and you run them after integration tests which you run after unit tests. It sounds to me like they're saying just skip to the end to end tests. > For many apps and use cases, the overhead in managing container state is worth it. Yeah, and typically you'd run them after you run unit and integration tests. If I have 10 libraries to test that have database access, I have to run 10 database containers simultaneously every few minutes as part of the development process? That's overkill.
    - deleted 3y ago
      [deleted]
    - lmm 3y ago
      > Yeah, and typically you'd run them after you run unit and integration tests. If I have 10 libraries to test that have database access, I have to run 10 database containers simultaneously every few minutes as part of the development process? That's overkill. If it's actually causing you problems, then by all means replace some of them with more lightweight tests, at the cost of some test environment faithfulness. But don't optimise prematurely.
nonethewiser 3y ago
Its great but I find it harder to debug. And I have to say, I usually dont need it. Typically i just have some command which spins everything up from docker-compose files. I prefer this over putting configuration in code like you often do with test containers. You can also load from docker compose files but at that point the test container API isn’t really doing much. Its pretty much required when you want to setup/teardown in between tests though. This just usually isnt the case for me.
- mellutussa 3y ago
  > I usually dont need it But I need it to catch the bugs you commit to CI so you can fix them right away instead of letting me catch them and report them and wait wreck my productivity. (this is of course not directed at you personally, feel free to replace you/I/me with whatever names you can imagine!)
sigmonsays 3y ago
been doing this for years, I would not say this gets rid of testing though. Running integration tests are significantly more complicated to write and take longer to run. There is also race conditions present that you need to account for programmatically.. Such as waiting for a db to come up and schema to be applied. Or waiting for a specific event to occur in the daemon. That being said, this looks like a decent start. One thing that seems to be missing is the ability to tail logs and assert specific marks in the logs. Often you need to do an operation and wait until you see an event.
- klysm 3y ago
  I’ve never liked testing for logs, I’m not sure I see the point
bloopernova 3y ago
Somewhat related: anyone here using AWS Neptune graphql database? How do you develop locally against Neptune? Apart from Localstack, is there a way to mock Neptune for local testing and development?
- dboreham 3y ago
  You might take a look at Tinkerpop: https://tinkerpop.apache.org/ https://tinkerpop.apache.org/
m3kw9 3y ago
Not everything can be placed in a docker which seem as a requirement.
- PaoloBarbolini 3y ago
  Could you provide an example?
  - gui77aume 3y ago
    Legacy systems I guess, maybe cloud specific services
paxys 3y ago
I read through the docs and am still confused about what this actually does beyond running a single docker run command in the background and returning control to your code when the container is up.
gokulkrishh09 3y ago
[dead]
paulv 3y ago
Does this work with podman or is it docker only?
- zer00eyz 3y ago
  If your running podman you should pull your production deployment configs down and tweak those. You will get a much more complete env that way (routing, network, scale, load balance) https://www.redhat.com/sysadmin/kubernetes-workloads-podman-systemd https://www.redhat.com/sysadmin/kubernetes-workloads-podman-... << as a for instance ;)
- ozarker 3y ago
  I’ve used it with podman before, worked fine
deathanatos 3y ago
> Unit tests with real dependencies That's an integration test. These are integration tests. You're literally testing multiple units (e.g., Redis, and the thing using Redis) to see if they're integrating. Why do we even have words. These are valuable in their own right. They're just complicated & often incredibly slow compared to a unit test. Which is why I prefer mocks, too: they're speedy. You just have to get the mock right … and that can be tricky, particularly since some APIs are just woefully underdocumented, or the documentation is just full of lies. But the mocks I've written in the past steadily improve over time. Learn to stop worrying, and love each for what they are. (Our CI system actually used to pretty much directly support this pattern. Then we moved to Github Actions. GHA has "service containers", but unfortunately the feature is too basic to address real-world use cases: it assumes a container image can just … boot! … and only talk to the code via the network. Real world use cases often require serialized steps between the test & the dependencies, e.g., to create or init database dirs, set up certs, etc.)
- shykes 3y ago
  > GHA has "service containers", but unfortunately the feature is too basic to address real-world use cases: it assumes a container image can just … boot! … and only talk to the code via the network. Real world use cases often require serialized steps between the test & the dependencies, e.g., to create or init database dirs, set up certs, etc.) My biased recommendation is to write a custom Dagger function, and run it in your GHA workflow. https://dagger.io https://dagger.io If you find me on the Dagger discord, I will gladly write a code snippet summarizing what I have in mind, based on what you explained of your CI stack. We use GHA ourselves and use this pattern to great effect. Disclaimer: I work there :)
pylua 3y ago
I found test containers to be slow to startup last year. It wasn’t worth the effort considering how long it took to run compared to traditional spring IT h2 hibernate.
nslindtner 3y ago
Wow ... the syntax reminds me so much of aspire (microsoft new "composer"-syntax). Makes a lot of sense. Why not keep this information in code .. often the developers are ending up doing those task anyway. (not recommended .. but seen it so many times) Link: Microsoft aspire (https://learn.microsoft.com/en-us/dotnet/aspire/get-started/aspire-overview https://learn.microsoft.com/en-us/dotnet/aspire/get-started/...)
et1337 3y ago
I looked at testcontainers and ended up rolling my own version. One issue I had is that Docker is a very leaky abstraction. I needed to write one test and have it run in all these scenarios: - on a Mac - on a Linux VM - in a Docker container on a Linux VM, with a Docker socket mounted The networking for each of these is completely different. I had to make some opinionated choices to get code that could run in all cases. And running inside Docker prevented the test from being able to mount arbitrary files into the test containers, which turns out to be a requirement often. I ended up writing code to build a new image for each container, using ADD to inject files. I also wanted all the tests to run in parallel and spit out readable logs from every container (properly associated with the correct test). Not sure if any of these things have changed in testcontainers since I last looked, but these are the things I ran into. It took maybe a month of off and on tweaking, contrary to some people here claiming it can be done in an hour. As always, the devil is in the details. edit: I did end up stealing ryuk. That thing can’t really be improved upon.
- Sammi 3y ago
  "ended up rolling my own version" What does that mean in this case? What does a hand rolled version of this look like?
  - et1337 3y ago
    A golang library that uses the Docker SDK to create and run containers. My version is more opinionated than testcontainers and can really only be used inside Go tests (relies on a testing.TB)
- jackcviers3 3y ago
  Many of the Mac networking specifics have become less of a problem. I use Rancher Desktop, which uses the correct virtualization framework based on OSX versions and allows you to customize the lima and virtual machine provisioning scripts so that you don't have cross-platform headaches like this (for the most part). Also runs kubernetes locally out of the box so you can test app deployments without waiting around for resources to free up on your shared dev cluster. Newest versions have almost everything Docker Desktop has. Highly recommend if you are on mac.
circusfly 3y ago
No thanks, I will continue to roll my own to control it full, at home and at work.
leonardXu 3y ago
My team maintain a lot of flink connectors, We've changed external test resources to testcontainer as much as possible, it makes things simple and saves money as well.
asciii 3y ago
Never heard of this, also the label in footer for careers said "We're hiring" but then lists no open positions :/
SOLAR_FIELDS 3y ago
I did not come in here expecting to read such effusive praise for testcontainers. If you’re coming from a place where docker wasn’t really a thing I can see how it looks beautiful. And in a fair amount of use cases it can be really nice. But if you want it to play well with any other containerized workflow, good freaking luck. Testcontainers is the library that convinced me that shelling out to docker as an abstraction via bash calls embedded in a library is a bad idea. Not because containerization as an abstraction is a bad idea. Rather it’s that having a library that custom shell calls to the docker CLI as part of its core functionality creates problems and complexity as soon as one introduces other containerized workflows. The library has the nasty habit of assuming it’s running on a host machine and nothing else docker related is running, and footguns itself with limitations accordingly. This makes it not much better than some non dockerized library in most cases and oftentimes much much worse.
- dustedcodes 3y ago
  > The library has the nasty habit of assuming it’s running on a host machine and nothing else docker related is running To be honest given that most tests run as part of an isolated CI/CD pipeline this is a very reasonable assumption to make.
- cdata 3y ago
  I would guess that this speaks to an unattended (developer) user story related to other workflows, or perhaps the container-adjacent ecosystem overall. Testing with any workflow is always tricky to get just right, and tools that make it easy (like, "install a package and go" easy) are underrated.
- simonkagedal 3y ago
  Testcontainers is not shelling out to the docker CLI; at least on Java, it is using a Java implementation of the docker network protocol, and I believe that’s the case also for the other platforms. Not sure this matters for the core argument you are making, just thought I’d point it out.
  - doctorpangloss 3y ago
avensec 3y ago
Reading through the comments, I'm quite shocked to see how many deterrent conversations are happening without any understanding of the underlying tech stacks being tested. Testcontainers can be fantastic, especially when you are facing test environment overhead challenges, assuming you have the appropriate architectures / service boundaries to support it. I believe there is more code out there in existence with architectures that make using Testcontainers more challenging than it is worth.
badoongi 3y ago
I see testcontainers being used in tests making the test code style feel more like typical unit tests with fake implementations for system components. Which is misleading as these are more on the integration testing side typically. In essence this is another DSL (per language) for managing containers locally. And this DSL comes in addition to whatever system is actually used for managing containers in production for the project.
febed 3y ago
I tried to use Testcontainers just last week but ended up using simple docker commands instead. I didn’t find an easy way to connect an already running set of containers started via docker compose. Was straightforward to do with a set of scripts that just call docker exec.
jake_morrison 3y ago
You can do something similar with docker compose, driving the system from the outside. Create dockerized versions of dependencies like the database, build and run tests, and then run tests against the production app container. It's particularly useful for testing a set of microservices. See https://github.com/cogini/phoenix_container_example https://github.com/cogini/phoenix_container_example for a full example. This blog post describes it in detail: https://www.cogini.com/blog/breaking-up-the-monolith-building-testing-and-deploying-microservices/ https://www.cogini.com/blog/breaking-up-the-monolith-buildin...
- m00x 3y ago
  We use docker environments like this for tests, but it does have its issues. You often need to add custom behavior like waiting for the app to load and start serving, healthchecks, etc. Having it all in code is pretty useful, and it's self-contained within the code itself vs having to set up the environment in different places (CI, Github actions, local dev, etc). The negative is that code isn't portable to prod, it doesn't test your environment as well (important for staging), and you're missing out on sharing some environment settings. I feel like it definitely has its place in the stack and in certain companies.
joeevans1000 3y ago
Forgive the question... but... why can't 'test' containers be 'prod' containers?
- jamesdepp 3y ago
  Some tests might have side effects. Probably not a great idea to test the function “bill customer” on a prod deployment. That’s why containers for testing is great—it’s easy to spin up an environment that can be messed around with without consequences (even if things go wrong or your tests have side effects).
jillesvangurp 3y ago
I never really liked testcontainers. Too complicated. And I don't want to have my tests make too many assumptions about what level of control there is over their environment. IMHO it's just the wrong place to be messing with docker. I don't like layering abstractions on top of abstractions that were fine to begin with. Docker-compose is pretty much perfect for the job. An added complexity is that the before/after semantics of the test suite in things like JUnit are a bit handwavy and hard to control. Unlike testng, there's no @BeforeSuite (which is really what you want). The @BeforeAll that junit has is actually too late in the process to be messing around with docker. And more importantly, if I'm developing, I don't want my docker containers to be wasting time restarting in between tests. That's 20-30 seconds I don't want to add on top of the already lengthy runtime of compiling/building, firing up Spring and letting it do it's thing before my test runs in about 1-2 seconds. All this is trivially solved by doing docker stuff at the right time: before your test process starts. So, I do that using good old docker compose and a simple gradle plugin that calls it before our tests run and then again to shut it down right after. If it's already running (it simply probes the port) it skips the startup and shut down sequence and just leaves it running. It's not perfect but it's very simple. I have docker-compose up most of my working day. Sometimes for days on end. My tests don't have to wait for it to come up because it's already up. On CI (github actions), gradle starts docker compose, waits for it to come up, runs the tests, and then shuts it down. This has another big advantage that the process of running a standalone development server for manual testing, running our integration tests, and running our production server are very similar. Exactly the same actually; the only difference configuration and some light bootstrapping logic (schema creation). Configuration basically involves telling our server the hosts and ports of all the stuff it needs to run. Which in our case is postgres, redis, and elasticsearch. Editing the setup is easy; just edit the docker compose and modify some properties. Works with jvm based stuff and it's equally easy to replicate with other stuff. There are a few more tricks I use to keep things fast. I have ~300 integration tests that use db, redis, and elasticsearch. They run concurrently in under 1 minute on my mac. I cannot emphesize how important fast integration tests are as a key enabler for developer productivity. Enabling this sort of thing requires some planning but it pays off hugely. I wrote up a detailed article on how to do this some years ago. https://www.jillesvangurp.com/blog/2016-05-25-functional-tests-and-flakyness.html https://www.jillesvangurp.com/blog/2016-05-25-functional-tes... That's still what I do a few projects and companies later.
iamkoch 3y ago
If you build inside docker, running tests that use docker is a pain. Go has a lot of in-memory versions of things for tests, which run so much quicker than leaning on docker. Similarly, I found C# has in-memory versions of deps you can lean on. I really feel that test containers, although solving a problem, often introduces others for no great benefit
- AlfeG 3y ago
  Why test inside of build process?
nsteel 3y ago
> Each test gets a fresh, clean instance of the browser, without having to worry about variations in plugins or required updates. Except where everyone is saying that's too slow and instead they have a long-lived instance which they manually teardown each time. That's even what the examples do (some, at least, I didn't check them all). If you've already bought into the container world then why not embrace a few more. For everyone else, not sure there's much point in extra complexity (they call it simplicity) or bloat.
Claudiusseibt 3y ago
Duke as the logo for java?
omeid2 3y ago
About 7 years ago I wrote conex, which is basically testcontainers for Go, with first class integration with the Go's official testing framework: https://github.com/omeid/conex https://github.com/omeid/conex
domano 3y ago
I dont understand how this is better than a docker-compose.yml with your dependencies, which plays nicer with all other tooling. Especially if there are complex dependencies between required containers it seems to be pretty weak in comparison. But i also only used it like 5 years ago, so maybe things are significantly better now.
- iFreilicht 3y ago
  One specific case that I encountered recently was implementing "integration" tests, where I needed to test some behavior that relies on the global state of a database. All other tests before were easily parallelized, and this meant our whole service could be fully tested within 10-30 seconds (dev machine vs. pipeline). However, the new tests could not be run in parallel with the existing ones, as the changes in global state in the database caused flaky failures. I know there will be other tests like them in the future, so I want a robust way of writing these kinds of "global" tests without too much manual labor. Spinning up a new postgres instance for each of these specific tests would be one solution. I would like to instead go for running the tests inside of transactions, but that comes with its own sorts of issues.
- callamdelaney 3y ago
  Because you may want to spin up a new postgres database to test a specific scenario in an automated way. Testcontainers allows you to do that from code, for example you could write a pytest fixture to provide a fresh database for each test.
  - silon42 3y ago
    If you need that, sure, but often this will be too expensive (heat my laptop too much).
  - perfectspiral 3y ago
    I don't know about Postgres but a MySQL container can take at least a few seconds to start up and report back as healthy on a Macbook Pro. You can just purge tables on the existing database in between tests, no need to start up a whole new database server each time.
mellutussa 3y ago
This is a very nice project, but it's awful that they're blurring the line of unit test and integration test. They are very different and both very important. Things in the software world are very trendy. If this starts a trend of making people think that they're writing unit tests when they are writing integrations tests, we are fucked. If I need to change code that you wrote I need a lightning fast way to figure out that I haven't broken your code according to the tests that you wrote. That's unit tests. My changes might break the whole system. That's integration tests. I just to run that once and then I can go back to unit tests while I fix the mess I've made.
friedrich_zip 3y ago
works on my container
fesc 3y ago
Testcontainers is awesome and all the hate it gets here is undeserved. Custom shell scripts definitely can't compete. For example one feature those don't have is "Ryuk": A container that testcontainers starts which monitors the lifetime of the parent application and stops all containers when the parent process exits. It allows the application to define dependencies for development, testing, CI itself without needing to run some command to bring up docker compose beforehand manually. One cool usecase for us is also having a ephemeral database container that is started in a Gradle build to generate jOOQ code from tables defined in a Liquibase schema.
aranw 3y ago
I've been using Docker containers for integration testing for last few years. I usually roll my own custom solution in Go using the https://github.com/ory/dockertest https://github.com/ory/dockertest package though that adds necessary functionality around running migrations, creating kafka topics, or similar. Will definitely need to check out this next time I'm writing a test package
supahfly_remix 3y ago
This looks pretty useful! Question on the nginx container. For my tests, this container is only useful when files are mounted into the container. For example, it needs an nginx.conf passed in. How do I do this with NginxContainer?
- shepherdjerred 3y ago
  You can copy files into the container before it starts. Testcontainers provides APIs for this.
xyst 3y ago
I think it’s a step in the right direction. There’s probably a few uses cases where the dependent system is configured much differently in your “test container” vs production instance. But if the aim is to programmatically spin up dependencies and provide 99% guarantee that your app/workflows will work, then this seems it can do the job. One small note: test run time will probably increase. If a person has an outdated computer, I suspect they will have a hard time running the IT suite. Especially if it’s a complicated system with more than one dependency.
jackcviers3 3y ago
I see a lot of people commenting to use docker-compose instead. Testcontainers does have a docker compose integration [1]. 1. https://java.testcontainers.org/modules/docker_compose/ https://java.testcontainers.org/modules/docker_compose/
anton-107 3y ago
Can I run this in GitHub Actions?
- jackcviers3 3y ago
  Yes [1]. 1. https://www.atomicjar.com/2023/06/running-testcontainers-tests-using-github-actions/ https://www.atomicjar.com/2023/06/running-testcontainers-tes...
jackcviers3 3y ago
Yes [1]. 1. https://www.atomicjar.com/2023/06/running-testcontainers-tests-using-github-actions/ https://www.atomicjar.com/2023/06/running-testcontainers-tes...

719 points by floriangosse 3y ago

I'm surprised this is getting so much attention. I thought this just standard practice at this point? If you use things like Gitlab CI then you get this via the `services` in your pipeline. The CI job itself runs in a container too. I use a very similar thing via pytest-docker: https://github.com/avast/pytest-docker https://github.com/avast/pytest-docker The only difference seems to be you declare your containers via a docker-compose file which I prefer because it's a standard thing you can use elsewhere.

whalesalad 3y ago

Integrated this in an afternoon. It was surprisingly simple and works great locally and also inside GitHub actions to do psql integration tests of our core app.

srid 3y ago

If your project uses Nix, checkout services-flake for running services via Nix. https://github.com/juspay/services-flake https://github.com/juspay/services-flake We actually do this in Nammayatri, an OSS project providing "Uber" for autos in India. https://github.com/nammayatri/nammayatri https://github.com/nammayatri/nammayatri There is a services-flake module allowing you to spin the entire nammayatri stack (including postgres, redis, etc.) using a flake app. Similarly, there's one for running load test, which is also run in Jenkins CI.

nym3r0s 3y ago

Something that improved developer experience by far and also sped up our builds is starting the container dependencies via docker-compose and connect to it for integration testing. This allows reuse of containers, you can connect to it after/during an integration test to debug without having to keep searching for ports constantly. With TestContainers - I've perceived that running integration tests / a single test repeatedly locally is extremely slow as the containers are shut down when the java process is killed. This approach allows for this while also allowing to keep it consistent - example, just mount the migrations folder in the start volume of your DB container and you have a like-for-like schema of your prod DB ready for integration tests. I've found the https://github.com/avast/gradle-docker-compose-plugin/ https://github.com/avast/gradle-docker-compose-plugin/ very useful for this.

mrklol 3y ago

I am using https://github.com/ory/dockertest https://github.com/ory/dockertest for tests, specifically for databases. Is there any advantage to use Testcontainers?

puradawid 3y ago

Interesting. I would say this should not be any close to the "unit test", but I see it doesn't matter anymore.

Hack0133 3y ago

[flagged]

redact207 3y ago

I didn't quite understand why this was made. We create our local test environments using docker-compose, and so I read: > Creating reliable and fully-initialized service dependencies using raw Docker commands or using Docker Compose requires good knowledge of Docker internals and how to best run specific technologies in a container This sounds like a <your programming language> abstraction over docker-compose, which lets you define your docker environment without learning the syntax of docker-compose itself. But then > port conflicts, containers not being fully initialized or ready for interactions when the tests start, etc. means you'd still need a good understanding of docker networking, dependencies, healthchecks to know if your test environment is ready to be used. Am I missing something? Is this basically change what's starting your docker test containers?

AlfeG 3y ago
We create own DB env for each set of test fixtures to run them parallel. There is no way I can achieve this with this little amount of frictions.
stonecolddevin 3y ago
Testcontainers is great. It's got seamless junit integration and really Just Works. I've never once had to even think about any of the docker aspects of it. There's really not much to it.
mleo 3y ago
It’s not coming across in your comment, but Testcontainers can work with unit tests to start a container, run the unit tests and shutdown. For example, to verify database operations against the actual database, the unit test can start an instance of Postgres run tests and then shut it down. If running tests in parallel, each test can start its own container and shutdown at the end.
- DanHulton 3y ago
  Wouldn't that just massively, _massively_ slow down your tests, if each test was spinning up its own Postgres container? I ask because I really like this and would love to use it, but I'm concerned that that would add just an insane amount of overhead to the point where the convenience isn't worth the immense amount of extra time it would take.

senorrib 3y ago

Pulling up infra to run unit tests is an anti-pattern. This is a great tool for integration tests, though.

deleted 3y ago
[deleted]
marginalia_nu 3y ago
What if you are unit testing something that is dependent on infra?
- perlclutcher 3y ago
  [dead]
- devthane 3y ago
  Typically you mock them in unit tests.
  - globular-toast 3y ago
    If the "thing" is a database, for example, then the way to mock it is to bring up a database container and load it with dummy data.
  - nomel 3y ago
    I've rarely found this to be worth it, for the effort required for a proper mock, in a complex system. I've seen most people mock in ways that are so superficial that it's basically a no-op.
    - CuriouslyC 3y ago
      Mocks are a contentious topic as you've probably guessed. In my opinion they're a sign of coupled code, you should be able to hit very high coverage without a single mock, but if you're a dev in an org that tracks code coverage you'll probably end up writing a fair number of them since the odds are high you'll be consuming coupled code.

alifaziz 3y ago

Unit test suppose to be fast. Especially during coding. I wonder how this is necessary & not affecting the test feedback speed

marginalia_nu 3y ago
Testcontainers are surprisingly fast. Like seconds of runtime for a suite.
alemanek 3y ago
This is for integration tests.
- thomasfromcdnjs 3y ago
  Homepage hero says "Unit tests with real dependencies"
  - alemanek 3y ago
    A unit test with real dependencies is by definition an integration test isn’t it? I guess the unit in “Unit Test” is a bit subjective but every place I have worked we wrote both unit and integration tests that lived side by side. Integration tests used Test Containers and Unit Tests were isolated with mocks. So, only functional difference was the “unit” under test being more, or less, isolated.

dm03514 3y ago

Test containers is such a game changer for integration testing, they have language specific docker apis that make it trivial to bring up containers and verify that they are fully initialized and ready to accept connections. Pretty much every project I create now has testcontainers for integration testing :) I setup CI so it lints, builds, unit tests then integration tests (using testcontainers) https://github.com/turbolytics/latte/blob/main/.github/workflows/ci.yml#L22 https://github.com/turbolytics/latte/blob/main/.github/workf... Their language bindings provide nice helper functions for common database operations (like generating a connection uri from a container user) https://github.com/turbolytics/latte/blob/main/internal/source/metric/mongodb/integration_test.go#L16 https://github.com/turbolytics/latte/blob/main/internal/sour... I use them in $day job use them in side projects use them everywhere :)

weq 3y ago
Can you explain more in more detail why this is a game changer if i already have an inhouse framework that is similiar in using docker for integration tests? Does it start docker up faster then you could do normally? Is it just the out of the box apis it provides? I dont know why integration testing like this is considered a gamechanger. the testing pyramid is a testing pyramid for a reason and its always considered them important. Sometimes starting with integration tests in your project is right because your dont waste time doing manual point and clicks. Instead you design your system around being able to integration test, this includes when you choose dependancies. You think to yourself "how easily will that be able to be stood up on its own from a command?" If the answer is "not very good" then you move on.
- sverhagen 3y ago
  If you have an existing in-house framework for anything, maybe it's not worth switching over. It does help though when a best practice bubbles to the top and makes this in reach for those who don't have an existing in-house framework and who wouldn't know how to get started on one. It also helps for more people to have a shared understanding about a subject like this thanks to a popular implementation. Meanwhile, Testcontainers is done quite well. It's not perfect, but it's sure better than the in-house stuff I built in the past (for the same basic concept). No, it does not start faster than other Docker containers. I do challenge the testing pyramid, though. At the risk of repeating my other comment on a different branch of the discussion: the value of integration tests is high, as the cost of integration tests has decreased, it makes sense to do more integration testing, at the expense of unit testing. The cost has decreased exactly due to Docker and mature application frameworks (like in Java: Spring). (See: Testing Trophy.)

simonw 3y ago

I was intrigued to see that they have PyPI packages for a ton of different things - like https://pypi.org/project/testcontainers-postgres https://pypi.org/project/testcontainers-postgres That wheel file is only 2.9KB, so I grabbed a copy to see how it works. I've put the contents in a Gist here: https://gist.github.com/simonw/c53f80a525d573533a730f5f28858f84 https://gist.github.com/simonw/c53f80a525d573533a730f5f28858... It's pretty neat - it depends on testcontainers-core, sqlalchemy and psycopg2-binary and then defines a PostgresContainer class which fires up a "postgres:latest" container and provides a helper function for getting the right connection URL.

gv83 3y ago
This is also its downfall as my organization uses asyncpg and compatibility with it is still absent iirc :(

mrAssHat 3y ago

Testcontainers aren't even compatible with kubernetes, that's a tool from the past.

the_jeremy 3y ago
We use [kubedock](https://github.com/joyrex2001/kubedock https://github.com/joyrex2001/kubedock) to run testcontainers in kubernetes clusters. As long as you're only pulling the images, not building or loading them (explicitly not supported by kubedock), it works pretty well.
marginalia_nu 3y ago
Why'd you run them in kubernetes? Seems like extreme overkill for launching a short lived container for an integration test. What could kubernetes possibly add to that?
- mrAssHat 3y ago
  Because we are a big company and would like to utilize resources better. We also want homogeneity in tech when possible (we already heavily use kubernetes, we don't want to keep docker hosts anymore). Teams of testers need to be accounted in terms of resource quotas and RBAC. What exactly do you see as an overkill in wanting to run short-lived containers in kubernetes rather than in docker (if we already have kubernetes and "cook" it ourselves)?
  - politelemon 3y ago
    That reasoning seems more like one from policy/cargo cult rather than reasoning specific to your org. For something short lived and meant to be isolated I wouldn't want to subject them to even more infrastructural dependencies outside their control.
    - mrAssHat 3y ago
      Better resources utilization definitely sounds like cargo cult, riiight.

simonw 3y ago

Not sure how I hadn't encountered this before, I LOVE this pattern. I find integration tests that exercise actual databases/Elasticsearch/Redis/Varnish etc to be massively more valuable than traditional unit tests. In the past I've gone to pretty deep lengths to do things like spin up a new Elasticsearch index for the duration of a test suite and spin it down again at the end. It looks like Testcontainers does all of that work for me. My testing strategy is to have as much of my application's functionality covered by proper end-to-end integration-style tests as possible - think tests that simulate an incoming HTTP request and then run assertions against the response (and increasingly Playwright-powered browser automation tests for anything with heavy JavaScript). I'll use unit tests sparingly, just for the bits of my code that have very clear input/output pairs that afford unit testing. I only use mocks for things that I don't have any chance of controlling - calls to external APIs for example, where I can't control if the API provider will be flaky or not.

trevor-e 3y ago
Do you find this results in less overall test code to maintain since you likely have fewer but higher quality/signal tests?
- simonw 3y ago
  Yeah - I find that sticking to tests like this means I don't have hundreds of tiny unit tests that rely on mocks, and it's still very supportive of refactoring - I can make some pretty big changes and be confident that I've not broken anything because a given request continues to return the expected response.
  - gorjusborg 3y ago
    The choice isn't unit tests vs . end-to-end tests, its between testing things you don't really care about and those you do. You care about real use cases and verifying design constraints are met. You don't care about internal implementation details. The nuance is that there are often things one cares about at multiple levels.
- 3y ago

ants_everywhere 3y ago

> No more need for mocks or complicated environment configurations. Define your test dependencies as code, then simply run your tests and containers will be created and then deleted. Wait what? They think you don't need unit tests because you can run integration tests with containers? It's trivial to set up a docker container with one of your dependencies, but starting containers is painful and slow.

tomnipotent 3y ago
No mocks doesn't mean no tests. It means running tests against the full code path which includes requests to running instances of the services you might otherwise mock. For many apps and use cases, the overhead in managing container state is worth it.
- ants_everywhere 3y ago
  > It means running tests against the full code path which includes requests to running instances of the services you might otherwise mock. Yeah, those are called end to end tests and you run them after integration tests which you run after unit tests. It sounds to me like they're saying just skip to the end to end tests. > For many apps and use cases, the overhead in managing container state is worth it. Yeah, and typically you'd run them after you run unit and integration tests. If I have 10 libraries to test that have database access, I have to run 10 database containers simultaneously every few minutes as part of the development process? That's overkill.
  - deleted 3y ago
    [deleted]
  - lmm 3y ago
    > Yeah, and typically you'd run them after you run unit and integration tests. If I have 10 libraries to test that have database access, I have to run 10 database containers simultaneously every few minutes as part of the development process? That's overkill. If it's actually causing you problems, then by all means replace some of them with more lightweight tests, at the cost of some test environment faithfulness. But don't optimise prematurely.

nonethewiser 3y ago

Its great but I find it harder to debug. And I have to say, I usually dont need it. Typically i just have some command which spins everything up from docker-compose files. I prefer this over putting configuration in code like you often do with test containers. You can also load from docker compose files but at that point the test container API isn’t really doing much. Its pretty much required when you want to setup/teardown in between tests though. This just usually isnt the case for me.

mellutussa 3y ago
> I usually dont need it But I need it to catch the bugs you commit to CI so you can fix them right away instead of letting me catch them and report them and wait wreck my productivity. (this is of course not directed at you personally, feel free to replace you/I/me with whatever names you can imagine!)

sigmonsays 3y ago

been doing this for years, I would not say this gets rid of testing though. Running integration tests are significantly more complicated to write and take longer to run. There is also race conditions present that you need to account for programmatically.. Such as waiting for a db to come up and schema to be applied. Or waiting for a specific event to occur in the daemon. That being said, this looks like a decent start. One thing that seems to be missing is the ability to tail logs and assert specific marks in the logs. Often you need to do an operation and wait until you see an event.

klysm 3y ago
I’ve never liked testing for logs, I’m not sure I see the point

bloopernova 3y ago

Somewhat related: anyone here using AWS Neptune graphql database? How do you develop locally against Neptune? Apart from Localstack, is there a way to mock Neptune for local testing and development?

dboreham 3y ago
You might take a look at Tinkerpop: https://tinkerpop.apache.org/ https://tinkerpop.apache.org/

m3kw9 3y ago

Not everything can be placed in a docker which seem as a requirement.

PaoloBarbolini 3y ago
Could you provide an example?
- gui77aume 3y ago
  Legacy systems I guess, maybe cloud specific services

paxys 3y ago

I read through the docs and am still confused about what this actually does beyond running a single docker run command in the background and returning control to your code when the container is up.

gokulkrishh09 3y ago

[dead]

paulv 3y ago

Does this work with podman or is it docker only?

zer00eyz 3y ago
If your running podman you should pull your production deployment configs down and tweak those. You will get a much more complete env that way (routing, network, scale, load balance) https://www.redhat.com/sysadmin/kubernetes-workloads-podman-systemd https://www.redhat.com/sysadmin/kubernetes-workloads-podman-... << as a for instance ;)
ozarker 3y ago
I’ve used it with podman before, worked fine

deathanatos 3y ago

> Unit tests with real dependencies That's an integration test. These are integration tests. You're literally testing multiple units (e.g., Redis, and the thing using Redis) to see if they're integrating. Why do we even have words. These are valuable in their own right. They're just complicated & often incredibly slow compared to a unit test. Which is why I prefer mocks, too: they're speedy. You just have to get the mock right … and that can be tricky, particularly since some APIs are just woefully underdocumented, or the documentation is just full of lies. But the mocks I've written in the past steadily improve over time. Learn to stop worrying, and love each for what they are. (Our CI system actually used to pretty much directly support this pattern. Then we moved to Github Actions. GHA has "service containers", but unfortunately the feature is too basic to address real-world use cases: it assumes a container image can just … boot! … and only talk to the code via the network. Real world use cases often require serialized steps between the test & the dependencies, e.g., to create or init database dirs, set up certs, etc.)

shykes 3y ago
> GHA has "service containers", but unfortunately the feature is too basic to address real-world use cases: it assumes a container image can just … boot! … and only talk to the code via the network. Real world use cases often require serialized steps between the test & the dependencies, e.g., to create or init database dirs, set up certs, etc.) My biased recommendation is to write a custom Dagger function, and run it in your GHA workflow. https://dagger.io https://dagger.io If you find me on the Dagger discord, I will gladly write a code snippet summarizing what I have in mind, based on what you explained of your CI stack. We use GHA ourselves and use this pattern to great effect. Disclaimer: I work there :)

pylua 3y ago

I found test containers to be slow to startup last year. It wasn’t worth the effort considering how long it took to run compared to traditional spring IT h2 hibernate.

nslindtner 3y ago

Wow ... the syntax reminds me so much of aspire (microsoft new "composer"-syntax). Makes a lot of sense. Why not keep this information in code .. often the developers are ending up doing those task anyway. (not recommended .. but seen it so many times) Link: Microsoft aspire (https://learn.microsoft.com/en-us/dotnet/aspire/get-started/aspire-overview https://learn.microsoft.com/en-us/dotnet/aspire/get-started/...)

et1337 3y ago

I looked at testcontainers and ended up rolling my own version. One issue I had is that Docker is a very leaky abstraction. I needed to write one test and have it run in all these scenarios: - on a Mac - on a Linux VM - in a Docker container on a Linux VM, with a Docker socket mounted The networking for each of these is completely different. I had to make some opinionated choices to get code that could run in all cases. And running inside Docker prevented the test from being able to mount arbitrary files into the test containers, which turns out to be a requirement often. I ended up writing code to build a new image for each container, using ADD to inject files. I also wanted all the tests to run in parallel and spit out readable logs from every container (properly associated with the correct test). Not sure if any of these things have changed in testcontainers since I last looked, but these are the things I ran into. It took maybe a month of off and on tweaking, contrary to some people here claiming it can be done in an hour. As always, the devil is in the details. edit: I did end up stealing ryuk. That thing can’t really be improved upon.

Sammi 3y ago
"ended up rolling my own version" What does that mean in this case? What does a hand rolled version of this look like?
- et1337 3y ago
  A golang library that uses the Docker SDK to create and run containers. My version is more opinionated than testcontainers and can really only be used inside Go tests (relies on a testing.TB)
jackcviers3 3y ago
Many of the Mac networking specifics have become less of a problem. I use Rancher Desktop, which uses the correct virtualization framework based on OSX versions and allows you to customize the lima and virtual machine provisioning scripts so that you don't have cross-platform headaches like this (for the most part). Also runs kubernetes locally out of the box so you can test app deployments without waiting around for resources to free up on your shared dev cluster. Newest versions have almost everything Docker Desktop has. Highly recommend if you are on mac.

circusfly 3y ago

No thanks, I will continue to roll my own to control it full, at home and at work.

leonardXu 3y ago

My team maintain a lot of flink connectors, We've changed external test resources to testcontainer as much as possible, it makes things simple and saves money as well.

asciii 3y ago

Never heard of this, also the label in footer for careers said "We're hiring" but then lists no open positions :/

SOLAR_FIELDS 3y ago

I did not come in here expecting to read such effusive praise for testcontainers. If you’re coming from a place where docker wasn’t really a thing I can see how it looks beautiful. And in a fair amount of use cases it can be really nice. But if you want it to play well with any other containerized workflow, good freaking luck. Testcontainers is the library that convinced me that shelling out to docker as an abstraction via bash calls embedded in a library is a bad idea. Not because containerization as an abstraction is a bad idea. Rather it’s that having a library that custom shell calls to the docker CLI as part of its core functionality creates problems and complexity as soon as one introduces other containerized workflows. The library has the nasty habit of assuming it’s running on a host machine and nothing else docker related is running, and footguns itself with limitations accordingly. This makes it not much better than some non dockerized library in most cases and oftentimes much much worse.

dustedcodes 3y ago
> The library has the nasty habit of assuming it’s running on a host machine and nothing else docker related is running To be honest given that most tests run as part of an isolated CI/CD pipeline this is a very reasonable assumption to make.
cdata 3y ago
I would guess that this speaks to an unattended (developer) user story related to other workflows, or perhaps the container-adjacent ecosystem overall. Testing with any workflow is always tricky to get just right, and tools that make it easy (like, "install a package and go" easy) are underrated.
simonkagedal 3y ago
Testcontainers is not shelling out to the docker CLI; at least on Java, it is using a Java implementation of the docker network protocol, and I believe that’s the case also for the other platforms. Not sure this matters for the core argument you are making, just thought I’d point it out.
- doctorpangloss 3y ago

avensec 3y ago

Reading through the comments, I'm quite shocked to see how many deterrent conversations are happening without any understanding of the underlying tech stacks being tested. Testcontainers can be fantastic, especially when you are facing test environment overhead challenges, assuming you have the appropriate architectures / service boundaries to support it. I believe there is more code out there in existence with architectures that make using Testcontainers more challenging than it is worth.

badoongi 3y ago

I see testcontainers being used in tests making the test code style feel more like typical unit tests with fake implementations for system components. Which is misleading as these are more on the integration testing side typically. In essence this is another DSL (per language) for managing containers locally. And this DSL comes in addition to whatever system is actually used for managing containers in production for the project.

febed 3y ago

I tried to use Testcontainers just last week but ended up using simple docker commands instead. I didn’t find an easy way to connect an already running set of containers started via docker compose. Was straightforward to do with a set of scripts that just call docker exec.

jake_morrison 3y ago

You can do something similar with docker compose, driving the system from the outside. Create dockerized versions of dependencies like the database, build and run tests, and then run tests against the production app container. It's particularly useful for testing a set of microservices. See https://github.com/cogini/phoenix_container_example https://github.com/cogini/phoenix_container_example for a full example. This blog post describes it in detail: https://www.cogini.com/blog/breaking-up-the-monolith-building-testing-and-deploying-microservices/ https://www.cogini.com/blog/breaking-up-the-monolith-buildin...

m00x 3y ago
We use docker environments like this for tests, but it does have its issues. You often need to add custom behavior like waiting for the app to load and start serving, healthchecks, etc. Having it all in code is pretty useful, and it's self-contained within the code itself vs having to set up the environment in different places (CI, Github actions, local dev, etc). The negative is that code isn't portable to prod, it doesn't test your environment as well (important for staging), and you're missing out on sharing some environment settings. I feel like it definitely has its place in the stack and in certain companies.

joeevans1000 3y ago

Forgive the question... but... why can't 'test' containers be 'prod' containers?

jamesdepp 3y ago
Some tests might have side effects. Probably not a great idea to test the function “bill customer” on a prod deployment. That’s why containers for testing is great—it’s easy to spin up an environment that can be messed around with without consequences (even if things go wrong or your tests have side effects).

jillesvangurp 3y ago

I never really liked testcontainers. Too complicated. And I don't want to have my tests make too many assumptions about what level of control there is over their environment. IMHO it's just the wrong place to be messing with docker. I don't like layering abstractions on top of abstractions that were fine to begin with. Docker-compose is pretty much perfect for the job. An added complexity is that the before/after semantics of the test suite in things like JUnit are a bit handwavy and hard to control. Unlike testng, there's no @BeforeSuite (which is really what you want). The @BeforeAll that junit has is actually too late in the process to be messing around with docker. And more importantly, if I'm developing, I don't want my docker containers to be wasting time restarting in between tests. That's 20-30 seconds I don't want to add on top of the already lengthy runtime of compiling/building, firing up Spring and letting it do it's thing before my test runs in about 1-2 seconds. All this is trivially solved by doing docker stuff at the right time: before your test process starts. So, I do that using good old docker compose and a simple gradle plugin that calls it before our tests run and then again to shut it down right after. If it's already running (it simply probes the port) it skips the startup and shut down sequence and just leaves it running. It's not perfect but it's very simple. I have docker-compose up most of my working day. Sometimes for days on end. My tests don't have to wait for it to come up because it's already up. On CI (github actions), gradle starts docker compose, waits for it to come up, runs the tests, and then shuts it down. This has another big advantage that the process of running a standalone development server for manual testing, running our integration tests, and running our production server are very similar. Exactly the same actually; the only difference configuration and some light bootstrapping logic (schema creation). Configuration basically involves telling our server the hosts and ports of all the stuff it needs to run. Which in our case is postgres, redis, and elasticsearch. Editing the setup is easy; just edit the docker compose and modify some properties. Works with jvm based stuff and it's equally easy to replicate with other stuff. There are a few more tricks I use to keep things fast. I have ~300 integration tests that use db, redis, and elasticsearch. They run concurrently in under 1 minute on my mac. I cannot emphesize how important fast integration tests are as a key enabler for developer productivity. Enabling this sort of thing requires some planning but it pays off hugely. I wrote up a detailed article on how to do this some years ago. https://www.jillesvangurp.com/blog/2016-05-25-functional-tests-and-flakyness.html https://www.jillesvangurp.com/blog/2016-05-25-functional-tes... That's still what I do a few projects and companies later.

iamkoch 3y ago

If you build inside docker, running tests that use docker is a pain. Go has a lot of in-memory versions of things for tests, which run so much quicker than leaning on docker. Similarly, I found C# has in-memory versions of deps you can lean on. I really feel that test containers, although solving a problem, often introduces others for no great benefit

AlfeG 3y ago
Why test inside of build process?

nsteel 3y ago

> Each test gets a fresh, clean instance of the browser, without having to worry about variations in plugins or required updates. Except where everyone is saying that's too slow and instead they have a long-lived instance which they manually teardown each time. That's even what the examples do (some, at least, I didn't check them all). If you've already bought into the container world then why not embrace a few more. For everyone else, not sure there's much point in extra complexity (they call it simplicity) or bloat.

Claudiusseibt 3y ago

Duke as the logo for java?

omeid2 3y ago

About 7 years ago I wrote conex, which is basically testcontainers for Go, with first class integration with the Go's official testing framework: https://github.com/omeid/conex https://github.com/omeid/conex

domano 3y ago

I dont understand how this is better than a docker-compose.yml with your dependencies, which plays nicer with all other tooling. Especially if there are complex dependencies between required containers it seems to be pretty weak in comparison. But i also only used it like 5 years ago, so maybe things are significantly better now.

iFreilicht 3y ago
One specific case that I encountered recently was implementing "integration" tests, where I needed to test some behavior that relies on the global state of a database. All other tests before were easily parallelized, and this meant our whole service could be fully tested within 10-30 seconds (dev machine vs. pipeline). However, the new tests could not be run in parallel with the existing ones, as the changes in global state in the database caused flaky failures. I know there will be other tests like them in the future, so I want a robust way of writing these kinds of "global" tests without too much manual labor. Spinning up a new postgres instance for each of these specific tests would be one solution. I would like to instead go for running the tests inside of transactions, but that comes with its own sorts of issues.
callamdelaney 3y ago
Because you may want to spin up a new postgres database to test a specific scenario in an automated way. Testcontainers allows you to do that from code, for example you could write a pytest fixture to provide a fresh database for each test.
- silon42 3y ago
  If you need that, sure, but often this will be too expensive (heat my laptop too much).
- perfectspiral 3y ago
  I don't know about Postgres but a MySQL container can take at least a few seconds to start up and report back as healthy on a Macbook Pro. You can just purge tables on the existing database in between tests, no need to start up a whole new database server each time.

mellutussa 3y ago

This is a very nice project, but it's awful that they're blurring the line of unit test and integration test. They are very different and both very important. Things in the software world are very trendy. If this starts a trend of making people think that they're writing unit tests when they are writing integrations tests, we are fucked. If I need to change code that you wrote I need a lightning fast way to figure out that I haven't broken your code according to the tests that you wrote. That's unit tests. My changes might break the whole system. That's integration tests. I just to run that once and then I can go back to unit tests while I fix the mess I've made.

friedrich_zip 3y ago

works on my container

fesc 3y ago

Testcontainers is awesome and all the hate it gets here is undeserved. Custom shell scripts definitely can't compete. For example one feature those don't have is "Ryuk": A container that testcontainers starts which monitors the lifetime of the parent application and stops all containers when the parent process exits. It allows the application to define dependencies for development, testing, CI itself without needing to run some command to bring up docker compose beforehand manually. One cool usecase for us is also having a ephemeral database container that is started in a Gradle build to generate jOOQ code from tables defined in a Liquibase schema.

aranw 3y ago

I've been using Docker containers for integration testing for last few years. I usually roll my own custom solution in Go using the https://github.com/ory/dockertest https://github.com/ory/dockertest package though that adds necessary functionality around running migrations, creating kafka topics, or similar. Will definitely need to check out this next time I'm writing a test package

supahfly_remix 3y ago

This looks pretty useful! Question on the nginx container. For my tests, this container is only useful when files are mounted into the container. For example, it needs an nginx.conf passed in. How do I do this with NginxContainer?

shepherdjerred 3y ago
You can copy files into the container before it starts. Testcontainers provides APIs for this.

xyst 3y ago

I think it’s a step in the right direction. There’s probably a few uses cases where the dependent system is configured much differently in your “test container” vs production instance. But if the aim is to programmatically spin up dependencies and provide 99% guarantee that your app/workflows will work, then this seems it can do the job. One small note: test run time will probably increase. If a person has an outdated computer, I suspect they will have a hard time running the IT suite. Especially if it’s a complicated system with more than one dependency.

jackcviers3 3y ago

I see a lot of people commenting to use docker-compose instead. Testcontainers does have a docker compose integration [1]. 1. https://java.testcontainers.org/modules/docker_compose/ https://java.testcontainers.org/modules/docker_compose/

anton-107 3y ago

Can I run this in GitHub Actions?

jackcviers3 3y ago
Yes [1]. 1. https://www.atomicjar.com/2023/06/running-testcontainers-tests-using-github-actions/ https://www.atomicjar.com/2023/06/running-testcontainers-tes...

jackcviers3 3y ago

Yes [1]. 1. https://www.atomicjar.com/2023/06/running-testcontainers-tests-using-github-actions/ https://www.atomicjar.com/2023/06/running-testcontainers-tes...