4 ms·

> I guess that you have never worked on big systems I work on search and indexing systems at Google, and have done for more than a decade. I'm by no means the

> I guess that you have never worked on big systems I work on search and indexing systems at Google, and have done for more than a decade. I'm by no means the best, nor an authority. But I have spent some time living in this problem space, and this is my conclusion from my years of experience. FWIW a single run of the tests for the system I work on now takes . . . I don't know. I would guess hours or maybe even a day of computer time? But, computers are cheap, and defects are expensive. So you split the test suite into bits small enough that the whole thing runs in ten minutes, and run them all in parallel. And, yes, there have been cases where I have had to debug the slow startup of a dependency so that I can make it fast enough to run in my test. But, that's what they pay me for, I enjoy that type of work, and it has ancillary benefits to the dependency when it needs to start up in production.

tigershark 9y ago
In the real world you don’t have pretty much unlimited resources. If I remember correctly those tests I was speaking about were already running in parallel on 2 or 3 machines otherwise the total time would have been 2 or 3 times more. Obviously we couldn’t use hundreds of machines to make them run in 10 minutes. In most of the places were I worked even getting a couple of new machines for production use would take months, I think if I requested even a couple of machines dedicated only for the integration testing I would have been laughed in the face. So if you can run a full suite of integration tests in minutes on hundreds/thousands of machines then good for you, but don’t expect that every company has the resources for doing it. And in those cases unit tests that run continuously are more useful than integration tests that run once a day.
- sdhgaiojfsa 9y ago
  > In most of the places were I worked even getting a couple of new machines for production use would take months OK, well I guess this would be another caveat for my advice. If bringing up a production simulacra would take more resources than you have, then you might have to lean more on various types of doubles. This may be more often in environments that have insane prioritization of dev time vs. machine resources. If I may ask, what was the bottleneck in your test suite? Did it saturate the cpu, ram, disk?
  - tigershark 9y ago
    I don’t really remember. The software was a kind of ORM but it was built to interface the java code with an object database, so it doesn’t make much sense to call it ORM. Probably the main bottlenecks were cpu and disk if I have to guess. I arrived very late in the project, at the point where we were going to migrate everything to oracle and we had to add oracle support to that behemoth.

tigershark 9y ago

In the real world you don’t have pretty much unlimited resources. If I remember correctly those tests I was speaking about were already running in parallel on 2 or 3 machines otherwise the total time would have been 2 or 3 times more. Obviously we couldn’t use hundreds of machines to make them run in 10 minutes. In most of the places were I worked even getting a couple of new machines for production use would take months, I think if I requested even a couple of machines dedicated only for the integration testing I would have been laughed in the face. So if you can run a full suite of integration tests in minutes on hundreds/thousands of machines then good for you, but don’t expect that every company has the resources for doing it. And in those cases unit tests that run continuously are more useful than integration tests that run once a day.

sdhgaiojfsa 9y ago
> In most of the places were I worked even getting a couple of new machines for production use would take months OK, well I guess this would be another caveat for my advice. If bringing up a production simulacra would take more resources than you have, then you might have to lean more on various types of doubles. This may be more often in environments that have insane prioritization of dev time vs. machine resources. If I may ask, what was the bottleneck in your test suite? Did it saturate the cpu, ram, disk?
- tigershark 9y ago
  I don’t really remember. The software was a kind of ORM but it was built to interface the java code with an object database, so it doesn’t make much sense to call it ORM. Probably the main bottlenecks were cpu and disk if I have to guess. I arrived very late in the project, at the point where we were going to migrate everything to oracle and we had to add oracle support to that behemoth.