4 ms·

Just saw the question about testing splitting - yes there is automation magic. You give us a command to list your tests and we split them as optimally as we can

Just saw the question about testing splitting - yes there is automation magic. You give us a command to list your tests and we split them as optimally as we can.

quesera 4y ago
These are really good answers. I couldn't find any of this info in the docs on your site though! And BTW, it's been a while since I last checked the numbers, plus we've moved to GitHub Actions recently. Our current setup time is almost 5 minutes for each parallel runner. Saving that time by using prebuilt runner images starts to sound like a great idea. I'm curious, do you keep an image for each customer? And you must have some heuristic for determining when their setup environment changes and requires rebuilding?
- reillyse 4y ago
  Yes, I'll need to update the site. I'm always torn between putting too much info there or not enough, but I guess I need more. We don't keep an image per customer. But we do have general images, so something like ruby-2.8-node-14 that contain a lot of the setup. Then building is just running the customers build commands, "yarn install" etc. The rebuilding heuristic is a list of relevant files. We hash the contents and if the hash doesn't match we know we need to rebuild. What stack are you using for your tests ?
  - quesera 4y ago
    I sympathize with the tension. These are the big questions I still had after reading your site docs. Thanks for taking the time to answer them! We use RSpec, Capybara, Selenium, Chrome Driver. We parallelize our tests semi-manually, which is a minor inconvenience. Recently moved to GitHub Actions, which makes it easy to cache dependencies (mostly gems), although it's not quite a pre-built image. Startup cost with fully warmed cache is about 55 seconds, almost all of which appears to be spinning up the container. Not sure what the underlying tech is, but that seems slow.
    - reillyse 4y ago
      I appreciate the feedback and I'll be updating the site to add it all in. Thanks! I think we can definitely help out with your stack, my email is in my profile if you want to get in touch.

Just saw the question about testing splitting - yes there is automation magic. You give us a command to list your tests and we split them as optimally as we can.

quesera 4y ago

These are really good answers. I couldn't find any of this info in the docs on your site though! And BTW, it's been a while since I last checked the numbers, plus we've moved to GitHub Actions recently. Our current setup time is almost 5 minutes for each parallel runner. Saving that time by using prebuilt runner images starts to sound like a great idea. I'm curious, do you keep an image for each customer? And you must have some heuristic for determining when their setup environment changes and requires rebuilding?

reillyse 4y ago
Yes, I'll need to update the site. I'm always torn between putting too much info there or not enough, but I guess I need more. We don't keep an image per customer. But we do have general images, so something like ruby-2.8-node-14 that contain a lot of the setup. Then building is just running the customers build commands, "yarn install" etc. The rebuilding heuristic is a list of relevant files. We hash the contents and if the hash doesn't match we know we need to rebuild. What stack are you using for your tests ?
- quesera 4y ago
  I sympathize with the tension. These are the big questions I still had after reading your site docs. Thanks for taking the time to answer them! We use RSpec, Capybara, Selenium, Chrome Driver. We parallelize our tests semi-manually, which is a minor inconvenience. Recently moved to GitHub Actions, which makes it easy to cache dependencies (mostly gems), although it's not quite a pre-built image. Startup cost with fully warmed cache is about 55 seconds, almost all of which appears to be spinning up the container. Not sure what the underlying tech is, but that seems slow.
  - reillyse 4y ago
    I appreciate the feedback and I'll be updating the site to add it all in. Thanks! I think we can definitely help out with your stack, my email is in my profile if you want to get in touch.