3 ms·
I think this really depends on the code base. If I had a website where a change would trigger more than 5 minutes of integrating and testing I would go mad. I'
by __mp 10y ago
I think this really depends on the code base. If I had a website where a change would trigger more than 5 minutes of integrating and testing I would go mad.
I'm working with a weather model and it takes around 1 hour to build and test everything: We build against 2 super computers, 3 different compilers, 2 different architectures (CPU, GPU) and single and double precision. Building alone takes 30 minutes (more in some cases). For testing we reserve 1x node with 8 GPUs and 9 CPU cores on one machine and 2x nodes with 1 GPUs in a 30 minutes debug slot.
With a pull-request based workflow we are able to push to the master multiple times a day. 40 minutes might be achievable by massively revamping our build mechanism and getting jenkins to store gigabytes in build artifacts for each run. However, I do not think it is worth it, because changes can take multiple days, or in some rare cases months to implement. If people need to wait for an hour for their tests to validate that's not so bad.
- maxxxxx 10y agoOur tests can take hours to days and need multiple hardware configurations. I am already happy if all tests run once a week or even once a month. In the past a lot of tests would be run maybe once or twice during a year long release cycle so once a month is already huge progress.
- __mp 10y agoI think this would be too expensive for us. Why do they take so long? What area are you working in? Would it make sense to scale down the tests to a sensible size and do the longer tests over the weekends?