3 ms·
How does it find which tests are relevant? How big can the solutions and test suites be before it starts to...not work? I have a decent sized project (100 proj
by alkonaut 10y ago
How does it find which tests are relevant? How big can the solutions and test suites be before it starts to...not work?
I have a decent sized project (100 projects, 10k+ tests, different suites with longer running smoke tests that take hours and so on). How does it know which tests it can run, and how does it know which will cover a certain line?
- topbanana 10y agoI can only speak for NCrunch, but once it's done a full coverage run, it knows which tests are likely to be affected by any changes, so it runs those tests first.
- gokhan 10y agoYou can automatically re-run affected UI tests after any code change with UI tests recorded in Microsoft Test Manager, since 2012 if I remember correctly. So the technology is already there, I guess it was relatively simple to apply it to unit tests. Test Professional is an interesting product, with recording manual tests, turning them into code, running them as a suite etc. https://www.visualstudio.com/vs/test-professional/ https://www.visualstudio.com/vs/test-professional/
- topbanana 10y agoSounds fragile as hell. How does it work in practise?
- Metasyntactic 10y agoHi Alkonaut, i'm dev on the Roslyn team, and i work closely with the group working on LiveUnitTesting (LUT). By default LUT just runs all your tests in realtime, as you make edits. It smartly prioritized tests based on seeing what code was previously hit by tests. Now, that said, such an approach would definitely not scale up to projects with huge amount of tests, or tests that take hours and hours to run. As such, LUT provides ways to create 'test sets'. i.e. sets of tests scoped down to just the work you're doing. The intent is to have flexible way to create these sets and the individual sets can be set up to run automatically or manually depending on your preference. A nice things is that all of these test run outside of code VS. So the perf of the tests is far less impactful on VS than it would be otherwise. i.e. you don't have a single GC engine trying to manage all the memory for core VS as well as whatever garbage the tests are generating. With this, and with tests running at lower pri, your editing experience should not be impacted (and we take perf impact very seriously with this feature). Cheers!
- alkonaut 10y agoGreat answer, thanks