3 ms·

I have used Pitest at work, it is very good and on more than one occasion it unearthed wanting tests. Unfortunately, we had to remove it from our build. Our CI

I have used Pitest at work, it is very good and on more than one occasion it unearthed wanting tests. Unfortunately, we had to remove it from our build. Our CI pipeline uses VM's that were not provisioned with this type of testing in mind and pitest ended up slowing the build enough to make it painful. If we could get past this, I would turn it on in the morning.

0hjc 11y ago
Did you try using the history file option?
- johnflan 11y ago
  For every check in we get generate a new build environment, so managing a history file would have been quite difficult.
  - paulmd 11y ago
    What about checking the history file into its own rep? Clone it as part of the build-environment setup, and commit/push after validation. You can even use this across a group of people and share each other's mutations. Or even simpler, why not have build actions copy it to/from a specific location outside the build target dir? You could even make it a shared network drive, if you didn't care about the possibility of losing someone's changes if two people run tests at the same time.
the_af 11y ago
That's too bad that you had to stop using it. I've never used one of these tools, but I knew they existed. Just today I was discussing "testing the tests" with a coworker. In my opinion, where I work we write lots of incomplete/illogical tests, sometimes bordering on cargo culting. Aside from code-coverage, which is flawed, we have no real measure of whether the tests we write are effective or not.
- oliverc2 11y ago
  There is a tool to detect duplicate tests which many have found useful: http://ortask.com/testless/ http://ortask.com/testless/ For other kinds of test quality, try Mutator: http://ortask.com/mutator/ http://ortask.com/mutator/
  - the_af 11y ago
    Thanks for the links! I hate to sound negative, but something about that website seems dodgy. I'm unconvinced duplicate/overlapping tests -- while obviously undesirable -- have a direct correlation with code quality. Unfortunately, in order to read their "papers" where they elaborate on this, I have to register :/ I'd rather use something open source like pitest. (Again, thanks for the links! I don't want to sound too negative)

0hjc 11y ago

Did you try using the history file option?

johnflan 11y ago
For every check in we get generate a new build environment, so managing a history file would have been quite difficult.
- paulmd 11y ago
  What about checking the history file into its own rep? Clone it as part of the build-environment setup, and commit/push after validation. You can even use this across a group of people and share each other's mutations. Or even simpler, why not have build actions copy it to/from a specific location outside the build target dir? You could even make it a shared network drive, if you didn't care about the possibility of losing someone's changes if two people run tests at the same time.

the_af 11y ago

That's too bad that you had to stop using it. I've never used one of these tools, but I knew they existed. Just today I was discussing "testing the tests" with a coworker. In my opinion, where I work we write lots of incomplete/illogical tests, sometimes bordering on cargo culting. Aside from code-coverage, which is flawed, we have no real measure of whether the tests we write are effective or not.

oliverc2 11y ago
There is a tool to detect duplicate tests which many have found useful: http://ortask.com/testless/ http://ortask.com/testless/ For other kinds of test quality, try Mutator: http://ortask.com/mutator/ http://ortask.com/mutator/
- the_af 11y ago
  Thanks for the links! I hate to sound negative, but something about that website seems dodgy. I'm unconvinced duplicate/overlapping tests -- while obviously undesirable -- have a direct correlation with code quality. Unfortunately, in order to read their "papers" where they elaborate on this, I have to register :/ I'd rather use something open source like pitest. (Again, thanks for the links! I don't want to sound too negative)