4 ms·

Do you simply mean regression testing?

avgcorrection 4y ago
Yeah, weird to see all these variations - Aha, an expect test! - Oh, you mean a snapshot test! - This here is akin to UI testing framework X where the test framework can compare an expected screenshot of the UI to a screenshot of the actual UI! The last one basically requires automation if you want anyone to make use of it. The regression testing automation described in the OP is a nice-to-have, not a so-good-that-it-gets-a-new-name.
- avgcorrection 4y ago
  … And apparently also “change detector test” https://news.ycombinator.com/item?id=34379175 https://news.ycombinator.com/item?id=34379175
  - drothlis 4y ago
    ...and my favourite term, "characterization test": https://en.wikipedia.org/wiki/Characterization_test https://en.wikipedia.org/wiki/Characterization_test "Regression test" means something else, at least at the companies I've worked at: It means a test that was written after a defect was found in production, to ensure that the same defect doesn't happen again (that the fix doesn't "regress"). It can be a manual test or an automated test. https://en.wikipedia.org/wiki/Regression_testing https://en.wikipedia.org/wiki/Regression_testing
    - avgcorrection 4y ago
      That’s fine and I have no objective argument against it. But I don’t see much reason to need two different names for tests that do the same thing merely based on how they were introduced. Sometimes I add a regression test because I fixed a bug, and sometimes I add a regression test because I just implemented a feature that I don’t want my future self to ruin: six months from now they will co-exist in the same suite and serve the same function. One reason to call bug fix tests for “regression tests” (and only those kinds of tests) is that someone might regress the code base through a merge conflict (maybe they effectively undo a commit?). So that’s one argument I suppose.
      - drothlis 4y ago
        "Regression testing" can also refer to a process: When the QA team says they're doing regression testing, it means they're testing that existing functionality hasn't regressed (as opposed to testing a new feature). I'm not particularly wedded to any of these terms, I'm just pointing out that "regression testing" has an established meaning, and it isn't snapshot testing (outside of certain industries, at least). I do find it amusing that one implementation of snapshot testing (https://pypi.org/project/pytest-regtest/ https://pypi.org/project/pytest-regtest/) links to https://en.wikipedia.org/wiki/Regression_testing https://en.wikipedia.org/wiki/Regression_testing but that article doesn't describe snapshot testing at all! Maybe the article changed? Oh well, language changes too. ¯\_(ツ)_/¯
        avgcorrection 4y ago
        Good points.

Yeah, weird to see all these variations - Aha, an expect test! - Oh, you mean a snapshot test! - This here is akin to UI testing framework X where the test framework can compare an expected screenshot of the UI to a screenshot of the actual UI! The last one basically requires automation if you want anyone to make use of it. The regression testing automation described in the OP is a nice-to-have, not a so-good-that-it-gets-a-new-name.

avgcorrection 4y ago
… And apparently also “change detector test” https://news.ycombinator.com/item?id=34379175 https://news.ycombinator.com/item?id=34379175
- drothlis 4y ago
  ...and my favourite term, "characterization test": https://en.wikipedia.org/wiki/Characterization_test https://en.wikipedia.org/wiki/Characterization_test "Regression test" means something else, at least at the companies I've worked at: It means a test that was written after a defect was found in production, to ensure that the same defect doesn't happen again (that the fix doesn't "regress"). It can be a manual test or an automated test. https://en.wikipedia.org/wiki/Regression_testing https://en.wikipedia.org/wiki/Regression_testing
  - avgcorrection 4y ago
    That’s fine and I have no objective argument against it. But I don’t see much reason to need two different names for tests that do the same thing merely based on how they were introduced. Sometimes I add a regression test because I fixed a bug, and sometimes I add a regression test because I just implemented a feature that I don’t want my future self to ruin: six months from now they will co-exist in the same suite and serve the same function. One reason to call bug fix tests for “regression tests” (and only those kinds of tests) is that someone might regress the code base through a merge conflict (maybe they effectively undo a commit?). So that’s one argument I suppose.
    - drothlis 4y ago
      "Regression testing" can also refer to a process: When the QA team says they're doing regression testing, it means they're testing that existing functionality hasn't regressed (as opposed to testing a new feature). I'm not particularly wedded to any of these terms, I'm just pointing out that "regression testing" has an established meaning, and it isn't snapshot testing (outside of certain industries, at least). I do find it amusing that one implementation of snapshot testing (https://pypi.org/project/pytest-regtest/ https://pypi.org/project/pytest-regtest/) links to https://en.wikipedia.org/wiki/Regression_testing https://en.wikipedia.org/wiki/Regression_testing but that article doesn't describe snapshot testing at all! Maybe the article changed? Oh well, language changes too. ¯\_(ツ)_/¯
      - avgcorrection 4y ago
        Good points.