5 ms·

What does your test suite look like? (What's the library is maybe a better question)

by ketzo 3y ago

latchkey 3y ago
Good question. `@testing-library/react` and jest to run it. snapshots and a bunch of action code too. There are more modern frameworks now too that can be used for integration tests, but I haven't bothered yet as the snapshots have almost always caught the issue. It boggles my mind that react.dev doesn't start off teaching, with writing tests.
- c-hendricks 3y ago
  Are snapshot tests the things that fail because they can't tell "background-color: #ff0000" and "background-color:#f00" are the same?
  - jtms 3y ago
    Yes, that’s right… they are of dubious utility imho.
    - latchkey 3y ago
      4 years of maintaining a react project downloaded 40k times a month argues otherwise.
      - zeroonetwothree 3y ago
        It probably works ok for a solo project but IME with large scale codebases snapshot tests are awful. You update some implementation detail of a common shared component and suddenly 5000 tests break despite the look and behavior being unchanged.
        latchkey 3y ago
        Give me an example. Those components depend on this common shared component... if you do something that changes that shared component in such a way that it causes other snapshots to fail, I'd absolutely want to know about it. That's the whole point of dependency failures. My dependency is on MUI, which is a massively used common shared component library. If MUI changes, and it breaks my snapshots, I'd absolutely want to know about it.
        cbovis 3y ago
        This has been my experience in the past with a heavily snapshot covered codebase. Class names can change, the structure of your HTML can change, the underlying CSS can even change and the end result is still the same because you were just refactoring. At a large enough scale it can be painful to have hundreds of snapshots break for a simple change - especially when you add required code review by others into the mix. Currently figuring out a strategy for introducing testing into an already large codebase and being very cautious of snapshot tests because of this. Experimenting with visual regression testing but early indicators suggest it could get very expensive if we're not careful about what is covered.
        latchkey 3y ago
        Give people the tools to easily update the snapshots and read the diff's of PR's (github is quite good at this imho).
        c-hendricks 3y ago
        I kinda think "just blindly update the snapshots" is teaching the wrong lesson. We removed them from our projects and haven't missed them. Visual diffing, cypress >>>> snapshots IMO.
        latchkey 3y ago
        Who said blindly update snapshots? Agreed that cypress today is probably a better solution.
  - latchkey 3y ago
    It depends on the tool doing the diff on the snapshot and what you're putting into it to begin with. https://jestjs.io/docs/snapshot-testing https://jestjs.io/docs/snapshot-testing The components I test all use classes, so there aren't embedded styles. Occasionally the generated class names change across different versions, but it is a small price to pay and easy enough to just update the shots. I'd also argue that I'd want that snapshot to fail. I'd want to know why someone made that change, especially if they were going in the other direction... f00 -> ff0000.
  - williamdclt 3y ago
    Is your point that snapshot tests are not perfect? I don’t think anyone argues against that. Are they useful and do they prevent bugs is the better question

What does your test suite look like? (What's the library is maybe a better question)

by ketzo 3y ago

What does your test suite look like? (What's the library is maybe a better question)

Good question. `@testing-library/react` and jest to run it. snapshots and a bunch of action code too. There are more modern frameworks now too that can be used for integration tests, but I haven't bothered yet as the snapshots have almost always caught the issue. It boggles my mind that react.dev doesn't start off teaching, with writing tests.

c-hendricks 3y ago
Are snapshot tests the things that fail because they can't tell "background-color: #ff0000" and "background-color:#f00" are the same?
- jtms 3y ago
  Yes, that’s right… they are of dubious utility imho.
  - latchkey 3y ago
    4 years of maintaining a react project downloaded 40k times a month argues otherwise.
    - zeroonetwothree 3y ago
      It probably works ok for a solo project but IME with large scale codebases snapshot tests are awful. You update some implementation detail of a common shared component and suddenly 5000 tests break despite the look and behavior being unchanged.
      - latchkey 3y ago
        Give me an example. Those components depend on this common shared component... if you do something that changes that shared component in such a way that it causes other snapshots to fail, I'd absolutely want to know about it. That's the whole point of dependency failures. My dependency is on MUI, which is a massively used common shared component library. If MUI changes, and it breaks my snapshots, I'd absolutely want to know about it.
      - cbovis 3y ago
        This has been my experience in the past with a heavily snapshot covered codebase. Class names can change, the structure of your HTML can change, the underlying CSS can even change and the end result is still the same because you were just refactoring. At a large enough scale it can be painful to have hundreds of snapshots break for a simple change - especially when you add required code review by others into the mix. Currently figuring out a strategy for introducing testing into an already large codebase and being very cautious of snapshot tests because of this. Experimenting with visual regression testing but early indicators suggest it could get very expensive if we're not careful about what is covered.
        latchkey 3y ago
        Give people the tools to easily update the snapshots and read the diff's of PR's (github is quite good at this imho).
        c-hendricks 3y ago
        I kinda think "just blindly update the snapshots" is teaching the wrong lesson. We removed them from our projects and haven't missed them. Visual diffing, cypress >>>> snapshots IMO.
        latchkey 3y ago
        Who said blindly update snapshots? Agreed that cypress today is probably a better solution.
- latchkey 3y ago
  It depends on the tool doing the diff on the snapshot and what you're putting into it to begin with. https://jestjs.io/docs/snapshot-testing https://jestjs.io/docs/snapshot-testing The components I test all use classes, so there aren't embedded styles. Occasionally the generated class names change across different versions, but it is a small price to pay and easy enough to just update the shots. I'd also argue that I'd want that snapshot to fail. I'd want to know why someone made that change, especially if they were going in the other direction... f00 -> ff0000.
- williamdclt 3y ago
  Is your point that snapshot tests are not perfect? I don’t think anyone argues against that. Are they useful and do they prevent bugs is the better question