3 ms·
We do something similar, but from my experience small changes, like fonts or lines rendering a tad different after library changes can be quite frequent. Usuall
by zubspace 4y ago
We do something similar, but from my experience small changes, like fonts or lines rendering a tad different after library changes can be quite frequent. Usually small changes you can't really see, only if you compare them as 2 layers in paint.net or something.
Adding something like an error margin for all pixels or subsections sometimes makes sense, but this can be tricky. Downscaling the image and comparing grayscale values with a small error margin is another option. It all depends on how accurate your tests have to be.
- izacus 4y agoWell, but those changes are triggered by something aren't they? So when you upgrade your font lib or pdf rendering library, you're warned that you're now generating different output and can update the golden set. Your dependencies aren't changing without a cause are they?
- deleted 4y ago[deleted]
- zubspace 4y agoYeah sure, it just starts to be a problem when you're having dozens of tests failing because of small rendering changes which can be ignored. Someone still has to look at all the test output, compare it to the old state and update the tests with the new state. In our case this happened quite a lot. This is not an issue at first, but the more you use tests like this and the more people work with your code, false positives start to drag you down.