3 ms·
How do you handle "meaningless" visual changes? E.g. chrome version change introduces super-subtle blending changes to drop-shadows, and suddenly there are hug
by mattlondon 4y ago
How do you handle "meaningless" visual changes? E.g. chrome version change introduces super-subtle blending changes to drop-shadows, and suddenly there are huge diffs because the least significant bit of a drop-shadow's grey colour changed. What is your approach to that?
So far all tools I have used like this suffer from "false positives" like this (it's a true positive technically of course, but likely false from an engineering perspective).
- divdev_ 4y agoWe don’t at the moment differentiate, but it is definitely a feature on the roadmap. There are multiple ways to solve this and we are considering quite some of them: 1. Grayscaling browser before screenshot 2. Using custom diffing engines that enable for ignoring regions like odiff 3. Trying to be smart and analyzing images with machine learning to identify how meaningful the difference is (moonshot but worth exploring) Our approach to this in general is that if false positive like this happens we can just batch approve all the differences and be ok with that as it was a subtle rendering change not dependent on us or our direct third party dependencies.
- testingwaters4 4y agoYou could run the suite on the updated browser on the latest passed build, then automatically batch approve since you know it has to be green. Now you can run all the newest test suites in this updated browser against this approved build.
- divdev_ 4y agoexactly this is the current workflow for many people who are using visual regression tests!
- andsoitis 4y agoDid you look at Applitools? https://applitools.com/ https://applitools.com/ Their system has a way to demarcate volatile sections.