3 ms·

Can you give more details about Visual Regression Testing? If you're comparing screenshots, how do you account for dynamic content?

• mpotter 13y ago
  Sure! We use a web driver to simulate browsing in an actual client, so Javascript, etc. is executed as it normally would be.
  • 3am 13y ago
    WebDriver/Selenium? That's not really visual, like some kind of CV/OCR tool like Sikuli would be. It's a nice tool, I would be more inclined to use it if it supported more robust test environment provisioning (eg, if the pull request is a schema migration) and it gave more control over how to define my own tests, like Sauce Labs. edit: never mind, I realized you're probably using the built in screenshot features with webdriver and just diffing the images.
  • jawns 13y ago
    But what about content that changes on each page load, like a timestamp or a rotating banner ad? Would that be flagged as a regression?
    • mpotter 13y ago
      Yeah, it's not perfect but right now we're solving this with tolerance thresholds on the image diffs.
      • maerek 13y ago
        Can you share any more details on how the image diffing is done? BBC has a github repo called Wraith that uses ImageMagick and PhantomJS to accomplish a similar task (without the awesomeness of on-demand testing environments). Always curious to learn more about how people are solving the GUI testing problem on the web.
        • dbarlett 13y ago
          There's also Depicted (dpxdt)[1] - more workflow, Python instead of Ruby. [1] https://github.com/bslatkin/dpxdt https://github.com/bslatkin/dpxdt