4 ms·

This but automated visual regression testing (eg testing screenshots are the same) would be amazing. Anyone got any ideas how this might work with shot-scraper?

simonw 5y ago
You could absolutely get this working with GitHub Actions with a bit of creativity. I've been playing around with my own image-diff tool for this kind of thing, but it's not yet in a decent state: https://github.com/simonw/image-diff https://github.com/simonw/image-diff - there are other, better options out there such as https://github.com/mapbox/pixelmatch https://github.com/mapbox/pixelmatch Needle is an older system that did this using Selenium - updating that to work with Playwright (or Playwight via shot-scraper) would be an interesting project: https://github.com/python-needle/needle https://github.com/python-needle/needle
- trevinhofmann 5y ago
  I used pixelmatch + puppeteer at a previous job to automate this. The service would screenshot each page on the master branch and the PR branch, then comment on the PR with a link showing the Before, Diff, and After of each page that changed. I highly recommend this type of setup.
  - simonw 5y ago
    I love that idea.
  - lloydatkinson 5y ago
    Where did you upload the screenshots?
    - trevinhofmann 5y ago
      The same service kept the screenshots in its file system, and served them along with with the web pages that displayed them.
- dewski 5y ago
  I’ve built something like this and stored the diff regions in PostGIS, really easy way to build a visual diff search tool using bounding boxes.
  - simonw 5y ago
    Now you've got me thinking if that would be possible using SpatiaLite.
- westurner 5y ago
  Awesome Visual Regression Testing > lists quite a few tools and online services: https://github.com/mojoaxel/awesome-regression-testing#tools-and-frameworks https://github.com/mojoaxel/awesome-regression-testing#tools... "visual-regression": https://github.com/topics/visual-regression https://github.com/topics/visual-regression Cypress.io can run in a CI job, does Time Travel, works with DevTools debugger, can take screenshots and [headless] video, and it looks like there's a visual regression testing thing for it: https://github.com/mjhea0/cypress-visual-regression https://github.com/mjhea0/cypress-visual-regression https://docs.cypress.io/guides/overview/why-cypress#Features https://docs.cypress.io/guides/overview/why-cypress#Features

simonw 5y ago

You could absolutely get this working with GitHub Actions with a bit of creativity. I've been playing around with my own image-diff tool for this kind of thing, but it's not yet in a decent state: https://github.com/simonw/image-diff https://github.com/simonw/image-diff - there are other, better options out there such as https://github.com/mapbox/pixelmatch https://github.com/mapbox/pixelmatch Needle is an older system that did this using Selenium - updating that to work with Playwright (or Playwight via shot-scraper) would be an interesting project: https://github.com/python-needle/needle https://github.com/python-needle/needle

trevinhofmann 5y ago
I used pixelmatch + puppeteer at a previous job to automate this. The service would screenshot each page on the master branch and the PR branch, then comment on the PR with a link showing the Before, Diff, and After of each page that changed. I highly recommend this type of setup.
- simonw 5y ago
  I love that idea.
- lloydatkinson 5y ago
  Where did you upload the screenshots?
  - trevinhofmann 5y ago
    The same service kept the screenshots in its file system, and served them along with with the web pages that displayed them.
dewski 5y ago
I’ve built something like this and stored the diff regions in PostGIS, really easy way to build a visual diff search tool using bounding boxes.
- simonw 5y ago
  Now you've got me thinking if that would be possible using SpatiaLite.
westurner 5y ago
Awesome Visual Regression Testing > lists quite a few tools and online services: https://github.com/mojoaxel/awesome-regression-testing#tools-and-frameworks https://github.com/mojoaxel/awesome-regression-testing#tools... "visual-regression": https://github.com/topics/visual-regression https://github.com/topics/visual-regression Cypress.io can run in a CI job, does Time Travel, works with DevTools debugger, can take screenshots and [headless] video, and it looks like there's a visual regression testing thing for it: https://github.com/mjhea0/cypress-visual-regression https://github.com/mjhea0/cypress-visual-regression https://docs.cypress.io/guides/overview/why-cypress#Features https://docs.cypress.io/guides/overview/why-cypress#Features