3 ms·

I mean, we don't rely on pixels because we usually care more about behavior than we do the actual design. It's too brittle to rely on exact pixel locations in t

by dack 10y ago

I mean, we don't rely on pixels because we usually care more about behavior than we do the actual design. It's too brittle to rely on exact pixel locations in the face of added features and small design tweaks. Maybe it makes sense to have one or two of these types of visual checks, but they should be automated so that regenerating them is easy.

tlarkworthy 10y ago
The visual matchers are fuzzy matchers, not pixel perfect.
- jakozaur 10y ago
  It can be some nice fuzzy algorithm with learning capabilities. E.g. if you change background, but button is in same place, you can just update the test.
- marxidad 10y ago
  you can adjust the fuzziness
- mmebane 10y ago
  From my experience, the fuzzy matching works well enough to handle picking out text or icons on interfaces with semi-transparent backgrounds that change on scrolling (e.g., due to a parallax effect). It's pixel-based and not scale-invariant, though, so it won't automatically handle multiple zoom levels/resolutions.
IanCal 10y ago
> I mean, we don't rely on pixels because we usually care more about behavior than we do the actual design. True, although for some things I do still like a visual test. Your users may be used to clicking a button that looks a certain way in a certain place, and suddenly shifting that might be worthy of requiring a change to a test. You wouldn't want all your behavioural tests done like this though. > Maybe it makes sense to have one or two of these types of visual checks, but they should be automated so that regenerating them is easy. I agree. Perhaps there's a nice integration between the two? If you were able to give, say, a css selector and a template image rather than just one or the other to look for you'd be able to report these errors: * I can't find the button I was looking for, but there is something that matches the css selector [blah] that looks like this: [image] instead of [image]. Do you want to update the image in the tests? Y/N * I can't find the css selector [blah] but the button appears here [image] with [class] and [id], do you want to update the css selector in the tests? Y/N If the program is actually still fine then it'd be a quick auto-update, and if it's not then these bits of information are probably the first bit of debug you'd want to do anyway to figure out what broke the tests.
- mmebane 10y ago
  > Perhaps there's a nice integration between the two? They both run in Java, so you can combine Sikuli with Selenium pretty easily [0], so long as you're not doing headless testing. I've played around with it a bit, using Selenium to get coordinates of a container element and then having SikuliX operate just on that region of the screen. [0]: http://www.softwaretestinghelp.com/sikuli-tutorial-part-2/ http://www.softwaretestinghelp.com/sikuli-tutorial-part-2/
- fizzbatter 10y ago
  > True, although for some things I do still like a visual test. Your users may be used to clicking a button that looks a certain way in a certain place, and suddenly shifting that might be worthy of requiring a change to a test. You wouldn't want all your behavioural tests done like this though. A decent example of this would be that i've implemented frontend features before but did not notice an unintended pixel shift. I introduced a bug, albeit just visual, but a bug nonetheless - and i had no idea. Likewise, a minor visual issue like that can get through many stages of review and even deployment. Pixel interaction may be terrible for early prototyping, but the idea has merit. Or, at the very least, perhaps it could identify by css/dom, and throw warnings for pixel alterations. I like the idea of letting design people programmatically enforce standards (Though, no idea how to make them do that programmatically lol)

by dack 10y ago

tlarkworthy 10y ago

The visual matchers are fuzzy matchers, not pixel perfect.

jakozaur 10y ago
It can be some nice fuzzy algorithm with learning capabilities. E.g. if you change background, but button is in same place, you can just update the test.
marxidad 10y ago
you can adjust the fuzziness
mmebane 10y ago
From my experience, the fuzzy matching works well enough to handle picking out text or icons on interfaces with semi-transparent backgrounds that change on scrolling (e.g., due to a parallax effect). It's pixel-based and not scale-invariant, though, so it won't automatically handle multiple zoom levels/resolutions.

IanCal 10y ago

> I mean, we don't rely on pixels because we usually care more about behavior than we do the actual design. True, although for some things I do still like a visual test. Your users may be used to clicking a button that looks a certain way in a certain place, and suddenly shifting that might be worthy of requiring a change to a test. You wouldn't want all your behavioural tests done like this though. > Maybe it makes sense to have one or two of these types of visual checks, but they should be automated so that regenerating them is easy. I agree. Perhaps there's a nice integration between the two? If you were able to give, say, a css selector and a template image rather than just one or the other to look for you'd be able to report these errors: * I can't find the button I was looking for, but there is something that matches the css selector [blah] that looks like this: [image] instead of [image]. Do you want to update the image in the tests? Y/N * I can't find the css selector [blah] but the button appears here [image] with [class] and [id], do you want to update the css selector in the tests? Y/N If the program is actually still fine then it'd be a quick auto-update, and if it's not then these bits of information are probably the first bit of debug you'd want to do anyway to figure out what broke the tests.

mmebane 10y ago
> Perhaps there's a nice integration between the two? They both run in Java, so you can combine Sikuli with Selenium pretty easily [0], so long as you're not doing headless testing. I've played around with it a bit, using Selenium to get coordinates of a container element and then having SikuliX operate just on that region of the screen. [0]: http://www.softwaretestinghelp.com/sikuli-tutorial-part-2/ http://www.softwaretestinghelp.com/sikuli-tutorial-part-2/
fizzbatter 10y ago
> True, although for some things I do still like a visual test. Your users may be used to clicking a button that looks a certain way in a certain place, and suddenly shifting that might be worthy of requiring a change to a test. You wouldn't want all your behavioural tests done like this though. A decent example of this would be that i've implemented frontend features before but did not notice an unintended pixel shift. I introduced a bug, albeit just visual, but a bug nonetheless - and i had no idea. Likewise, a minor visual issue like that can get through many stages of review and even deployment. Pixel interaction may be terrible for early prototyping, but the idea has merit. Or, at the very least, perhaps it could identify by css/dom, and throw warnings for pixel alterations. I like the idea of letting design people programmatically enforce standards (Though, no idea how to make them do that programmatically lol)