5 ms·

Test recorders are an antipattern. They lure you with their "no programming required!" siren call, but the generated scripts invaribly end up as a fragile, poor

Test recorders are an antipattern. They lure you with their "no programming required!" siren call, but the generated scripts invaribly end up as a fragile, poorly abstracted mess. GUI tests suites, especially those created with recorders, are prone to become unmaintainable beasts full of flaky tests. This old interview with Bret Pettichord makes good points about the use of recorders: https://youtu.be/s_CUPs6xAWw?t=590 https://youtu.be/s_CUPs6xAWw?t=590

neves 9y ago
All you said is true, but this doesn't stop Selenium IDE of being a valuable tool. You just need to know when to use it. Without it a lot systems in my company would have zero tests. Not even bad and fragile ones.
- bshacklett 9y ago
  One could argue that bad or fragile tests can be more harmful than having no tests at all.
  - danidiaz 9y ago
    Exactly, there is a degreee of flakiness beyond which you don't gain much assurance from your tests and you spend more time fighting fires in the testsuite than actually testing your app.
- mariusmg 9y ago
  >You just need to know when to use it When exactly is that ? Never ? That clusterfuck of generated code is useless.
camus2 9y ago
> Test recorders are an antipattern. Selenium IDE is a useful tool for QA and a lot of user use case. Can people stop calling anything they don't like "antipattern"? All tools are useful when used correctly. Anything can be "an anti pattern" when used incorrectly.
slgt 9y ago
Disagree 51%. Recording tests works well and saves tons of time in specific scenarios. It works especially well for "throw away" test cases that you need only for a few days, for example during development of a specific feature. So I consider your statement way too black and white.
fulafel 9y ago
Sounds like app specific recorders could be more robust and resilient to changes.
romaniv 9y ago
UI-level tests are the only kind that reliably test the full stack of your application. No amount of unis testing will catch a missing button or a wrong config setting. And while recorders present some challenges, manually coding UI-level tests make the maintainability issue an order of magnitude worse. The best way to make web UI tests stable is to involve engineers and make sure that the pages are structured in the way that make automation and validation easy. Usually the changes required for this are trivial. When they're not, it often points out something in the UI that doesn't make sense anyway, like having no confirmations for user actions. And in any case, automated UI tests are still better than manual UI test, which is what a lot of people are stuck with.
- danidiaz 9y ago
  I agree that is is important to structure the application to facilitate testing. But I find this surprising: > And while recorders present some challenges, manually coding UI-level tests make the maintainability issue an order of magnitude worse. In my experience, writing the tests manually for sufficiently "regular" applications ends up being more maitainable because you can put repeated logic in shared functions and classes, making it easier to adapt to changes in the GUI. Page Objects are an example of that: https://github.com/SeleniumHQ/selenium/wiki/PageObjects https://github.com/SeleniumHQ/selenium/wiki/PageObjects Also, writing the tests manually lets you use "explicit waits". http://elementalselenium.com/tips/47-waiting http://elementalselenium.com/tips/47-waiting Specifying exactly what we expect to happen tends to be more robust (and faster) than either using static sleeps or some kind of implicit wait. Recorders tend to have problems finding concrete things to wait for, especially for very dynamic pages. (That said, I'm not against using something like Chrome DevTools to identify elements in the page, evaluate potential selectors, and so on. That is indeed useful.)
rgrieselhuber 9y ago
I co-founded a company called Functionize (https://www.functionize.com/ https://www.functionize.com/) and we found the same, but the problem is one of assumptions: should you be generating scripts in the first place? What we ended up doing on our platform was discard the notion of generated scripts entirely. Everything becomes data, not source code. If you're willing to make the tradeoff to a cloud service vs. having the code (many businesses are), it's a great model. If you prefer having control over the code, it will drive you crazy.
lesbaker 9y ago
That's why I always used the selenium UI script as a starting point when I was doing web app QA. It was easier in my experience to take the generated output and eliminate unnecessary steps and clean up the selectors than to start from scratch.

neves 9y ago

All you said is true, but this doesn't stop Selenium IDE of being a valuable tool. You just need to know when to use it. Without it a lot systems in my company would have zero tests. Not even bad and fragile ones.

bshacklett 9y ago
One could argue that bad or fragile tests can be more harmful than having no tests at all.
- danidiaz 9y ago
  Exactly, there is a degreee of flakiness beyond which you don't gain much assurance from your tests and you spend more time fighting fires in the testsuite than actually testing your app.
mariusmg 9y ago
>You just need to know when to use it When exactly is that ? Never ? That clusterfuck of generated code is useless.

camus2 9y ago

> Test recorders are an antipattern. Selenium IDE is a useful tool for QA and a lot of user use case. Can people stop calling anything they don't like "antipattern"? All tools are useful when used correctly. Anything can be "an anti pattern" when used incorrectly.

slgt 9y ago

Disagree 51%. Recording tests works well and saves tons of time in specific scenarios. It works especially well for "throw away" test cases that you need only for a few days, for example during development of a specific feature. So I consider your statement way too black and white.

fulafel 9y ago

Sounds like app specific recorders could be more robust and resilient to changes.

romaniv 9y ago

UI-level tests are the only kind that reliably test the full stack of your application. No amount of unis testing will catch a missing button or a wrong config setting. And while recorders present some challenges, manually coding UI-level tests make the maintainability issue an order of magnitude worse. The best way to make web UI tests stable is to involve engineers and make sure that the pages are structured in the way that make automation and validation easy. Usually the changes required for this are trivial. When they're not, it often points out something in the UI that doesn't make sense anyway, like having no confirmations for user actions. And in any case, automated UI tests are still better than manual UI test, which is what a lot of people are stuck with.

danidiaz 9y ago
I agree that is is important to structure the application to facilitate testing. But I find this surprising: > And while recorders present some challenges, manually coding UI-level tests make the maintainability issue an order of magnitude worse. In my experience, writing the tests manually for sufficiently "regular" applications ends up being more maitainable because you can put repeated logic in shared functions and classes, making it easier to adapt to changes in the GUI. Page Objects are an example of that: https://github.com/SeleniumHQ/selenium/wiki/PageObjects https://github.com/SeleniumHQ/selenium/wiki/PageObjects Also, writing the tests manually lets you use "explicit waits". http://elementalselenium.com/tips/47-waiting http://elementalselenium.com/tips/47-waiting Specifying exactly what we expect to happen tends to be more robust (and faster) than either using static sleeps or some kind of implicit wait. Recorders tend to have problems finding concrete things to wait for, especially for very dynamic pages. (That said, I'm not against using something like Chrome DevTools to identify elements in the page, evaluate potential selectors, and so on. That is indeed useful.)

rgrieselhuber 9y ago

I co-founded a company called Functionize (https://www.functionize.com/ https://www.functionize.com/) and we found the same, but the problem is one of assumptions: should you be generating scripts in the first place? What we ended up doing on our platform was discard the notion of generated scripts entirely. Everything becomes data, not source code. If you're willing to make the tradeoff to a cloud service vs. having the code (many businesses are), it's a great model. If you prefer having control over the code, it will drive you crazy.

lesbaker 9y ago

That's why I always used the selenium UI script as a starting point when I was doing web app QA. It was easier in my experience to take the generated output and eliminate unnecessary steps and clean up the selectors than to start from scratch.