4 ms·

> Automation suites are a way to harden the software, so that even minor changes will be significant and may halt development. I find this to be the biggest is

> Automation suites are a way to harden the software, so that even minor changes will be significant and may halt development. I find this to be the biggest issue. Specifically, the "best" time to write tests is also when the code is least likely to be stable! It's easy to think "if we had just started writing browser tests when we had fewer features, we could have kept up with a testing suite". But that means your browser tests are constantly breaking and people just give up on them. The app eventually gets big enough, and has enough users, that you have to bolt on the tests later to keep bugs from hurting too many customers. This seems problem seems to be way less of an issue with regular unit tests. I guess the lack of changing dom selectors would be a big reason why.

swsieber 9y ago
Indeed - automated tests introduce more friction because they are hard to maintain and to write. So really, the best thing to do would be to make it easier to write and update automated tests. Recording clicks is of course one way, but the tests are probably more brittle for it. If you have access to the source templates, you could probably right up some kind of rudimentary compile time type checking and validation of selectors, so you could know when a new feature breaks something. The best would probably be some seamless mixture of the two - record tests in terms of the templates, with the ability to detect breaking Ng template changes...
- dokko1230 9y ago
  Hey, that's exactly what we're trying to make! I think there is a sweet spot for building a better IDE for automation...
  - swsieber 9y ago
    We're making a push for more e2e tests where I work. And we very much have a focus on reducing the friction surrounding these tests. We've settled on aiming to build a tool that keeps an index of selectors used <=> page/page structure. Then if something changes, you only have to update it once. So when you create a selector, it'll save it for that page, you can name it and reuse it. Given that we have the source templates too, I think we'll try to validate that the selector will be valid. We might be able to even add tasks for the front end guys that alerts them if the changes they're checking in will break selectors. Combine that will an idea for suggesting selectors based off the source, along with e2e script managent, it'd be a pretty slick tool. It's a pretty ambitious project though. We do hope to make it general enough to open source though. We'll see if we make it (pun intended).

swsieber 9y ago

Indeed - automated tests introduce more friction because they are hard to maintain and to write. So really, the best thing to do would be to make it easier to write and update automated tests. Recording clicks is of course one way, but the tests are probably more brittle for it. If you have access to the source templates, you could probably right up some kind of rudimentary compile time type checking and validation of selectors, so you could know when a new feature breaks something. The best would probably be some seamless mixture of the two - record tests in terms of the templates, with the ability to detect breaking Ng template changes...

dokko1230 9y ago
Hey, that's exactly what we're trying to make! I think there is a sweet spot for building a better IDE for automation...
- swsieber 9y ago
  We're making a push for more e2e tests where I work. And we very much have a focus on reducing the friction surrounding these tests. We've settled on aiming to build a tool that keeps an index of selectors used <=> page/page structure. Then if something changes, you only have to update it once. So when you create a selector, it'll save it for that page, you can name it and reuse it. Given that we have the source templates too, I think we'll try to validate that the selector will be valid. We might be able to even add tasks for the front end guys that alerts them if the changes they're checking in will break selectors. Combine that will an idea for suggesting selectors based off the source, along with e2e script managent, it'd be a pretty slick tool. It's a pretty ambitious project though. We do hope to make it general enough to open source though. We'll see if we make it (pun intended).