4 ms·

This feels a lot like a reincarnation of cucumber/gherkin, except they've replaced business-facing text DSL with a no-code visual UI. The intention is the same

This feels a lot like a reincarnation of cucumber/gherkin, except they've replaced business-facing text DSL with a no-code visual UI. The intention is the same - to have the customer own the tests. This looks like it has a shallower learning curve to get started, but I would imagine that after a certain point it winds up being less productive to use the UI than to write code and this is ultimately why visual coding hasn't taken off. At some point someone will need to become a power user and be their organization's resident expert on Rainforest, but at that point they're spending all of their time in Rainforest and they're no longer a product owner embedded in the business. At the end of the day the business owner should be writing specifications, the engineers should be using them to create tests and they each should be collaborating closely on both.

thrower123 5y ago
I've had a couple goes at teams trying to roll out cucumber tests, and I still don't understand quite what the point is. Nobody but developers could actually manage to write any tests, and it was harder than just using the normal tools, plus maintaining all the glue besides.
- billyruffian 5y ago
  The gherkin language that cucumber uses for its specifications, written well, are brilliant at distilling what the customer wants, what the developer is going to build and how the QA is going to assert that acceptance is measured. But it has to be written collaboratively by all of those together and by the end of it you get a shared understanding. This is the most important part. The glue code behind it is a different fish, and needs someone with software engineering trainging to build it and love it.
ukd1 5y ago
If you want something it's closer to, it's sikuli script. Visually looking at the page (or whatever, tbh), then manipulating it using the keyboard and mouse. It's basically done using a KVM, so much closer to what a user would be able to see and do than something like cucumber or gherkin. However, we also allow you to test using a crowd of humans, should folks need more nuanced feedback about things, or have much more complex things to ask. Disagree on who should be writing tests; I think that's the case today as tooling doesn't support anyone but engineers (QA or not) automation things, or manual tests.
- wefarrell 5y ago
  Engineers should absolutely be writing unit tests and integration tests for their APIs. Personally I find that it brings a lot more integrity and a sense of ownership into the process when engineers are required to deliver tests. I disagree with the problem's that you mention in this article: Developers aren’t incentivized to prioritize QA testing Developers are typically evaluated based on the quantity of software they ship, and how fast they ship it. That's an organizational problem that is not universal and certainly won't be solved by a QA automation tool. Developers’ job satisfaction goes down when they’re in charge of QA Expanding upon one of the previous points:, we’ve seen that most developers just don’t enjoy doing QA. This might be the case for manual testing but for automated testing the opposite is true. Delivering tests along with code increases integrity, makes debugging significantly easier, and helps clearly communicate the intention of each feature. This only works if there is collaboration between the developers and the product owner. Clearly you have a viable product that works for many organizations, but it's certainly not a one-size-fits-all solution nor a best practice.
  - ukd1 5y ago
    Thanks. I agree re unit-tests and integration tests. Mostly we're focused on (and talking about) testing what humans end up using directly - e.g. interfaces to web apps or mobile apps. I think you're wrong re:organizational problems, it's part of it - but most developers (in my expereince) do not want to do QA outside of unit-tests and maybe integration tests. They want to write code, and ship things. Automation, at least traditionally is brittle as well as slow to write, and few love it. Whilst tooling like Cypress does improve things over Selenium, still, I've not met a developer that actually enjoys that kind of testing.

thrower123 5y ago

I've had a couple goes at teams trying to roll out cucumber tests, and I still don't understand quite what the point is. Nobody but developers could actually manage to write any tests, and it was harder than just using the normal tools, plus maintaining all the glue besides.

billyruffian 5y ago
The gherkin language that cucumber uses for its specifications, written well, are brilliant at distilling what the customer wants, what the developer is going to build and how the QA is going to assert that acceptance is measured. But it has to be written collaboratively by all of those together and by the end of it you get a shared understanding. This is the most important part. The glue code behind it is a different fish, and needs someone with software engineering trainging to build it and love it.

ukd1 5y ago

If you want something it's closer to, it's sikuli script. Visually looking at the page (or whatever, tbh), then manipulating it using the keyboard and mouse. It's basically done using a KVM, so much closer to what a user would be able to see and do than something like cucumber or gherkin. However, we also allow you to test using a crowd of humans, should folks need more nuanced feedback about things, or have much more complex things to ask. Disagree on who should be writing tests; I think that's the case today as tooling doesn't support anyone but engineers (QA or not) automation things, or manual tests.

wefarrell 5y ago
Engineers should absolutely be writing unit tests and integration tests for their APIs. Personally I find that it brings a lot more integrity and a sense of ownership into the process when engineers are required to deliver tests. I disagree with the problem's that you mention in this article: Developers aren’t incentivized to prioritize QA testing Developers are typically evaluated based on the quantity of software they ship, and how fast they ship it. That's an organizational problem that is not universal and certainly won't be solved by a QA automation tool. Developers’ job satisfaction goes down when they’re in charge of QA Expanding upon one of the previous points:, we’ve seen that most developers just don’t enjoy doing QA. This might be the case for manual testing but for automated testing the opposite is true. Delivering tests along with code increases integrity, makes debugging significantly easier, and helps clearly communicate the intention of each feature. This only works if there is collaboration between the developers and the product owner. Clearly you have a viable product that works for many organizations, but it's certainly not a one-size-fits-all solution nor a best practice.
- ukd1 5y ago
  Thanks. I agree re unit-tests and integration tests. Mostly we're focused on (and talking about) testing what humans end up using directly - e.g. interfaces to web apps or mobile apps. I think you're wrong re:organizational problems, it's part of it - but most developers (in my expereince) do not want to do QA outside of unit-tests and maybe integration tests. They want to write code, and ship things. Automation, at least traditionally is brittle as well as slow to write, and few love it. Whilst tooling like Cypress does improve things over Selenium, still, I've not met a developer that actually enjoys that kind of testing.