3 ms·

In my experience end to end tests are often difficult to write tests for. It's been difficult to maintain frameworks that model arbitrary asynchronous end to e

by dm03514 10y ago

In my experience end to end tests are often difficult to write tests for. It's been difficult to maintain frameworks that model arbitrary asynchronous end to end tests, the test frameworks often tightly mirror the use cases. I haven't really seen any good tools that allow non-technical testers to write complex end-to-end tests. This is a shameless plug for a project that is hardly more than an idea, but to address this, I am trying to create a flexible end to end test framework that deftly handles asynchronous state transitions and allows non technical people to write tests in a declarative way. It has not even reached functional proof of concept state yet :( https://github.com/dm03514/async-states-task-engine https://github.com/dm03514/async-states-task-engine The goal is to model end to end tests as state machines and define conditions to trigger transitions. There should be a code based interface allowing for arbitrary transitions using any go library, and a higher level interface, allowing a handful of operations to allow non-technical testers to model end to end tests.

UweSchmidt 10y ago
Sounds very interesting. Could you maybe do a blogpost and explain how this is supposed to work?
ben_jones 10y ago
I wonder if such a process could be done without a "non-technical friendly" DSL. Teaching people to speak to computers through declarative languages is programming in one shape or form no matter what (IMO). I wonder if you could record a user action, and add variance to the recording as a test? Break a recording into ~5 actions and then perform those actions at various speeds and points of repetition. A recording of 1) Move mouse from A to B 2) Click button Produces a range of tests including 1) Move mouse from A to B 2) Click button 3 times
- crdoconnor 10y ago
  This is basically what I did with http://hitchtest.com/ http://hitchtest.com/ The declarative language is a readable YAML, which translates each named step into a python function call with arguments (e.g. - click: submit button). The asynchronous part is 'under the hood' where it runs services which log lines of JSON. Those are watched using epoll triggers and parsed into objects which can be verified by one of the steps (e.g. check email containing name of user is sent). I wouldn't say that it's necessarily possible nor desirable for a non-programmer to write the stories using YAML, however. Even with a perfect declarative language you need a mindset that requires precise thinking in order to write executable stories and that usually means programming ability. Stories also need refactoring as much as code does, alongside engine code. However, the YAML ought to be useful as a way programmers can collaboratively write and refine stories with product owners/managers and should serve as a useful way to communicate back changes to the product that a non-programmer can understand.

by dm03514 10y ago

UweSchmidt 10y ago

Sounds very interesting. Could you maybe do a blogpost and explain how this is supposed to work?

ben_jones 10y ago

I wonder if such a process could be done without a "non-technical friendly" DSL. Teaching people to speak to computers through declarative languages is programming in one shape or form no matter what (IMO). I wonder if you could record a user action, and add variance to the recording as a test? Break a recording into ~5 actions and then perform those actions at various speeds and points of repetition. A recording of 1) Move mouse from A to B 2) Click button Produces a range of tests including 1) Move mouse from A to B 2) Click button 3 times

crdoconnor 10y ago
This is basically what I did with http://hitchtest.com/ http://hitchtest.com/ The declarative language is a readable YAML, which translates each named step into a python function call with arguments (e.g. - click: submit button). The asynchronous part is 'under the hood' where it runs services which log lines of JSON. Those are watched using epoll triggers and parsed into objects which can be verified by one of the steps (e.g. check email containing name of user is sent). I wouldn't say that it's necessarily possible nor desirable for a non-programmer to write the stories using YAML, however. Even with a perfect declarative language you need a mindset that requires precise thinking in order to write executable stories and that usually means programming ability. Stories also need refactoring as much as code does, alongside engine code. However, the YAML ought to be useful as a way programmers can collaboratively write and refine stories with product owners/managers and should serve as a useful way to communicate back changes to the product that a non-programmer can understand.