3 ms·

My comment was a bit too late to be seen last time [0], so I'll repeat it here: --- I was heavily involved with the Fit project for a while. (Fit is Ward Cunn

My comment was a bit too late to be seen last time [0], so I'll repeat it here: --- I was heavily involved with the Fit project for a while. (Fit is Ward Cunningham's predecessor to Cucumber. It used HTML tables rather than ASCII sentences.) I find it interesting, if not surprising, that the Cucumber community is discovering exactly the same issues that we did with Fit: namely, that it encourages brittle integration tests, and that people don't use it for its intended purpose of collobaration. I've come to believe that these problems are unsolvable. I worked on and promoted Fit for several years. Eventually, after some deep soul searching, I concluded that Fit (and by extension, Cucumber) is solving the wrong problem. The value of Fit (and Cucumber) comes from discussions with domain experts. The tools encourage a certain amount of rigor, which is good, but you can get that rigor just as easily by discussing concrete examples at a whiteboard. The actual automation of those examples, which is where Fit and Cucumber focus your attention, has minimal value. In some cases, the tools have negative value. Their tests are fundamentally more complex and less refactorable than ordinary xUnit tests. If you want to automate the examples, you're better off doing it in a programmer tool. Some people got a lot of value out of Fit, and I'm sure the same is true for Cucumber. They got that value by using it for collaboration and focusing on domain rules rather than end-to-end scenarios. My experience, though, was that the vast majority used it poorly. When a tool is used so badly, so widely, you have to start questioning whether the problem is in the tool itself. Ward and I ended up retiring Fit [1]. I've written about my reasons for abandoning it before [2] [3]. [0] https://news.ycombinator.com/item?id=7514651 https://news.ycombinator.com/item?id=7514651 [1] http://www.hanselminutes.com/151/fit-is-dead-long-live-fitnesse-with-ward-cunningham-and-james-shore http://www.hanselminutes.com/151/fit-is-dead-long-live-fitne... [2] http://www.jamesshore.com/Blog/The-Problems-With-Acceptance-Testing.html http://www.jamesshore.com/Blog/The-Problems-With-Acceptance-... [3] http://www.jamesshore.com/Blog/Acceptance-Testing-Revisited.html http://www.jamesshore.com/Blog/Acceptance-Testing-Revisited....

jdlshore 13y ago
I should mention that Aslak Hellesoy (creator of Cucumber) responded with a well thought-out comment about the issues I raised above: https://news.ycombinator.com/item?id=7519369 https://news.ycombinator.com/item?id=7519369
dragonwriter 13y ago
> I find it interesting, if not surprising, that the Cucumber community is discovering exactly the same issues that we did with Fit: namely, that it encourages brittle integration tests, and that people don't use it for its intended purpose of collobaration. > I've come to believe that these problems are unsolvable. I think their quite solvable, its just that they aren't technical problems, they are social problems, and technical solutions fail, and the people that are good at solving technical problems often aren't particularly skilled at solving social problems. I think Cucumber could be a good tool in the right workflow, but establishing the right cross-functional workflow is a very hard social problem that the people that understand the technology very often don't have the skill to solve (and, generally, don't have the social position to effectively champion solutions even if they had them.)
- mattwynne 13y ago
  I don't think it's unsolvable either. I'm often asked to come in and help teams who want to use Cucumber, because they want to collaborate better. We train them in collaboration techniques like the 3-amigos as well as reminding (brainwashing?) them about the importance of conversations and ubiquitous language. We teach them how to use Cucumber for automation too, but these visits give us a chance to do much more than that.
Morendil 13y ago
> I've come to believe that these problems are unsolvable. "Unsolvable" is strong. My take is that these problems stem inevitably from approaching the goal ass-backwards. You're trying to force a particular testing framework on people who don't care about frameworks, supposedly in the name of "better communication". There's no reason to expect that to work. The way I've advocated doing it, for years now, is to first sit with the people in question, discover how they communicate about business goals that the programming effort is expected to assist, formalize their notation as little as you can get away with and use that for acceptance testing. This should be a process of active listening, not passive recording. The client should be gently nudged away from speaking in solution-terms, for instance.
- curiouscats 13y ago
  I agree, unsolvable is not what I believe. I agree, what we have to learn is that is a frequent problem with organizational systems we create. I think by thinking about the problems we can create systems that work, but it requires some real focus - attempts that don't provide some real focus on the areas likely to create problems are likely to fail. I think it is due to some systemic factors in most of our organizations so it often requires changes beyond some simple change. A huge key is what you mention the communication between "business owners" and the development team. You also need to stop the shouting down of people who say the new attempt won't work as being negative. Given the results around the globe there is lots of evidence it won't - we need to delve into why those who are criticizing the new hope think it won't work and try to adopt improvements to make it more likely to work.
- jdlshore 13y ago
  > My take is that these problems stem inevitably from approaching the goal ass-backwards. Yes, and that's the problem that's unsolvable. (Or very, very difficult.) I'm not saying people can't figure out how to use tools like Fit or Cucumber well; I'm saying that, for a lot of complicated human reasons, they don't. And I don't think there's anything a tool can do to change that. The reason is that using Fit (or Cucumber) well requires that you change the way that you interact with your business-oriented colleagues. That's hard. It's not a technical problem; it's a social problem, and it needs face-to-face politicking. My experience is that teams that turn to tools first are even less likely to be able to solve these problems than those that don't. Many of them don't even understand that the problem exists. In order for a tool to solve this problem, it would have to force—or strongly guide—the "proper" workflow and be highly desirable for everyone involved. Aslak & company are working on https://cucumber.pro/ https://cucumber.pro/, which is a valiant attempt; it will be interesting to see if it works out.

jdlshore 13y ago

I should mention that Aslak Hellesoy (creator of Cucumber) responded with a well thought-out comment about the issues I raised above: https://news.ycombinator.com/item?id=7519369 https://news.ycombinator.com/item?id=7519369

dragonwriter 13y ago

> I find it interesting, if not surprising, that the Cucumber community is discovering exactly the same issues that we did with Fit: namely, that it encourages brittle integration tests, and that people don't use it for its intended purpose of collobaration. > I've come to believe that these problems are unsolvable. I think their quite solvable, its just that they aren't technical problems, they are social problems, and technical solutions fail, and the people that are good at solving technical problems often aren't particularly skilled at solving social problems. I think Cucumber could be a good tool in the right workflow, but establishing the right cross-functional workflow is a very hard social problem that the people that understand the technology very often don't have the skill to solve (and, generally, don't have the social position to effectively champion solutions even if they had them.)

mattwynne 13y ago
I don't think it's unsolvable either. I'm often asked to come in and help teams who want to use Cucumber, because they want to collaborate better. We train them in collaboration techniques like the 3-amigos as well as reminding (brainwashing?) them about the importance of conversations and ubiquitous language. We teach them how to use Cucumber for automation too, but these visits give us a chance to do much more than that.

Morendil 13y ago

> I've come to believe that these problems are unsolvable. "Unsolvable" is strong. My take is that these problems stem inevitably from approaching the goal ass-backwards. You're trying to force a particular testing framework on people who don't care about frameworks, supposedly in the name of "better communication". There's no reason to expect that to work. The way I've advocated doing it, for years now, is to first sit with the people in question, discover how they communicate about business goals that the programming effort is expected to assist, formalize their notation as little as you can get away with and use that for acceptance testing. This should be a process of active listening, not passive recording. The client should be gently nudged away from speaking in solution-terms, for instance.

curiouscats 13y ago
I agree, unsolvable is not what I believe. I agree, what we have to learn is that is a frequent problem with organizational systems we create. I think by thinking about the problems we can create systems that work, but it requires some real focus - attempts that don't provide some real focus on the areas likely to create problems are likely to fail. I think it is due to some systemic factors in most of our organizations so it often requires changes beyond some simple change. A huge key is what you mention the communication between "business owners" and the development team. You also need to stop the shouting down of people who say the new attempt won't work as being negative. Given the results around the globe there is lots of evidence it won't - we need to delve into why those who are criticizing the new hope think it won't work and try to adopt improvements to make it more likely to work.
jdlshore 13y ago
> My take is that these problems stem inevitably from approaching the goal ass-backwards. Yes, and that's the problem that's unsolvable. (Or very, very difficult.) I'm not saying people can't figure out how to use tools like Fit or Cucumber well; I'm saying that, for a lot of complicated human reasons, they don't. And I don't think there's anything a tool can do to change that. The reason is that using Fit (or Cucumber) well requires that you change the way that you interact with your business-oriented colleagues. That's hard. It's not a technical problem; it's a social problem, and it needs face-to-face politicking. My experience is that teams that turn to tools first are even less likely to be able to solve these problems than those that don't. Many of them don't even understand that the problem exists. In order for a tool to solve this problem, it would have to force—or strongly guide—the "proper" workflow and be highly desirable for everyone involved. Aslak & company are working on https://cucumber.pro/ https://cucumber.pro/, which is a valiant attempt; it will be interesting to see if it works out.