9 ms·

Cucumber lets you write automated tests in plain language

74 points by nateb2022 1y ago

stuaxo 1y ago
Personal opinion: I absolutely hate these "Given" "When" "Then" type tests. I have to learn a whole new language to write tests, I don't get on with them at all.
headcanon 1y ago
Interesting this is being brought up, I used to use this back in 2013 when developing Rails apps, I've been thinking about what a good "standard" spec language would look like in a post-LLM world. I wonder if the cucumber syntax could help here?
- baal80spam 1y ago
  Cucumber was always touted as a DSL for business people.
- threetonesun 1y ago
  Seems to me most of the appeal of LLMs is the standard spec is just language. The next testing product is take this Jira ticket, make sure the requirements are met. Everything between code and plain language descriptions of what was built is additional complexity no one wants to deal with.
Guid_NewGuid 1y ago
In my misspent youth I've tried to introduce Cucumber/Gherkin a couple of different times but it's something with a deceptive promise. The preconditions are that you have domain experts or business people interested and willing to engage in writing or reviewing these tests. Unless you have this and it's something that those people are going to sustain when the going gets tough you're just making writing tests harder for no real benefit.
- c-hendricks 1y ago
  Ding ding ding. If it's not product / other people writing the specs, your job as a developer is now: - write specs in a cucumber fashion - write parsers to go from cucumber to whatever objects / data you expect - finally write your tests
- MoreQARespect 1y ago
  The problem isn't really that the domain experts or business people aren't interested in reviewing these tests. The problem is that the cucumber language is just very, very poorly suited for writing anything beyond very basic specifications. In practice when people try to use it for anything semi-complex they inevitably end up either writing very vague tests or repetitive tests, neither of which are of much interest to stakeholders. I've used docs generated from hitchstory for having conversations with stakeholders. I've also shared semi-human readable unit and e2e tests - especially when it's, say, JSON snippets in an API with technical stakeholders. Unfortunately, except for a few domains where specs can be expressed very concisely, cucumber isn't suitable for that.
- quietkoala 1y ago
  What is it about Cucumber that is so attractive to early in career engineers (myself included)? Is it the naive belief that others, non-engineers even, will actually want to read or write your tests? It's great in theory, but like many ideologies that attract young people, political or otherwise, its promise of egalitarian test authorship rarely holds up in practice. My experience is that it ends up as an extra abstraction layer to maintain.
  - bluGill 1y ago
    I spent a lot of time tryngito figure them out mid carreer before I realized that high priced consultants made a lot of money selling training me on it to management. The same salesmanship worked on me. Perhaps in part because they were also teaching unit testing which did turn out useful - once I got rid of the bad ideas they taught since they didn't know what worked in the real world but had a lot of money to make by sounding confident when asked questions.
  - 1y ago
dboreham 1y ago
Cobol reborn.
BJones12 1y ago
IME this allows the BAs who are writing ticket requirements to also write tests. It works. The dev may have to tweak the tests to get them to compile, but it an org that is willing to spend quite a bit of money on testing (e.g. public sector) this is a format that devs and BAs can read.
- genericspammer 1y ago
  IME? BA?
  - justusthane 1y ago
    In My Experience, Business Analyst
    - genericspammer 1y ago
      Thanks for the clarification.
  - deleted 1y ago
    [deleted]
  - tommy_axle 1y ago
    Most likely "In my experience" and "business analyst"
  - woah 1y ago
    If Mentored Excessively, Bachelor of Arts
genericspammer 1y ago
I’ve worked with cucumber for a few yeats and only ever seen developers writing Gherkin. To me it’s just another framework to learn - a failed abstraction. It’s always introduced by idealistic devs who regularly jerks off to conference talks.
aaron695 1y ago
Fuck Cucumber. Buried in the mess there are a couple of good concepts like it moves away for the 'busy-work' of atomic tests. But shit like "living documentation" makes me want to vomit. Are people dying quicker than we can teach this is bullshit? Or are people dying stupid? The idea non-coders might ever be involved with Cucumber is also fucking retarded. Last I used it, it was a mess, maybe it's better now. Part of that was those arrogant Selenium motherfuckers. All this shit is why LLMs are talking over. Fucking retards with their stupid as fuck theory's about auto documentation and Selenium refusing to implement X (i forgot) on some BS 'principal'. LLMs just get the job done, they might fuck shit up, but they are not arrogant autists about it, and LLMs don't care when they are called useless arrogant autists and double down.
fusslo 1y ago
I work with engineers in Sweden, Poland, mainland China, the Philippines, and half the US team is Indian. Even the US-born engineers cant always agree on how to communicate product ideas!
eloisius 1y ago
It sounds great on the label, but what you’ll end up with are single-use matchers for almost every line of your cukes, annoyance maintaining the regexes that map your cukes to the actual test implementation, and the “non-technical” people won’t be maintaining the cukes anyway. Sure, let them specify the cukes if they want, but then translate them into regular unit tests or specs and forget maintaining them. Just like all the other no-code inventions out there, they fail to reckon with the fact that essential complexity isn’t a problem with programming language syntax.
- deleted 1y ago
  [deleted]
- ajmurmann 1y ago
  I have had a similar experience. It just end up being a abstraction layer that gets harder and harder to map to underlying code as a project grows. If readable tests are the goal, I think the time is much better spent cleaning up regular tests and writing well-named helper methods. Especially in Ruby you can get pretty close to tests that can be deciphered on a high level by non-programmers. That said, I've never had a PM or other non-programmer actually want to do that.
- dmix 1y ago
  > but what you’ll end up with are single-use matchers for almost every line of your cukes Capybara has this problem as well as do other web acceptance tests. Maybe not every time a new matcher but frequent enough. Mostly because of the high variability of HTML paths. You have to design your HTML in a way (use lots of [title] tags and standard HTML) if you want to avoid it.
- afandian 1y ago
  I have ended up writing BDD scenarios as plain code comments with conventions like capital “GIVEN” etc. A simple script to scrape test files and generate a markdown file to browse. Then you refactor your common helper functions as natural, but you’re not forced to enclose them in steps.
- burnt-resistor 1y ago
pjm331 1y ago
I've always thought this looks cool but then you realize the whole thing is driven by regexs on the "plain english" and I recoil in horror Given /^a nice new bike$/ do expect(bike).to be_shiny end so I've traded translating business requirements into specs for trying to regex against business requirements - and probably a lot of back and forth telling people they wrote their gherkin wrong have never actually tried it for that reason - it just seems worse in every way
nativeit 1y ago
Aren’t syntaxes and semantics purposeful? Isn’t knowing the proper syntax and semantic structure part of how we design good software? I don’t understand the desire for “plain language” beyond the notion that ignorant people want to stay ignorant.
drewcoo 1y ago
Extra layers of busywork that nobody else wants to look at. If your job is polishing a turd, make the turd shine like a cucumber. I have seen countless hours wasted on making Cuke/Gherkin "pretty" instead of accomplishing anything for the business.
the_other 1y ago
Engineers should refuse to write Gherkin. It's not meant to be for them, it's meant to be for non-engineers to tell engineers (or the systems the engineers make) what they expect will happen. It looks like some flavour of user stories for this reason. Engineers' responsibility begins at the "code behind" the gherkin layer. Someone needs to enforce this. Sadly, IMO because gherkin looks like structured language (i.e "code"), and because that's what engineers do, they end up doing it. Either they tricked themselves into thinking they should, or the product team tacitly assumed it was a code thing. If you're an engineer and you're writing gherkin, you should consider this a "code smell" and consider ditching it. Go for a spec tool one layer closer to your main domain. i.e. just write normal unit & integration tests. I'm not saying "don't do BDD" or "don't do acceptance tests". You should do that! I'm saying "gherkin should allow product owners/managers to take ownership of reporting on how well the app meets their expectations". But most typically, they don't get it, or they don't want to do it. I recently worked somewhere where we had dedicated devs-in-test. They wrote most of the e2e tests for the application. They wrote it in Gherkin, which meant that they _also_ wrote the JS/Selenium tests behind them. So they were basically writing two sets of scripts for each test. At some point they explored using a 3rd party cross-device testing platform (lambda test, maybe?) who had their own in-house test script that looked like Gherkin, so it needed a JS/TS code-behind to actually run the tests. So they did it all again!! Such a waste.
- bluGill 1y ago
  Gherkin is useful as a test name. However this should be a string that the test framework uses only for the ui. The executable parts should be normal functions that if needed call subfunctions that take normal parameters.
- morkalork 1y ago
  This managerial anti-pattern crops up with so many DIY tools; like reporting/dashboard platforms where engineers go through all the work funneling data to a 3rd party tool just for management to open tickets asking the same engineers to build dashboards in the UI. If we knew we were going to get stuck making the reports too, we'd probably choose any option that let's you specify them in some kind of code/markup instead and commit it to version tracking rather than a proprietary platform and their own internal history etc. etc.
ianbicking 1y ago
I wish the doctest approach won over the BDD approach. I still find myself recreating the doctest approach in miniature... Specifically I try to make a nice string representation of state, and then tests match off that state. So for instance I don't want to say expect(x.length).toBe(0) (one syntax, Cucumber goes further in that syntactic direction). Because if the length is 1 then 1!=0 is a very opaque failure. Or if you do expect(x[0].attr).toBe("y") then you've tested one thing but you have to have another test for list length, other attributes, etc. Often you'll leave the others out, expecting them to be ok, but sometimes they aren't... Investing in a good, stable, matchable string representation will give you a great tool for any number of tests, and it's not just a great tool to verify but also to debug your tests.
- paradox460 1y ago
  In elixir land, the doctest is a common sight. I wouldn't say it won out over the unit test, but it's still fairly common
kunley 1y ago
Deja vu. Cucumber was all the rage in 2008, IIRC..
ascendantlogic 1y ago
Nothing made me more enraged than trying to express test steps via regex. I will never go down that road again.
ageitgey 1y ago
I posted this comment in 2013 [1] and I stand by it 12 years later: Cucumber tries to solve the problem of turning customer requirements into 'real code'. In exchange for that worthwhile benefit, it asks you to implement the most terrible, reg-ex based spaghetti code imaginable. The problem is that it doesn't solve the original problem AT ALL. And then you are left with terrible reg-ex driven spaghetti code. Like the Jamie Zawinski saying, "now you have two problems". The lesson here is that software development processes have to pass the 'human nature' test. The software industry has largely abandoned waterfall development because it just doesn't work well in practice. It doesn't work because people don't know perfectly what they want before they build it. Agile processes usually are much more efficient because they are more closely aligned to how humans solve problems in the real world. Cucumber suffers from the same issue of being disconnected with reality. In theory, you can find a customer who can give you perfectly written use cases and you can cut-and-paste those into your cukes. In practice, that never, ever works. So let's all stop wasting our time pretending it was a good idea now that it has been shown to not work. [1] https://news.ycombinator.com/item?id=6411787#6412391 https://news.ycombinator.com/item?id=6411787#6412391
- blablabla123 1y ago
  Nobody is going to read these test cases any way, except when troubleshooting or adding new code. But that is never any of the stakeholders It's quite funny that now with AI-based no/low code the same thing is attempted again. Even if the regexes might disappear, it's even more text, with even less structure (assuming anyone checked those prompts into git in the first place)
  - MoreQARespect 1y ago
    I use executable specs quite a lot with stakeholders. I'd never ask them to write them, but I will often write a spec/test based upon their two sentence jira and then screenshare and walk them through my interpretation to get feedback early (i.e. before ive wasted time building the wrong thing). Cucumber/gherkin is awful at this of course, and the regex thing was a terrible idea but it's not the only tool. The idea that tests should be split into a specification layer and execution layer is a good one that should have taken off by now.
Exoristos 1y ago
As a software developer, I have enjoyed occasionally writing Cucumber/Gherkin. However, the roles that were _supposed_ to be writing it did _not_ enjoy it, either refusing to write it or writing unusably-brief and -malformed versions even after extensive training.
MantisShrimp90 1y ago
This sparks a thought experiment I've been having. In this world where llms can be thought of as the new layer of compilers, things like pickle are likely going to be the main unit of work for humans. Only now instead of this developing brittle generated tests, it will instead be used by the llm as guidance to generate the actual code and tests. Before people jump down my throat, I know we are nowhere near that today and I promise I'm not pitching this to my leadership because they would gobble it up too fast. But for us engineers, I think there is an interesting space for thinking of llms as akin to garbage collection, a feature that allows us to abstract to a slightly higher level of thought. Yes we still need to know how to check under the hood, but this is looking like the right level of precision-flexibility ratio that llms thrive in
- jefflinwood 1y ago
  I'm building this over at https://zapcircle.com/ https://zapcircle.com/ - it's still a work in progress, but it's all open source. The idea behind it was that Behavior-Driven Development might be a great idea, but Gherkin was a pain to work with. LLMs bridge that gap now: https://www.jefflinwood.com/2025/zapcircle-bdd-2025/ https://www.jefflinwood.com/2025/zapcircle-bdd-2025/
hakunin 1y ago
Good old cucumber. I remember a discussion over coffee with someone who swore by it back in the day (~2010). They forced customers to sit down and write these things together side by side. Kind of cringe in retrospect. Wonder what happened to them. Business people don't really care about this stuff. Over the years I realize more and more that we engineers are naive in thinking that business side is concerned with edge cases and race conditions. Just to give an analogy software people might get better, if you come to a lawyer because, say, you want to buy a house, you are not going to sit down with them and say "given I want to buy a house, when the seller hides water damage costing over $2000, I get to walk away from the deal". You just hope the lawyer is good and will protect you from various edge cases. You have a lot more to deal with than just closing paperwork. You probably are thinking about renovating, moving, getting inspection, etc. Businesses are just like that with engineering. They don't want to sit down and meticulously analyze every possible edge case. They have other things to do. Especially when stakes are not that high. Most of these errors can probably be resolved with a phone call and a database edit. I think this is probably for the best. A good engineer will make sure you're standing on a solid ground, and ask the right questions at the right time. They wouldn't need this amount of hand holding. Leave business time to focus on making deals, connections, organizing the whole operation to move forward, etc. Let them give you vague requirements, and crystallize them yourself. It's way better than a micro-managing business that thinks they know exactly how everything should be. P.S. Also, I'm not sure why everyone is so hung up on regex = bad, it's not like switching to an AST-based language would've made anything better here. Regex is fine imo, just the entire concept isn't.
pftg 1y ago
You can easily build your own clear, simple DSL with Minitest, Pytest, or JUnit - no overthinking needed. Engineers can whip up more readable and reliable tests quickly. Since only engineers are writing tests, there's no need for unnecessary complications.
thom 1y ago
I am very much for internal DSLs for functional testing, but have never seen the point of external DSLs (it feels wrong that anyone but the implementing team could or should create or maintain them). I passionately believe you should refactor and use abstractions in your test code thought, and in my experience that ends up looking quite like parameterised steps (again, in code rather than English preferably).
- dgunay 1y ago
  We did a lot of this where I work to make using cucumber easier, and when we later stripped cucumber out it still was incredibly useful.
KeenanKeenan 1y ago
I love cucumber but I never follow the pattern to (1) write the feature file, (2) generate partially implemented tests with said file, (3) finish implementing the tests + subject under test. It's better to (1) write the test, (2) then later once all that's done, extract documentation in a human-friendly format.
didip 1y ago
This tool is awful. I hated it a decade ago and I still hate its successors in other languages. It creates unnecessary work for something that's never to be seen by real stakeholders. There is a new trend these days using LLM that is similar to Cucumber: Spec Driven Development using AI. You'd be left disappointed again.
solannou 1y ago
I'm a little bit astonished by the comments mentioning regex. I used Gerkins at work and there was no regex at all. Once you write it, you can use the sentence for building other tests if you design it for being flexible and variable, i.e. design well enough your code. Given I'm connected as ___ with password ___. When I click on the element __. Then the element ___ is present. Then the element ___ is present. [...] For critical use case, it was enough. For more that's a bad idea: it take way too much maintenance cost.
dgunay 1y ago
I've commented elsewhere on lack of buy-in from stakeholders as a huge pitfall. On a technical level, cucumber is also at odds with the need for a test suite to be easily maintainable. What I mean is that each test (especially e2e tests) will want to do some setup/initialization. This is usually expressed as step definitions. Over time, an undisciplined team may write several slightly different but effectively identical step definitions. They may also combine multiple steps into bigger steps because usually a spec writer doesn't want to exhaustively define every piece of setup, they just want to write "Given all 200 pieces of input data and mocks are magically in the right place..." I was able to wrangle the specs into composability using Rule and Background blocks, but at that point we were just programming tests with a shitty layer over the actual code.
antisol 1y ago
> Cucumber lets you write automated tests in plain language No, it doesn't. Or to be more precise: Cucumber lets you write your tests twice - once in something that isn't nearly as close "plain language" as it seems at first glance, and then again in a horrible mismash of regex spaghetti code that will make you wish you'd been born a thousand years before computers were invented. I was forced to try to use it about 15 years ago and spent literally twice as much time debugging the testing framework as I did actually implementing code - velocity on actual feature development ground to a halt. I think the most hilarious part about this was that it was completely incapable of finding the most common class of problem we tended to run into at the time - cross-browser presentation issues. One of the big problems you'll find if you try to use this trash is that nobody except a software engineer will be really able to write tests in a consistent enough way that you'll be able to implement them without a truly horrible regex soup: even if you can actually get a non-software-engineer to write cucumber tests for you (dubious), the non-engineer-types will write "given I'm logged in" for one test and then "given I have logged in" for another, and "given I gave gone through the login process" for a third. And it doesn't matter how many regexs you implement to try to cover all their variations in their language, or how clever your regexs are - the nature of natural language means there's near-infinite variations, and they'll come up with new and more annoying variations just when you start feeling confident. And that's before we even start talking about the ambiguity of natural language. I did manage to find a decent use for cucumber-style language a couple of years ago, though - we started commenting our unit tests using this style: def test_something(): """ test some thing """ # given some prerequisite set_up_prerequisite() # when I perform some action perform_some_action() # then some condition should be true, because <reason> assert some_condition is True At first I was dubious, given my previous experience, but I quickly found this to be a very nice way to document your unit tests and make them super-readable and easy to understand at a glance. And you don't even need to install any software to start doing it. I highly recommend it!
- burnt-resistor 1y ago
  It's dashboard-oriented development for MBAs and PMPs.
paradox460 1y ago
I thought we might have killed the crap cucumber was trying a decade ago. It's horrible. You wind up writing more matching code to turn that plain text into a test flow than you wouls of you just wrote the darn test
midenginedcoupe 1y ago
There's a reason why I call BDD a Billion Dollar Disaster. I've not yet seen a Cucumber or equivalent suite not permanently broken on a mature codebase. Typically a dedicated testing team will fight tooth and nail that this is what the customer needs, even though nobody from the business side has ever looked or wants to look at the test definitions or even the output. Then of course it can't match changes to app behaviour and some poor sod has to go in periodically to fix it up after the fact. BDD in whatever implemenation is a gigantic waste of everyone's time.
BobbyTables2 1y ago
I think the promise of cucumber/gherkin is that one cannot write overly complex tests with them. I’ve seen (somewhat) experienced engineers get lost in complex pytest environments that used a lot of mock objects. Over time, they lost clarity on what was being tested, how, and why. The real value is to have an automated set of simple smoke tests. Frankly, Cucumber serves a nonnegotiable reminder to me that the automated tests should be simple and straightforward. For anything extremely complex, I’ll write unit tests instead of trying to craft ridiculous gherkin steps only used for a specific scenario. None of this “when it is midnight of a blood moon in a leap year and the second request attempt has timed out…”
DmitryOlshansky 1y ago
Used cucumber as living specification for our search engine. Sooner or later we realized it was just matching a list of expected JSONs with a bunch of awkward Given, And Then mixed in. Scraped that stuff for our own Kotlin DSL, made the whole thing more flexible and faster to boot.

74 points by nateb2022 1y ago

Personal opinion: I absolutely hate these "Given" "When" "Then" type tests. I have to learn a whole new language to write tests, I don't get on with them at all.

headcanon 1y ago

Interesting this is being brought up, I used to use this back in 2013 when developing Rails apps, I've been thinking about what a good "standard" spec language would look like in a post-LLM world. I wonder if the cucumber syntax could help here?

baal80spam 1y ago
Cucumber was always touted as a DSL for business people.
threetonesun 1y ago
Seems to me most of the appeal of LLMs is the standard spec is just language. The next testing product is take this Jira ticket, make sure the requirements are met. Everything between code and plain language descriptions of what was built is additional complexity no one wants to deal with.

Guid_NewGuid 1y ago

In my misspent youth I've tried to introduce Cucumber/Gherkin a couple of different times but it's something with a deceptive promise. The preconditions are that you have domain experts or business people interested and willing to engage in writing or reviewing these tests. Unless you have this and it's something that those people are going to sustain when the going gets tough you're just making writing tests harder for no real benefit.

c-hendricks 1y ago
Ding ding ding. If it's not product / other people writing the specs, your job as a developer is now: - write specs in a cucumber fashion - write parsers to go from cucumber to whatever objects / data you expect - finally write your tests
MoreQARespect 1y ago
The problem isn't really that the domain experts or business people aren't interested in reviewing these tests. The problem is that the cucumber language is just very, very poorly suited for writing anything beyond very basic specifications. In practice when people try to use it for anything semi-complex they inevitably end up either writing very vague tests or repetitive tests, neither of which are of much interest to stakeholders. I've used docs generated from hitchstory for having conversations with stakeholders. I've also shared semi-human readable unit and e2e tests - especially when it's, say, JSON snippets in an API with technical stakeholders. Unfortunately, except for a few domains where specs can be expressed very concisely, cucumber isn't suitable for that.
quietkoala 1y ago
What is it about Cucumber that is so attractive to early in career engineers (myself included)? Is it the naive belief that others, non-engineers even, will actually want to read or write your tests? It's great in theory, but like many ideologies that attract young people, political or otherwise, its promise of egalitarian test authorship rarely holds up in practice. My experience is that it ends up as an extra abstraction layer to maintain.
- bluGill 1y ago
  I spent a lot of time tryngito figure them out mid carreer before I realized that high priced consultants made a lot of money selling training me on it to management. The same salesmanship worked on me. Perhaps in part because they were also teaching unit testing which did turn out useful - once I got rid of the bad ideas they taught since they didn't know what worked in the real world but had a lot of money to make by sounding confident when asked questions.
- 1y ago

dboreham 1y ago

Cobol reborn.

BJones12 1y ago

IME this allows the BAs who are writing ticket requirements to also write tests. It works. The dev may have to tweak the tests to get them to compile, but it an org that is willing to spend quite a bit of money on testing (e.g. public sector) this is a format that devs and BAs can read.

genericspammer 1y ago
IME? BA?
- justusthane 1y ago
  In My Experience, Business Analyst
  - genericspammer 1y ago
    Thanks for the clarification.
- deleted 1y ago
  [deleted]
- tommy_axle 1y ago
  Most likely "In my experience" and "business analyst"
- woah 1y ago
  If Mentored Excessively, Bachelor of Arts

genericspammer 1y ago

I’ve worked with cucumber for a few yeats and only ever seen developers writing Gherkin. To me it’s just another framework to learn - a failed abstraction. It’s always introduced by idealistic devs who regularly jerks off to conference talks.

aaron695 1y ago

Fuck Cucumber. Buried in the mess there are a couple of good concepts like it moves away for the 'busy-work' of atomic tests. But shit like "living documentation" makes me want to vomit. Are people dying quicker than we can teach this is bullshit? Or are people dying stupid? The idea non-coders might ever be involved with Cucumber is also fucking retarded. Last I used it, it was a mess, maybe it's better now. Part of that was those arrogant Selenium motherfuckers. All this shit is why LLMs are talking over. Fucking retards with their stupid as fuck theory's about auto documentation and Selenium refusing to implement X (i forgot) on some BS 'principal'. LLMs just get the job done, they might fuck shit up, but they are not arrogant autists about it, and LLMs don't care when they are called useless arrogant autists and double down.

fusslo 1y ago

I work with engineers in Sweden, Poland, mainland China, the Philippines, and half the US team is Indian. Even the US-born engineers cant always agree on how to communicate product ideas!

eloisius 1y ago

It sounds great on the label, but what you’ll end up with are single-use matchers for almost every line of your cukes, annoyance maintaining the regexes that map your cukes to the actual test implementation, and the “non-technical” people won’t be maintaining the cukes anyway. Sure, let them specify the cukes if they want, but then translate them into regular unit tests or specs and forget maintaining them. Just like all the other no-code inventions out there, they fail to reckon with the fact that essential complexity isn’t a problem with programming language syntax.

deleted 1y ago
[deleted]
ajmurmann 1y ago
I have had a similar experience. It just end up being a abstraction layer that gets harder and harder to map to underlying code as a project grows. If readable tests are the goal, I think the time is much better spent cleaning up regular tests and writing well-named helper methods. Especially in Ruby you can get pretty close to tests that can be deciphered on a high level by non-programmers. That said, I've never had a PM or other non-programmer actually want to do that.
dmix 1y ago
> but what you’ll end up with are single-use matchers for almost every line of your cukes Capybara has this problem as well as do other web acceptance tests. Maybe not every time a new matcher but frequent enough. Mostly because of the high variability of HTML paths. You have to design your HTML in a way (use lots of [title] tags and standard HTML) if you want to avoid it.
afandian 1y ago
I have ended up writing BDD scenarios as plain code comments with conventions like capital “GIVEN” etc. A simple script to scrape test files and generate a markdown file to browse. Then you refactor your common helper functions as natural, but you’re not forced to enclose them in steps.
burnt-resistor 1y ago

pjm331 1y ago

I've always thought this looks cool but then you realize the whole thing is driven by regexs on the "plain english" and I recoil in horror Given /^a nice new bike$/ do expect(bike).to be_shiny end so I've traded translating business requirements into specs for trying to regex against business requirements - and probably a lot of back and forth telling people they wrote their gherkin wrong have never actually tried it for that reason - it just seems worse in every way

nativeit 1y ago

Aren’t syntaxes and semantics purposeful? Isn’t knowing the proper syntax and semantic structure part of how we design good software? I don’t understand the desire for “plain language” beyond the notion that ignorant people want to stay ignorant.

drewcoo 1y ago

Extra layers of busywork that nobody else wants to look at. If your job is polishing a turd, make the turd shine like a cucumber. I have seen countless hours wasted on making Cuke/Gherkin "pretty" instead of accomplishing anything for the business.

the_other 1y ago

Engineers should refuse to write Gherkin. It's not meant to be for them, it's meant to be for non-engineers to tell engineers (or the systems the engineers make) what they expect will happen. It looks like some flavour of user stories for this reason. Engineers' responsibility begins at the "code behind" the gherkin layer. Someone needs to enforce this. Sadly, IMO because gherkin looks like structured language (i.e "code"), and because that's what engineers do, they end up doing it. Either they tricked themselves into thinking they should, or the product team tacitly assumed it was a code thing. If you're an engineer and you're writing gherkin, you should consider this a "code smell" and consider ditching it. Go for a spec tool one layer closer to your main domain. i.e. just write normal unit & integration tests. I'm not saying "don't do BDD" or "don't do acceptance tests". You should do that! I'm saying "gherkin should allow product owners/managers to take ownership of reporting on how well the app meets their expectations". But most typically, they don't get it, or they don't want to do it. I recently worked somewhere where we had dedicated devs-in-test. They wrote most of the e2e tests for the application. They wrote it in Gherkin, which meant that they _also_ wrote the JS/Selenium tests behind them. So they were basically writing two sets of scripts for each test. At some point they explored using a 3rd party cross-device testing platform (lambda test, maybe?) who had their own in-house test script that looked like Gherkin, so it needed a JS/TS code-behind to actually run the tests. So they did it all again!! Such a waste.

bluGill 1y ago
Gherkin is useful as a test name. However this should be a string that the test framework uses only for the ui. The executable parts should be normal functions that if needed call subfunctions that take normal parameters.
morkalork 1y ago
This managerial anti-pattern crops up with so many DIY tools; like reporting/dashboard platforms where engineers go through all the work funneling data to a 3rd party tool just for management to open tickets asking the same engineers to build dashboards in the UI. If we knew we were going to get stuck making the reports too, we'd probably choose any option that let's you specify them in some kind of code/markup instead and commit it to version tracking rather than a proprietary platform and their own internal history etc. etc.

ianbicking 1y ago

I wish the doctest approach won over the BDD approach. I still find myself recreating the doctest approach in miniature... Specifically I try to make a nice string representation of state, and then tests match off that state. So for instance I don't want to say expect(x.length).toBe(0) (one syntax, Cucumber goes further in that syntactic direction). Because if the length is 1 then 1!=0 is a very opaque failure. Or if you do expect(x[0].attr).toBe("y") then you've tested one thing but you have to have another test for list length, other attributes, etc. Often you'll leave the others out, expecting them to be ok, but sometimes they aren't... Investing in a good, stable, matchable string representation will give you a great tool for any number of tests, and it's not just a great tool to verify but also to debug your tests.

paradox460 1y ago
In elixir land, the doctest is a common sight. I wouldn't say it won out over the unit test, but it's still fairly common

kunley 1y ago

Deja vu. Cucumber was all the rage in 2008, IIRC..

ascendantlogic 1y ago

Nothing made me more enraged than trying to express test steps via regex. I will never go down that road again.

ageitgey 1y ago

I posted this comment in 2013 [1] and I stand by it 12 years later: Cucumber tries to solve the problem of turning customer requirements into 'real code'. In exchange for that worthwhile benefit, it asks you to implement the most terrible, reg-ex based spaghetti code imaginable. The problem is that it doesn't solve the original problem AT ALL. And then you are left with terrible reg-ex driven spaghetti code. Like the Jamie Zawinski saying, "now you have two problems". The lesson here is that software development processes have to pass the 'human nature' test. The software industry has largely abandoned waterfall development because it just doesn't work well in practice. It doesn't work because people don't know perfectly what they want before they build it. Agile processes usually are much more efficient because they are more closely aligned to how humans solve problems in the real world. Cucumber suffers from the same issue of being disconnected with reality. In theory, you can find a customer who can give you perfectly written use cases and you can cut-and-paste those into your cukes. In practice, that never, ever works. So let's all stop wasting our time pretending it was a good idea now that it has been shown to not work. [1] https://news.ycombinator.com/item?id=6411787#6412391 https://news.ycombinator.com/item?id=6411787#6412391

blablabla123 1y ago
Nobody is going to read these test cases any way, except when troubleshooting or adding new code. But that is never any of the stakeholders It's quite funny that now with AI-based no/low code the same thing is attempted again. Even if the regexes might disappear, it's even more text, with even less structure (assuming anyone checked those prompts into git in the first place)
- MoreQARespect 1y ago
  I use executable specs quite a lot with stakeholders. I'd never ask them to write them, but I will often write a spec/test based upon their two sentence jira and then screenshare and walk them through my interpretation to get feedback early (i.e. before ive wasted time building the wrong thing). Cucumber/gherkin is awful at this of course, and the regex thing was a terrible idea but it's not the only tool. The idea that tests should be split into a specification layer and execution layer is a good one that should have taken off by now.

Exoristos 1y ago

As a software developer, I have enjoyed occasionally writing Cucumber/Gherkin. However, the roles that were _supposed_ to be writing it did _not_ enjoy it, either refusing to write it or writing unusably-brief and -malformed versions even after extensive training.

MantisShrimp90 1y ago

This sparks a thought experiment I've been having. In this world where llms can be thought of as the new layer of compilers, things like pickle are likely going to be the main unit of work for humans. Only now instead of this developing brittle generated tests, it will instead be used by the llm as guidance to generate the actual code and tests. Before people jump down my throat, I know we are nowhere near that today and I promise I'm not pitching this to my leadership because they would gobble it up too fast. But for us engineers, I think there is an interesting space for thinking of llms as akin to garbage collection, a feature that allows us to abstract to a slightly higher level of thought. Yes we still need to know how to check under the hood, but this is looking like the right level of precision-flexibility ratio that llms thrive in

jefflinwood 1y ago
I'm building this over at https://zapcircle.com/ https://zapcircle.com/ - it's still a work in progress, but it's all open source. The idea behind it was that Behavior-Driven Development might be a great idea, but Gherkin was a pain to work with. LLMs bridge that gap now: https://www.jefflinwood.com/2025/zapcircle-bdd-2025/ https://www.jefflinwood.com/2025/zapcircle-bdd-2025/

hakunin 1y ago

Good old cucumber. I remember a discussion over coffee with someone who swore by it back in the day (~2010). They forced customers to sit down and write these things together side by side. Kind of cringe in retrospect. Wonder what happened to them. Business people don't really care about this stuff. Over the years I realize more and more that we engineers are naive in thinking that business side is concerned with edge cases and race conditions. Just to give an analogy software people might get better, if you come to a lawyer because, say, you want to buy a house, you are not going to sit down with them and say "given I want to buy a house, when the seller hides water damage costing over $2000, I get to walk away from the deal". You just hope the lawyer is good and will protect you from various edge cases. You have a lot more to deal with than just closing paperwork. You probably are thinking about renovating, moving, getting inspection, etc. Businesses are just like that with engineering. They don't want to sit down and meticulously analyze every possible edge case. They have other things to do. Especially when stakes are not that high. Most of these errors can probably be resolved with a phone call and a database edit. I think this is probably for the best. A good engineer will make sure you're standing on a solid ground, and ask the right questions at the right time. They wouldn't need this amount of hand holding. Leave business time to focus on making deals, connections, organizing the whole operation to move forward, etc. Let them give you vague requirements, and crystallize them yourself. It's way better than a micro-managing business that thinks they know exactly how everything should be. P.S. Also, I'm not sure why everyone is so hung up on regex = bad, it's not like switching to an AST-based language would've made anything better here. Regex is fine imo, just the entire concept isn't.

pftg 1y ago

You can easily build your own clear, simple DSL with Minitest, Pytest, or JUnit - no overthinking needed. Engineers can whip up more readable and reliable tests quickly. Since only engineers are writing tests, there's no need for unnecessary complications.

thom 1y ago

I am very much for internal DSLs for functional testing, but have never seen the point of external DSLs (it feels wrong that anyone but the implementing team could or should create or maintain them). I passionately believe you should refactor and use abstractions in your test code thought, and in my experience that ends up looking quite like parameterised steps (again, in code rather than English preferably).

dgunay 1y ago
We did a lot of this where I work to make using cucumber easier, and when we later stripped cucumber out it still was incredibly useful.

KeenanKeenan 1y ago

I love cucumber but I never follow the pattern to (1) write the feature file, (2) generate partially implemented tests with said file, (3) finish implementing the tests + subject under test. It's better to (1) write the test, (2) then later once all that's done, extract documentation in a human-friendly format.

didip 1y ago

This tool is awful. I hated it a decade ago and I still hate its successors in other languages. It creates unnecessary work for something that's never to be seen by real stakeholders. There is a new trend these days using LLM that is similar to Cucumber: Spec Driven Development using AI. You'd be left disappointed again.

solannou 1y ago

I'm a little bit astonished by the comments mentioning regex. I used Gerkins at work and there was no regex at all. Once you write it, you can use the sentence for building other tests if you design it for being flexible and variable, i.e. design well enough your code. Given I'm connected as ___ with password ___. When I click on the element __. Then the element ___ is present. Then the element ___ is present. [...] For critical use case, it was enough. For more that's a bad idea: it take way too much maintenance cost.

dgunay 1y ago

I've commented elsewhere on lack of buy-in from stakeholders as a huge pitfall. On a technical level, cucumber is also at odds with the need for a test suite to be easily maintainable. What I mean is that each test (especially e2e tests) will want to do some setup/initialization. This is usually expressed as step definitions. Over time, an undisciplined team may write several slightly different but effectively identical step definitions. They may also combine multiple steps into bigger steps because usually a spec writer doesn't want to exhaustively define every piece of setup, they just want to write "Given all 200 pieces of input data and mocks are magically in the right place..." I was able to wrangle the specs into composability using Rule and Background blocks, but at that point we were just programming tests with a shitty layer over the actual code.

antisol 1y ago

> Cucumber lets you write automated tests in plain language No, it doesn't. Or to be more precise: Cucumber lets you write your tests twice - once in something that isn't nearly as close "plain language" as it seems at first glance, and then again in a horrible mismash of regex spaghetti code that will make you wish you'd been born a thousand years before computers were invented. I was forced to try to use it about 15 years ago and spent literally twice as much time debugging the testing framework as I did actually implementing code - velocity on actual feature development ground to a halt. I think the most hilarious part about this was that it was completely incapable of finding the most common class of problem we tended to run into at the time - cross-browser presentation issues. One of the big problems you'll find if you try to use this trash is that nobody except a software engineer will be really able to write tests in a consistent enough way that you'll be able to implement them without a truly horrible regex soup: even if you can actually get a non-software-engineer to write cucumber tests for you (dubious), the non-engineer-types will write "given I'm logged in" for one test and then "given I have logged in" for another, and "given I gave gone through the login process" for a third. And it doesn't matter how many regexs you implement to try to cover all their variations in their language, or how clever your regexs are - the nature of natural language means there's near-infinite variations, and they'll come up with new and more annoying variations just when you start feeling confident. And that's before we even start talking about the ambiguity of natural language. I did manage to find a decent use for cucumber-style language a couple of years ago, though - we started commenting our unit tests using this style: def test_something(): """ test some thing """ # given some prerequisite set_up_prerequisite() # when I perform some action perform_some_action() # then some condition should be true, because <reason> assert some_condition is True At first I was dubious, given my previous experience, but I quickly found this to be a very nice way to document your unit tests and make them super-readable and easy to understand at a glance. And you don't even need to install any software to start doing it. I highly recommend it!

burnt-resistor 1y ago
It's dashboard-oriented development for MBAs and PMPs.

paradox460 1y ago

I thought we might have killed the crap cucumber was trying a decade ago. It's horrible. You wind up writing more matching code to turn that plain text into a test flow than you wouls of you just wrote the darn test

midenginedcoupe 1y ago

There's a reason why I call BDD a Billion Dollar Disaster. I've not yet seen a Cucumber or equivalent suite not permanently broken on a mature codebase. Typically a dedicated testing team will fight tooth and nail that this is what the customer needs, even though nobody from the business side has ever looked or wants to look at the test definitions or even the output. Then of course it can't match changes to app behaviour and some poor sod has to go in periodically to fix it up after the fact. BDD in whatever implemenation is a gigantic waste of everyone's time.

BobbyTables2 1y ago

I think the promise of cucumber/gherkin is that one cannot write overly complex tests with them. I’ve seen (somewhat) experienced engineers get lost in complex pytest environments that used a lot of mock objects. Over time, they lost clarity on what was being tested, how, and why. The real value is to have an automated set of simple smoke tests. Frankly, Cucumber serves a nonnegotiable reminder to me that the automated tests should be simple and straightforward. For anything extremely complex, I’ll write unit tests instead of trying to craft ridiculous gherkin steps only used for a specific scenario. None of this “when it is midnight of a blood moon in a leap year and the second request attempt has timed out…”

DmitryOlshansky 1y ago

Used cucumber as living specification for our search engine. Sooner or later we realized it was just matching a list of expected JSONs with a bunch of awkward Given, And Then mixed in. Scraped that stuff for our own Kotlin DSL, made the whole thing more flexible and faster to boot.