17 ms·
Cypress: Fast, easy, and reliable testing for anything that runs in a browser
- moxylush 6y agoCypress is a great tool for E2E testing!
- jevgeni 6y agoOT: “The Web has evolved. Finally” - proceeds to open several modals like the popups of the 90s.
- cj 6y agoWith the shift of logic from the backend to the frontend (with the rise of clientside frameworks) it feels one thing that simply hasn’t matched the pace of transition is tooling for frontend testing. It’s a tricky problem. We just started using Percy/Cypress at our company.
- arcturus17 6y agoIt all still makes you pull you pull your hair out. I’ve tried Selenium for E2E and RTL and others for integration and unit testing. As soon as you want to do anything remotely complex like testing an auth flow or a chained change of state you start running into issues. Mocking can be truly infernal. I tried to mock Firebase and I found myself thinking way too much about the internals of the Firebase SDK and their relationship with my own application to the point I didn’t know what I was testing anymore. Compare this to testing the back-end where I can crank pages of tests like I’m the Beethoven of testing. I love front-end dev but I cower at the thought of testing anything that’s not a pure data function or a very dumb presentational component.
- mhh__ 6y agohttps://www.cypress.com/ https://www.cypress.com/ (Have we run out of names?)
- maximp 6y agoI've been taking the lead on using Cypress on our small team, and it's been a delight so far.
- presidentender 6y agoThe junior engineers on my team are more enthusiastic about Cypress than any other single technology we work with. It has great ergonomics.
- jnordonez 6y agoAs someone who has recent been looking to learn either selenium or puppeteer, I'm surprised to not heard of this during my research into relevant tools. Would you mind helping me briefly weigh the pros and cons of each? Cheers.
- kidfiji 6y agoTo put it simply, its developer experience is highly-polished
- franklampard 6y agoCheck out testcafe
- lwigo 6y agoIt has the sexiness factor right now, especially in the southeast.
- arcturus17 6y agoTech is hot by clusters in the US?
- mekkkkkk 6y agoInteresting. I immediately thought asia.
- lwigo 6y agoAbsolutely. Angular / React was very much an east coast / west coast thing for awhile, as one example. Cypress was officed at ATDC at Georgia Tech (now downtown), so a lot of companies and startups in metro Atlanta adopted it and helped it take off. Further, any student from GT or SPSU trying to intern through ATDC (and probably ATV) would have been at least tangentially exposed to it, meaning these new grads / young devs got exposed early enough in their careers as to be impressionable, and more likely to adopt something new. It also helps to adopt a technology when you're three doors down from the devs.
- mrslave 6y agoI don't understand the difference between Selenium and Cypress. Is Cypress general control of the browser but just nicer somehow? Or is there a deeper disconnect in their designs/capabilities?
- fizzbuzzah2 6y agoor puppeteer.
- DaiPlusPlus 6y agoPuppeteer is Chromium-specific though.
- jitl 6y agoThat’s okay, Chrome is the only browser.
- Muromec 6y agoBut Playwright isn’t.
- svacko 6y agothere is already experimental Firefox support for a while in puppeteer https://github.com/puppeteer/puppeteer/#q-which-firefox-version-does-puppeteer-use https://github.com/puppeteer/puppeteer/#q-which-firefox-vers...
- mavelikara 6y agoHere is a good comparison published by one other Open Source and free tool in this space, Taiko: https://gauge.org/2019/08/21/how-taiko-compares-to-other-browser-automation-tools/ https://gauge.org/2019/08/21/how-taiko-compares-to-other-bro...
- sicromoft 6y agoHas anyone here used Taiko? The link above makes it sound pretty good.
- pchristensen 6y agoWe have used Cypress for a few projects. It has been far easier to use than Cucumber and Selenium (although it has been a few years since I used those).
- bigyikes 6y agoI’ve had to use Cypress at work, and I really don’t like it. I find the command chaining and assertion queuing to be very unintuitive. (“Do I chain this one or not? Maybe stick it in a callback? No idea!”) It’s been a few months since I’ve touched it, so I’m struggling to give more specific criticisms. I’ve had a really great time using Kent Dodd’s DOM Testing Library [1] for unit tests. The readability and writability of these tests has been fantastic. I wish I could use the exact same API for e2e tests. It looks like they actually do have a Cypress version of the Testing Library [2], but it seems to suffer from the same Cypress-isms that I was just complaining about. Has anyone else had a similar experience? It seems like I’m in the minority opinion. Maybe I’m just bothered because I don’t understand Cypress well. [1]: https://testing-library.com/docs/dom-testing-library/intro https://testing-library.com/docs/dom-testing-library/intro [2]: https://testing-library.com/docs/cypress-testing-library/intro https://testing-library.com/docs/cypress-testing-library/int...
- swsieber 6y agoTo hop on the unhappy train, it's actually quite hard to get an internal mirror mirror working correctly because the platform for which the extra file it downloads is based on a query string, not the path. Argh.
- dpc_pw 6y agoI very dislike Cypress as well. This even model is sooo confusing, clumsy and error-prone that writing any non-trivial test is a challenge. Whole product uses async/await or Promises and suddenly you have to switch to this baroque model just to write a test. Reusing logic between production code and tests is pretty much impossible. Dynamic pages etc. often confuse cypress. I'm actually not a frontend developer, so I don't have a lot of insight while exactly that is, but I just know that it has been a huge drag. Other than that, it would be OKish. Being able to test the UI is great, but I would advice everyone not to base the whole testing strategy on it. It's just too heavy, slow and unreliable. It doesn't not scale even moderately with the product growing. e2e testing is better done on API level, IMO. You'll want redesigns, changes of UX, and suddenly your changes will require updating tests, which otherwise (if done on the API level) would require no modifications.
- foolfoolz 6y agoi’ve spent quite a bit of time in cypres this year. i’ve also invested tons of time into selenium and a lesser amount into puppeteer. cypress is great. the video recordings are very useful. the paradigm of command driven actions is nice to see step by step. but the more time you invest you find the rough edges: xvfb and parallelization, cypress runtime crashes (vastly improved since 5.x), a sometimes top opinionated stdlib with no alternatives (cy.get errors are always fatal), and a somewhat high cost pricing model. i understand they need to run a business but i feel like the project is one custom scheduler away from losing its value prop. the cypress website itself is actually the worst part of the entire experience
- OdeToNode 6y agoWhat would you recommend out of those three?
- foolfoolz 6y agoyou can’t do everything in cypress so we have a mix of it and puppeteer cypress for most of the complex tests. puppeteer for more basic things you don’t need to debug / “should always work”. because the debugging experience is much poorer. but the tests are “free” to execute. i’ve only wasted time in selenium and don’t recommend it for any purpose
- Rapzid 6y agoI've had a lot of success with selenium-webdriver and Jest as the test runner. Use it with Async/Await, write some custom framework around it for stuff like before/after image capture and upload to s3(or wherever), and Bob's your uncle. ffmpeg can also be used to record the xvfb apparently so I may have a play with adding video capture soon. Debugging in VsCode is nice. It's a shame 4.0 has been in beta forever and people keep taking from the project and not giving back, and that async stacks got broke(but I created a proxy that creates a nested stack and preserves the original test code call site), but it works very well in general.
- 6y ago
- sickcodebruh 6y agoMy small team added Cypress to our large React project a few months ago and it’s been great. Took a bit of work for us to feel comfortable with it but it started catching bugs that justified the time investment in its first few weeks.
- amw-zero 6y agoLet’s get to brass tacks... does it still have any possibility of flaky tests? If so, I’m out.
- arusahni 6y agoOne can still write tests that are flaky - some of the challenges that Selenium et al. experience are still present: 1. Selectors still need to be specific enough `.button` is great until you add another button. 2. Selectors can be _too_ specific in weird ways. Recently, I encountered a bug where reformatting my markup broke a test that detected if an element was present by checking for its contained text. Turns out the newlines that I inserted between the opening and closing tags were significant for the test. Thankfully, it's all JS, so one can appropriate work around. 3. For SPAs, it's harder to convey that a page/view load has completed. So, if I click a button and the entire viewport updates (potentially downloading a JS bundle in tandem), I either need to insert timeouts of arbitrary lengths, or explicitly call out to the the provided network API hooks. This can be annoying because now I not only need to worry about what I _can_ see when I navigate through my app, but also what I can't. It's far better than what has come before, but it's not a silver bullet.
- azemetre 6y agoIt depends what is the root cause of your "flakiness." Sometimes it's due to the browser not painting your elements in time because of various layout thrashing other times it could be because of data you're waiting on before you can render a table. If your flakiness is due to items like layout thrashing, or triggering other HTML re-rendering like painting or compositing, IME you'll have issues with any test runner until you fix the problems of poor browser performance and layout thrashing If your tests are flaky due to backend calls being inconsistent or taking a long time, Cypress allows you the ability to intercept network requests [1] and return "fixtures" of what you want the data to be. The benefit of this is you can go directly to any route for your web app then just stub the data you need immediately and test against that. The benefit of this approach if you can easily test negative scenarios and how your application behaves. Negative scenarios like failing response (500 error) or having a 200 response with missing data fields. This is useful when testing something like pagination because you can make the "3rd page" fail when you trigger that API call by forcing it to give you a 500 error (or something else). The downside of this approach is that you are no longer using real services that your app relies on, but you gain some confidence on testing multiple scenarios in an easy format rather than setting up proper chaos engineering scenarios/experiments. Other testing libraries do allow you to intercept requests as well (I know playwright does this), but one of the selling points of cypress is that you have access to the same browser APIs that you have use when doing web development. [1] https://docs.cypress.io/guides/guides/network-requests.html#Testing-Strategies https://docs.cypress.io/guides/guides/network-requests.html#...
- mmmpetrichor 6y agoI've heard about cypress and testcafe both being better alternatives to selenium, but then when I looked into it they both seem pretty similar to selenium. Is there some killer feature making cypress better? Has anyone tried all three and can clarify the differences?
- richeyryan 6y agoBasically the main difference is that Selenium executes outside the browser and controls the browser through the network. This has benefits like you can use any language. The likes of Cypress and TestCafe execute in browser. The main downside to this is you have to use JS or a compile to JS language however the argument is you have less latency and better access to the DOM making tests more quicker and more robust. Arguably JS is a language that is well suited to asynchronous event driven applications like e2e tests
- nhooyr 6y agoI've found playwright more useful: https://github.com/microsoft/playwright https://github.com/microsoft/playwright It's an alternative to Cypress and from what I remember, it has a much better API.
- igotsideas 6y agoI just started using playwright 2 weeks ago. It’s great so far.
- Grizfisher 6y agoYou should try inglor's open source tool called Root cause for simplifying and automating screenshot capture.
- inglor 6y agoWe wrote "cypress for playwright" recently in order to use it internally and decided to open source it. Basically we took a bunch of Testim code for screenshots/console logs/network etc and made a package out of it at https://github.com/testimio/root-cause https://github.com/testimio/root-cause It's amazing how much you can get done just combining open source tools together (like sauce's HAR viewer)!
- intellix 6y agoI've seen a few people say this, but looking at the repository and documentation it seems more like an alternative to puppeteer for browser automation. I don't see anything in particular about testing?
- Kranthi229 6y agoYes we can consider playwright as alternative to puppeteer . But it also got more features involved , I would say it is an upgraded version of puppeteer and it has also got Webkit support.
- thegreatpeter 6y agoCypress is great. We use it at Draftbit. e2e testing is straight forward. We test most of our platform with this (on top of some stronger unit tests). Love it
- ch4s3 6y agoCypress is really nice and ergonomic locally, but I've found it to be miserable and slow to run in CI.
- deleted 6y ago[deleted]
- mikehearn 6y agoFor anyone new to Cypress, there are two things that I consider killer features: 1. It saves a snapshot of the DOM state before/after each test step. If you have a long acceptance test where it's deeply navigating through your app (i.e. visit "/admin", click on "Login", type "username", click "Submit"), using the GUI test runner will show each individual step in a side-panel. From there, you can hover over a step to see a snapshot of what the page looked like before that step was executed, and a snapshot of what it looked like after. I had previously only used test runners where, when it would run the acceptance test, you would see a series of really quick flashes of the runner executing a bunch of steps faster than my eye could track, and then a big FAILED message. What failed? How did it fail? Cypress solves this problem by saving the DOM state and allowing me to traverse backwards through the state of the app until I find where things started to go wrong. 2. The cli test runner records a video of its test runs. Super useful in CI where you don't have access to the frontend. Cypress will record its run to an MP4 and you can investigate failures after-the-fact. This has solved a lot of "it works on my machine but not in CI" problems that seem to come up pretty often. The other positive thing I have to say about Cypress is that I've actually found it more helpful as a development sidekick tool than as a test runner. Particularly when developing features on the front-end that require a lot of user interaction, many times it's easier for me to whip up a quick Cypress test that automates the user interaction (with assertions along the way to make sure the app is working) and then use the GUI Test Runner to do all my debugging. It's not perfect and I definitely have some frustrations with it, but on the whole it's helped my productivity quite a bit.
- byteofmydream 6y agothis 2 'killer features' could be implemented in your tests in a day or two. Also, you can use video recording features implemented in docker images like Zalenium
- mikehearn 6y agoIn all seriousness, if that's true you should do it and productize it. Based on the comments here it seems like many people prefer other browser scripting tools, so if you can replicate the novel features of Cypress in a better package, you could make money. Cypress raised almost $10M in their Series A.
- __abc 6y agoI am so excited for this
- scotty529 6y agoexcited for what?
- JMTQp8lwXL 6y agoI like Cypress because it helps us easily validate when our backend teams introduce breaking API changes without notifying UI teams. Quality is often left to UI teams, when service teams need to step up and perform more due diligence of their own. Until then, Cypress is the tool for the job. Been using it for 6+ months. I'm not crazy about it, but I have no known superior alternatives, so I am grateful for this open source tech.
- sverhagen 6y agoThere are cheaper ways to validate APIs than Cypress and its class of tools. How about something of the likes of Postman (which is scriptable) or (oversimplified) a JavaScript file making calls combined with an assertion library? If your goal isn't to test a frontend, Cypress seems inappropriate.
- JMTQp8lwXL 6y agoFront end teams shouldn't have to write tests for other team's services. This is more of an organizational concern. We do care about testing the front end, but it more often reveals back-end issues than regressions in UI code.
- sverhagen 6y agoIt sounded like these tests were only written to validate the backend. Great if that's not the case. I agree that the backend team should test their own stuff. End to end testing is a poor substitute for that. There may be value, though, in having some (light?) validation of a dependency, ie. the frontend team validating that the backend is still there. I heard the latest SpringOneTour talk refer to this as "enemy testing". Maybe really bad for internal teams, but I can sympathize when the dependency is external and you don't know when to expect changes.
- offtop5 6y agoWill instantly fail if you need to nav to a different domain at any point in your test . Say you have happysite.com , but your using parentsite.com to authenticate, Cypress will explode instantly. If your test will ever leave your first domain, same thing. Don't take shortcuts, use Puppeteer or it's successor Playwright
- andyhmltn 6y agoCan’t say we’ve experienced this at all. Part of our test involves navigating to a third party site (gocardless) to set up a direct debit. The test works fantastically well and is a pretty crucial part of our flow, so we’ve come to rely on it.
- offtop5 6y agoMaybe it's gotten fixed in the last year ? Thank you , I'll keep that in mind the next time I build a front end testing stack
- inglor 6y agoI agree with the sentiment but just want to mention Playwright isn't the "successor" of Puppeteer and both have pros and cons. Puppeteer is very much still in active development.
- taion 6y agoThe thing that strikes me as weird about Cypress is that, while the DX is quite nice, their core claim of being an end-to-end testing experience is, for the most part, not true. You can see that in how most events are simulated instead of native. As such, we're not really simulating user experience, and in fact the accuracy of testing doesn't seem to be meaningfully better than what's available in Karma. DX is fantastic, of course, but I'd feel quite uncomfortable in committing to a commercial offering that I thought was being less than fully honest in what they were offering, just as a matter of trust. It really is a great test runner on its own merits, though.
- bluntfang 6y agoTotally agree with this! cypress seems like a good unit/integration testing tool.
- MetaWhirledPeas 6y agoI'm not sure I agree with your first sentence. Simulated or not, the tests are accurate enough that you're going to be identifying real-world user experience failures the vast majority of the time. It won't be perfect, but then again neither is Selenium despite the difference in approach.
- folex 6y agoWould be awesome to have at least single test + run + output example in the README. At least as a screenshot, link to playground or something. I open website, see marketing, open github, see marketing.
- loh 6y agoHere's an open source alternative (actually a huge improvement to how most people do frontend tests) I put together a while back: https://github.com/testfront-io/testfront-devtools https://github.com/testfront-io/testfront-devtools (See also https://www.testfront.io/ https://www.testfront.io/ for an actual description of how it works, but don't bother signing up.) I'm really bad at marketing/getting the word out, and google took forever to approve the extension so I lost interest. I made it because I hate manually writing E2E tests like with Cypress. Maybe if others show interest in it, I'll be motivated to resume the plans I had for it.
- sverhagen 6y agoLike others in here you included "open source" in your phrasing, in a way that sounds to me as if Cypress wouldn't be open source. But it's rather MIT licensed, unless I've been fooled. Following that up with a question, what's their business model, consulting? I didn't immediately find any upsells on their website. Edit: finally found the pricing page, to but the core is still open source then?
- MetaWhirledPeas 6y agoIt's all open source. What you pay for is their dashboard tool, which is a good, affordable product. The dashboard also acts as an orchestrator for parallelization. I assume the orchestration is free up to the monthly free-limit, but if it falls back to single-threaded after that I don't know.
- mmmateo 6y agoThat looks like a massive productivity gain. Are there any other tools that build tests like that? Makes me feel like a caveman for writing them by hand.
- scotty529 6y agois this only front-end testing or also back-end?
- SulfurHexaFluri 6y agoIts sort of like selenium testing. It tests the frontend which ends up sending real requests to the backend so if the backend fails the frontend will too.
- jitl 6y agoWe used Cypress for a while at Notion but removed it due to high memory usage and crashes. Instead we use a very boring stack of mocha + selenium-webdriver + Chrome on Linux. That’s not my favorite either, because we have to paper over all of the crufty Webdriver/Selenium abstractions that feel like they were designed for Java. I would much prefer to use Puppeteer or Playwright now that they’re starting to stabilize - for a while they both either didn’t support Firefox, or required custom builds of browsers.
- veidr 6y agoWow!! I pay for both Notion (it's good, thanks!) and Cypress, and have literally described Cypress this way: "Cypress is to Selenium as Notion is to Confluence"... :-O So, I am very surprised (but interested) to hear your tale. It is literally the first time I have ever heard of somebody going back (I assume it is back?) to Selenium from Cypress. When we used Selenium (until mid-2018) our tests were hard to write and hard to parallelize, so they took like an hour on a beefy machine. Now they take 5 minutes (across 16 medium-ish VMs, admittedly, but parallelized "for money, not for free" by the Cypress dash board service. I'm curious to hear more about your experience moving off Cypress — did your test suites take longer to run? (Or did you perhaps move off of it before they introduced auto-parallelization?)
- defied 6y agoWe’ve recently released an option to run Cypress tests on browsers in our cloud: https://testingbot.com/blog/testingbot-cypress https://testingbot.com/blog/testingbot-cypress
- twsted 6y agoIt looks promising. I've used most of other front-end testing tool in the past (selenium, puppeteer, playwright) and none of them support drag and drop if I remember correctly.
- aerovistae 6y agoThis isn't a new tool, why is it randomly on the front page of HN? I should start sharing links to random front end tools. Check out reactjs.org! I feel this is the hacker news equivalent of karma farming.
- sumedh 6y agoThey got 25K stars on Github recently, maybe that is why it was submitted.
- InvOfSmallC 6y agoWe are removing it. When running on the CI is so slow! Also, we are moving towards contract based testing.
- beyondcompute 6y agoI would also recommend checking out Testcafe or maybe some other alternatives (I read recently about a project called Playwright which at least “on paper” looks promising). I’ve used Testcafe on couple commercial projects (including one for a large and well known company) and it was a positive experience without any reservations I could remember. Now it still makes me wonder, why do more people seem to prefer a less cross-browser solution that is constrained to one domain(!).
- byteofmydream 6y agoCypress is very bad. What frustrated me most of all is that you can't use node.js, only 'browser javascript'. Also, absence of promises makes Cypress way behind its alternatives. Don't be fool, use TypeScript + Cucumber + puppeteer/ axios, or Java + Selenium/RestAssured, or any other appropriate tool with open source model and adequate code base, maintained by community
- inglor 6y agoI think you can use Node.js (through tasks) in cypress. Agree with everything else though + the one time I opened an issue in their tracker (I asked a question about a crash in order to contribute!) they totally ignored me and then they opened a bunch of issues in projects I maintain and they use (not paying me, which is fair). Guess how much love I gave their issues?
- Supermancho 6y agoI dont know how someone comes to the cobclusion that cypress is behind Cucumber (3x the work) or Java Selenijm (less control and capabilitu), both of which domt use Node either (wtf does that matter in context?). That makes no sense. It's trivial to set these thongs up and test them so I have to wonder where this negative astroturfing is coming from. Obviously we prefer Cypress and it has sufficed or exceeded the capability of other frameworks, in practice, for our SPAs and page-model sites.
- azemetre 6y agoCypress uses bluebird for their promise implementation which are A+ spec. [1] Also (nearly) every command in Cypress is a promise which allows you to make them then-able. [2] I don't agree with using Cucumber for anything. The way most teams implement Cucumber makes it such a pain to write a test. I do like the suggestion of puppeteer tho. If you haven't heard of playwright [3] I'd look into that. From the same creator as puppeteer but was hired to work on playwright full time. They have the advantage of being to test on gecko, chromium, and webkit browsers which is really nice because IME anything Apple becomes such a pain to setup to properly test. [1] https://docs.cypress.io/api/utilities/promise.html#Syntax https://docs.cypress.io/api/utilities/promise.html#Syntax [2] https://docs.cypress.io/api/commands/then.html#Yields https://docs.cypress.io/api/commands/then.html#Yields [3] https://github.com/microsoft/playwright https://github.com/microsoft/playwright
- spondyl 6y agoAt my previous job, we had used Cypress for an internal React application but had to remove it due to incompatibility with our single sign on. Each application either used the same OAuth solution that customers used or they used Okta which was more common for internal applications. It was a little unclear which was supposed to be used as some applications used the former and then it was common to have a "backend for frontend" that would solely deal with communicating between... ah, you know what, I don't even want to think about it. Suffice it to say, the React application in question made use of the okta-react library which meant that the actual SSO dance was part of the Javascript bundle being shipped That in itself isn't a problem but Cypress seems strongly designed around the idea that you never leave the domain. We got that to work but then when 2FA was mandated, we were kind of screwed. We didn't have a TOTP seed to generate a passcode as part of the auth flow which is highly questionable of course but we also didn't have a solution to bypass the actual authentication step either, as it wasn't an API call to a backend (we controlled) Anyway, we tried making a service account that has 2FA disabled only to discover that 2FA was mandated at a network level (unless you were on a specific subnet we later discovered) meaning we would just see some arbitrary "error 53" Anyway, This isn't to say that Cypress is anyway at fault here but I have seered in my memory, from the tens of hours spend on this crap, the sight of threads where people would "How do I use this with Okta" and the developers essentially saying "Oh, yeah, you shouldn't be doing that". Having said all that, while I remember finding Cypress pretty handy, I'm more fond of Testcafe myself :) Ah also, I believe a lot of teams made use of a Zalenium... cluster? Bunch of instances anyway. I also recall a number of people having the above issue with no solutions
- miohtama 6y agoWe had the same issue regarding external domains. It has been somewhat addressed in the current Cypress releases.
- romanovcode 6y ago> That in itself isn't a problem but Cypress seems strongly designed around the idea that you never leave the domain. Same here with PayPal or 3d-secure. I am actually baffled by the decision - what kind of applications did they thought was going to get tested - Todo apps? Last I checked there was GH issue and they said that it will get addressed, I hope. What they are also saying is that it is bad practice, you should send API requests from the tests that simulate the payment, for example. I find this BS, IMO the automation tests simulate user and user would not send API requests but click buttons on the screen.
- timdaub 6y agoI'm quite surprised to see the overwhelming negativity towards cypress here. I recently started using it in a small project and I'm really enjoying myself. I mainly use cypress for high level UI testing and I mock my sever. I found the server mock tool (route and route2) brilliant. I find the syntax for browser commands easy (visit, click, get). Using mocha with should and chai is fine. Anyways, apart from some minor problems (you can't select a single test to run), I really love cypress. Compared to last time I used selenium, it's such a difference. Especially that cypress just seems to be one holistic package that works. Can't say the same for selenium.
- kinow 6y agoI'm using it for a Vue project at work, and it's really easy to use. Had to get used to chaining the commands, and working around a few issues like when I needed to access something JS object (simply exposed the service object I needed globally when Cypress tests were executed, and then verified what I needed). Used Selenium 1 and 2 in the past, and Cypress is definitely much easier. Can't say if the current Selenium is better though. But being able to reply the DOM states, and record videos, without needing to install other tools/libs is also very convenient.
- gtsop 6y agoI can see what people find annoying about cypress but to me, it's "cleverness" and tight browser integression provides such a much better experience than anything else I've used. It is not perfect but I think they have the correct idea of how web e2e tests should work and I hope they will keep improving the experience as time goes on.
- paulintrognon 6y ago> (you can't select a single test to run) Yes you can, by using it.only(): https://docs.cypress.io/guides/core-concepts/writing-and-organizing-tests.html#Excluding-and-Including-Tests https://docs.cypress.io/guides/core-concepts/writing-and-org...
- timdaub 6y agoAh thanks! That's great. What I meant is that you can't select it via text search in the shell. But `.only` should work for me fine too!
- inglor 6y agoI wrote an article about Cypress vs. other open source alternatives (like puppeteer/playwright) in case anyone is interested. The tl.dr is that I didn't end up using Cypress https://www.testim.io/blog/puppeteer-selenium-playwright-cypress-how-to-choose/ https://www.testim.io/blog/puppeteer-selenium-playwright-cyp... but it mentions lots of pros of cypress vs. the others.
- sagitariusrex 6y agoI think anyone who ever had to set up a cypress-like pipeline in a CI environment by hand really appreciates cypress.
- mxschmitt 6y agoI really like Playwright, there is less magic and it simply works: https://github.com/microsoft/playwright https://github.com/microsoft/playwright Especially with this test runner: https://github.com/microsoft/playwright-test https://github.com/microsoft/playwright-test
- Grizfisher 6y agoYou might like the root cause open source which runs with playwright to capture test results automatically including screenshots, logs, stacktrace in a nice ui. https://github.com/testimio/root-cause https://github.com/testimio/root-cause
- deleted 6y ago[deleted]
- vpfaulkner 6y agoI’ve found Cypress generally better than other alternatives such as Selenium. The screenshot feature is nice and it seems to be a little less flakey, although E2E tests using a browser are always going to be more prone to flakiness than unit tests. This last project we’ve actually been running true E2E tests by spinning up the backend in GH actions (looking for a branch with the same name or falling back to master), seeding it, and making snapshot-style assertions about the UI. It’s more complex and slower but tests a lot more code now that the backend is involved as well as the API layer. It has helped a lot in major refactors.
- bahmutov 6y agoWow, someone has submitted Cypress.io here - probably because we just crossed 25k stars on GitHub. We at Cypress do read these comments, and we definitely understand both the positive and negative opinions. If you have never tried Cypress yourself - take a look at the first test page https://on.cypress.io/writing-your-first-test https://on.cypress.io/writing-your-first-test Try it yourself, you might like the syntax and the developer experience. And if not - no worries, there are TestCafe, Puppeteer, Playwright, Selenium, Webdriver that might work better for you.
- frafra 6y agoI just wanted to suggest https://github.com/seleniumbase/SeleniumBase https://github.com/seleniumbase/SeleniumBase if you are comfortable with using Python and Selenium. I opened various issues last year and the main developer is very welcoming and he fixes/improves the code very quickly.
- djlewald 6y agoLike a couple other people I'm very surprised by the negativity around this tool. I was introduced to it when I started my current position a few months ago and I've found it everything BUT hard to use. The method chaining feels good as long as you check the return type of the function your calling. Most of the time it returns the element you just asserted against. And when you're trying to drill down into deeper sections of the DOM you can speed up tests by calling .within from a higher level element to keep the search to a specific part of the tree. The individual steps with their state available for viewing in the sidebar is great as others have mentioned. The test doesn't run at light speed, so locally you can actually VIEW what is happening. Plus when it does fail there's typically a long timeout (~40 seconds) before the test actually fails. So it's very noticeable when your test hits a snag. Also helper functions are not hard at all, and we have a few useful ones we use in almost every section. I have more I could say, but I'm not going to write a huge raving post. Just wanted to try to post some positive contrarian perspective.
- bluntfang 6y agoCypress not following the Web Driver spec is a huge turn off. https://www.w3.org/TR/webdriver/ https://www.w3.org/TR/webdriver/
- hugocbp 6y agoWe've been using Cypress at work and we (well, at least I) have a love-hate relationship with it. Coming from Capybara in Rails, it is a very noticeable improvement. Unlike others here, we never really had issues with the chaining. We don't mess too much with custom commands. We just have a library of about 30-40 small helper functions, mostly to do simple steps to test UI elements from Material UI, React Select, etc. So, for example, if we need to select something with React Select, it can get tricky. It is a multi-step process with Cypress to ensure the thing is actually rendered. But after we do that once, it is only a matter of making a `selectDropdownOption(tag: string, option: string)` helper function. We have dozens of tests that are just 3 or 4 lines of those helper functions. One of the major drawbacks we have with Cypress, though, is about the data. Since Cypress just uses your `localhost`, it uses whatever DB you are connected. I really miss the Capybara/Rails integration of just spinning up a proper test DB, seeding it and then destroying it after the tests were done. I bet there are ways to do this with Cypress, but it is just not as straightforward as it was in Rails.
- acemarke 6y agoNot sure what you mean by "just uses your localhost". I recently set up Cypress for the app I'm working on, which is a legacy MEAN app that I've been modernizing (migrating to React+TS, etc). It already had a set of Express server API "unit tests" that were really more like integration tests, using `supertest` to load the Express app and make calls to the real API endpoints. I updated it a few months back to use `mongodb-memory-server` to spin up a dynamic `mongod` instance solely for the test sequence. When I set up Cypress, I was also able to use `mongodb-memory-server`. I configured the app to check for an environment variable indicating an E2E test environment, had it connect to the in-memory `mongod` instance, and used a Cypress task to seed the DB before every test. Seemed fairly straightforward, and is working pretty well.
- intellix 6y agoUsing Cypress in NX Angular CLI after having used Protractor previously. In comparison to Protractor it's an absolute god send and so much easier. The documentation for Cypress is pretty good IMO. I love that when you use should('be.visible') etc it'll continuously retry until the timeout as it makes your tests so much less flakey. There's a couple of annoyances still: - I hate that some things are wrapped in JQueryLike and then other times they're not. - I still hate that there's async/await magic and wish we could stop it for e2e testing and just be explicit so I know what is or isn't going to be magic.
- nojvek 6y agoComing from a world of using puppeteer/playwright for testing in an actual browser with full access to dom/Network/console/perf stats etc, I don’t get cypress. What is the value add over puppeteer ?
- Macha 6y agoIt operates at a higher level than puppeteer/playwright, and (at least in my personal and professional experience - others in the thread disagree) more reliably than selenium or libraries which wrap it.
- Kranthi229 6y agoI've used Cypress at work , it was really good experience in creation of test-scripts. Had a few hurdles in the start but achieved the tasks with the help of documentations available. Now we have configured E2e and code-coverage with it .
- PaywallBuster 6y agoCypress: Except fast and reliable
- zaszrespawned 6y agocypress does not support popups. its a problem if you wish to test payment gateway or social login that opens in a new popup for login. so we could not use cypress..
- robbywashere_ 6y agoAt my job I inherited a codebase which was completely both untested and undocumented aside from a general consensus from the backend guys on how it should behave. Cypress was instrumental in black boxing the behavior of the ui and then testing our refactors and changes against it. It has its worts. The chaining is not intuitive or consistent, the documentation never indicates fully return types or any types for that matter. Also you can only mock xhr requests and not the more modern fetch which is a bit hokey. You have to roll your own. But it’s free and offers a heck of a lot more in stability and features than what’s out there.
- 7357 6y agoI tried it in all honesty, even though I found the attitude of the founder not so nice, but it is acceptable for very bright people, which I am not. Unfortunately, I hit a bump very soon as it could not deal with the way my sut was doing login (with azure) and as far as I could see on SO for example, people were relying upon puppeteer scripts to work around that. cypress looks super smart and all, but it did not do it for me. I am a 20+ years test automater and I am having lots of fun using taiko.
- samtuke 6y agoWe migrated from Cypress to Gauge/Taiko for fuzzier (more human readable and guessable) selectors.