5 ms·
Detecting and Handling Flaky Tests in Playwright
- willhackett 3y agoI really wanted to read this, but I think we may have hugged your site to death.
- lucgagan 3y agoIts back online and just added Cloudflare in front of it
- jawr 3y agoIt seems to be doing a redirect loop
- oefrha 3y agoCurrently in a redirect loop: > curl -D- https://ray.run/blog/detecting-and-handling-flaky-tests-in-playwright HTTP/1.1 200 Connection established HTTP/2 301 date: Mon, 03 Jul 2023 10:43:44 GMT content-length: 0 location: https://ray.run/blog/detecting-and-handling-flaky-tests-in-playwright ...
- lucgagan 3y agoLooks like it is back for some users, but not others. Probably just the time it takes to propagate the DNS. What a bummer! Thank you for helping to troubhelshoot
- edelans 3y agoJust got ERR_TOO_MANY_REDIRECTS
- onion2k 3y agoSomething that makes your end-to-end tests a heck of a lot more resilient is mocking your API so you're not hitting the network, and so your API responses are deterministic. Doing that also makes the tests run a lot faster. MSW is my preferred library for doing that, and there's a Playwright integration (https://www.npmjs.com/package/playwright-msw https://www.npmjs.com/package/playwright-msw). Mocking the API makes it really easy to response with failures too - having a test that checks the app handles a 500 error or a 401 Unauthorized is really useful, and a total pain to do without mocks. You can test the API is working and returning what you'd expect with a different set of tests. :)
- dikei 3y agoIf you mock out one end of the test, can it really be considered end-to-end anymore? Isn't it just UI testing now ?
- easton 3y agoWe had this discussion at work too, and decided it was worth keeping our e2e tests e2e (no mocking, pointing the UI code at our integration environment). However, I think there's value in also having a set of tests that do mock the dependencies while testing UI flows because of how long it takes to run the real e2e tests. so "both" is the dream, with e2e tests being more critical.
- tough 3y agocan't you have both and just run the mocks if not specified else like in a ci or whatever
- RicDan 3y agoYes you can. Our tests are layered as follows: - unit tests - mocking tests - integration tests (i.e. service + db) - system tests (full api to db level) integration starts providing flaky tests already sadly, i.e. with kafka on our end… (i.e. calling the listener twice or thrice even tho it should only once..)
- onion2k 3y agoIt's hard to name them I suppose. They're more than just UI tests because there's usually a decent amount of logic in the frontend apps I build (significantly more than just driving the UI), so I call them end-to-end or integration tests.
- hitchstory 3y agoIt's a hermetic end to end test. The ideal is to mock out the API with some sort of recording mode to autoupdate the mock.
- petercooper 3y agoIt's still not loading for me (redirect loop) but archive.org has it already so I could still read it at https://web.archive.org/web/20230703090352/https://ray.run/blog/detecting-and-handling-flaky-tests-in-playwright https://web.archive.org/web/20230703090352/https://ray.run/b...