4 ms·

Great to see you guys here, I've heard a few mentions of your team's e2e integration in the works for a bit! We indeed do approach from a different direction,

Great to see you guys here, I've heard a few mentions of your team's e2e integration in the works for a bit! We indeed do approach from a different direction, having been built around the e2e testing use case from the start, we've focused on integrating with existing CI setups without swapping browsers (some teams really love the bundled Electron runner!), as well as knowing exactly what actions are running in your test runner relative to the replay in a likely familiar UI (answering questions like what DOM element did Cypress match for this command? What was the DOM like exactly when Cypress started running that command? Which command executed before this network request?). Since you're from the team - one thing I've always wondered is how the pricing will work out for CI test runs? From the current public pricing I've found, it looks like it works great for developers recording test runs manually, but is extremely cost prohibitive at any scale if you're running it continually in CI.

acemarke 4y ago
Nice, yeah, I can see the focus on "what actions" being useful. I'm an engineer, not GTM, so I'm not sure how the test runs feature plays into the listed "X recordings per month" at https://www.replay.io/pricing https://www.replay.io/pricing . Agreed that there's a distinct difference between engineers/QA making manual recordings, and CI cranking them out - right now we've got 52 E2E tests that run on every PR, often multiple times due to re-pushes, and each of those tests generates a recording per run. So, obviously that burns through real fast :) If I had to guess we'd probably distinguish between those two use cases. I've tagged our main GTM person in case he wants to respond here.
- mikeshi42 4y ago
  Hahaha yup! We get a ton of volume on our platform from teams churning out test runs per-commit in their CI, where debugability is most important. Glad to hear from others in the space! Hope to learn more if your team's GTM person jumps on :)
jasonlaster11 4y ago
Congrats on launching! Using Session Replay in the CI space makes a lot of sense. And agreed that there’s a lot that can be done by hooking into Cypress’s events. By the way, we’re doing the same and hope to show a Cypress reporter soon as well. Re pricing: we’re still refining the pricing, but we assume that most of the time you’ll only want to debug the failing tests and while the recordings are fairly small, the larger piece is actually replaying the browser as it ran before so you can add print statements in your application and play to when a network request was returned or an error was thrown.
- mikeshi42 4y ago
  Thanks Jason, can't wait to see what your team's been working on the Cypress end soon then!

acemarke 4y ago

Nice, yeah, I can see the focus on "what actions" being useful. I'm an engineer, not GTM, so I'm not sure how the test runs feature plays into the listed "X recordings per month" at https://www.replay.io/pricing https://www.replay.io/pricing . Agreed that there's a distinct difference between engineers/QA making manual recordings, and CI cranking them out - right now we've got 52 E2E tests that run on every PR, often multiple times due to re-pushes, and each of those tests generates a recording per run. So, obviously that burns through real fast :) If I had to guess we'd probably distinguish between those two use cases. I've tagged our main GTM person in case he wants to respond here.

mikeshi42 4y ago
Hahaha yup! We get a ton of volume on our platform from teams churning out test runs per-commit in their CI, where debugability is most important. Glad to hear from others in the space! Hope to learn more if your team's GTM person jumps on :)

jasonlaster11 4y ago

Congrats on launching! Using Session Replay in the CI space makes a lot of sense. And agreed that there’s a lot that can be done by hooking into Cypress’s events. By the way, we’re doing the same and hope to show a Cypress reporter soon as well. Re pricing: we’re still refining the pricing, but we assume that most of the time you’ll only want to debug the failing tests and while the recordings are fairly small, the larger piece is actually replaying the browser as it ran before so you can add print statements in your application and play to when a network request was returned or an error was thrown.

mikeshi42 4y ago
Thanks Jason, can't wait to see what your team's been working on the Cypress end soon then!