4 ms·
> Running a test every minute, or 1440 times a day, will show up quite a lot in logs, metrics, and traces. ...not to mention that automated tests are by defini
by testthetest 1y ago
> Running a test every minute, or 1440 times a day, will show up quite a lot in logs, metrics, and traces.
...not to mention that automated tests are by definition bot traffic, and websites do/should have protections against spam. Cloudflare or AWS WAF tends to filter out some of our AWS DeviceFarm tests, and running automated tests directly from EC2 instances is pretty much guaranteed to be caught by Captcha. Which is not a complaint: this is literally what they were designed to do.
A way to mitigate this issue is to implement "test-only" user agents or tokens to make sure that synthetic requests are distinguishable from real ones, but that means that our code does something in testing that it doesn't do in "real life". (The full Volkswagen effect.)