I agree that this is a problem but I wouldn't say it is a big problem in comparison to the problem that most companies have - they don't run any tests at all.

I agree that this is a problem but I wouldn't say it is a big problem in comparison to the problem that most companies have - they don't run any tests at all. I actively need to prevent myself from going too deep into the maths and caveats of A/B testing with clients. Running tests with bad maths trumps running no tests at all in my experience