I agree with most of what your wrote, but "hire an expert" or "you just have to learn" are non-answers to the question that I am seeing a lot in these comments

I agree with most of what your wrote, but "hire an expert" or "you just have to learn" are non-answers to the question that I am seeing a lot in these comments including your: > mastering the discipline of UX... There is no reason to be ambiguous about the practice of good UX design. The main mistake made by most A/B testing is to think that just because you do some testing, it will magically make your UI somehow better. You can make any tests you want, but how do you know they the right tests? And test results do not interpret themselves. There are no obvious conclusions to be drawn from the results of a test. The correct way to do user testing is to start with clear, articulated goals, then select tests to see if the goals are failing or succeeding. Anything else is just back-dating hypotheses - a known anti-science practice. The other misinterpretation of A/B testing is that it will make user interfaces "better" for the end user. Testing can only support or disprove the stated goal and it works just as well for goals that are foolish, short-sighted, self-destructive, and malevolent as it does for those that are ethical and beneficial.