Yes, it often is. In a "controlled" environment you control what you control and not the stuff that you can't control or don't even know about. It's tedious and

Yes, it often is. In a "controlled" environment you control what you control and not the stuff that you can't control or don't even know about. It's tedious and sort of a chore to setup and a source of ongoing woes afterwards. On the other hand natural experiments abound when you have large scale. Just having really large N on both arms of an experiment where the difference is the compiled program, and there are no systematic biases in traffic patterns, will reveal smaller regressions that weren't found in the controlled load tests.