You may not always have the ability to properly make those measurements... it takes me a week before I can gather enough data to even weakly show the effects of

You may not always have the ability to properly make those measurements... it takes me a week before I can gather enough data to even weakly show the effects of a change, and I run a top 100,000 site. Also it is much more difficult to measure long term impact. For example, I made a change a couple years ago that quadrupled conversion rates, but irreversibly dampened growth. By the time it was evident that it was -that- change, it was too late to recover. This stuff is easy to talk about in theory, but real life ain't ceteris paribus.