That's a pretty good question. Ideally you're measuring "paid us money" as conversion, and that was the case for the anecdote I had in mind in the above post:

That's a pretty good question. Ideally you're measuring "paid us money" as conversion, and that was the case for the anecdote I had in mind in the above post: we had a checkout screen with a decent price tag at the end of the flow that we could measure against. We could track the impact of changes throughout that funnel on the bottom line. But I have seen other decisions made based on proxies to "paid us money." Eg we find that X% people who perform action Y wind up paying us money, and for really long funnels it's easier to get statistical significance for action Y than for "paid us money." And so we make some change, find that it increases the rate at which people do action Y, and declare victory. We try to keep in mind potential downstream impacts (like loss of trust), but it's hard to argue a vague, potential downside against a solid, measured number sitting in a google doc in front of you. I suspect the phenomenon you call out is somewhat common.