3 ms·
Why would they have to change it for an entire year? Maybe it's enough to test it for 1 day, or 10 minutes for all I know. If it improved clicking rate and reve
by nandemo 17y ago
Why would they have to change it for an entire year? Maybe it's enough to test it for 1 day, or 10 minutes for all I know. If it improved clicking rate and revenue by 9%(+-1%) and after 1 year they've got $1 billion of revenue, then you could say that 8~10% of that billion likely came from Magic Blue.
- mtarnovan 17y agoYou want to sample for 10 minutes or 1 day and extrapolate for 1 year ? The results would be meaningless. Even if you sample 1 continuous month you could have badly flawed results (for example you might have sampled only early adopters, which might behave differently; and that's just one example of the many, many factors that can make signal indistinguishable from noise). Now I'm sure that the guys working on Bing A/B testing know what they are doing, but I just can't believe a change in link color would bring a "$80-$100 million dollars a year increase" (from a total of ~2000 million $/year revenue which I extrapolate from this http://www.microsoft.com/msft/earnings/fy10/earn_rel_q2_10.mspx#income http://www.microsoft.com/msft/earnings/fy10/earn_rel_q2_10.m... but which includes all revenue of the Microsoft Online Service Business - I don't know how much of that is generated by Bing)