4 ms·
Dunno about Facebook/Amazon/Twitter, but Google uses a control group on every single experiment. They also usually have holdbacks (control groups over time, wh
by nostrademons 8y ago
Dunno about Facebook/Amazon/Twitter, but Google uses a control group on every single experiment. They also usually have holdbacks (control groups over time, where the a small user population does not receive a new feature to measure the performance as people get accustomed to it) and ablation experiments (where a previously-launched feature is turned off for a small population of users to verify the expected loss in performance).
Advertising has always been a business of extremely large numbers. The chance that a given user will purchase a product because of an ad is imperceptible; aggregated over hundreds of millions of ads shown, imperceptible changes add up to large increases in sales. People tend to forget just how bad ads were before Google & Facebook; it's not that the latter have solved advertising, it's that TV, print, and billboards were so terrible that it didn't take much to improve markedly over them.