4 ms·

Totally agree I wonder if they ever had a control group that they showed things in random just to measure how good their tech is.

• marcinzm 8y ago
  Possibly, I used to work for a smaller ad company and one of the tests we did was random ranking of ads. Problem is that it can be a very expensive test to run so you wouldn't run it on anything except a tiny fraction of traffic.
• nostrademons 8y ago
  Dunno about Facebook/Amazon/Twitter, but Google uses a control group on every single experiment. They also usually have holdbacks (control groups over time, where the a small user population does not receive a new feature to measure the performance as people get accustomed to it) and ablation experiments (where a previously-launched feature is turned off for a small population of users to verify the expected loss in performance). Advertising has always been a business of extremely large numbers. The chance that a given user will purchase a product because of an ad is imperceptible; aggregated over hundreds of millions of ads shown, imperceptible changes add up to large increases in sales. People tend to forget just how bad ads were before Google & Facebook; it's not that the latter have solved advertising, it's that TV, print, and billboards were so terrible that it didn't take much to improve markedly over them.