It's done randomly. They do a controlled test of a feature by turning the feature "on" for a window of time, and measure users' reactions in the control vs. ex

It's done randomly. They do a controlled test of a feature by turning the feature "on" for a window of time, and measure users' reactions in the control vs. experimental groups in real time. Experimental bias is minimized by the traffic being simultaneous, having a large sample size (visitors), and randomly deciding who gets put in which group.