2 ms·
That's true only if the system in question has already had a lot of optimization. By the time I got to Google about 90% of experiments failed to improve metric
by nostrademons 7y ago
That's true only if the system in question has already had a lot of optimization. By the time I got to Google about 90% of experiments failed to improve metrics. When they started A/B testing nearly every experiment yielded large improvements.
I would bet that education is much more like Google c. 2000 rather than Google c. 2010. A general rule of thumb is that in the absence of extensive training and repeated failures, human intuition is terrible, and that any system based on people's opinions without hard data has a lot of room for optimization.