> Research of A/B experiments carried out independently by multiple companies suggests that, at best, 1 in 3 ideas will have any measurable impact, and the ind

> Research of A/B experiments carried out independently by multiple companies suggests that, at best, 1 in 3 ideas will have any measurable impact, and the industry average is likely 1 in 7 (14%). Everything else either has no impact or creates negative impact Some fairly big leaps here: - There's a big gap between "measurable impact" and "no impact", which is mostly dependent on measurement accuracy. For example, driving my car to work has no measurable impact on the air quality of earth as a whole, but that doesn't mean it has no impact. - The link for "1 in 3" doesn't even seem to mention that, as far as I can find. The closest is "At Google and Bing, only about 10% to 20% of experiments generate positive results". This is pretty skewed by the fact that those companies intentionally make minor or unintuitive changes to A/B test them, but can't really be generalised to all feature work at other companies. The link for "1 in 7" is also specifically for A/B testing, so generalising that to all "ideas" is quite a stretch.