3 ms·
I think you would be hard pushed to find any big tech company which doesn't do some kind of A B testing. It's pretty much required if you want to build a gr
by londons_explore 7mo ago
I think you would be hard pushed to find any big tech company which doesn't do some kind of A B testing. It's pretty much required if you want to build a great product.
- embedding-shape 7mo agoYeah, that's why we didn't have anything anyone could possibly consider as a "great product" until A/B testing existed as a methodology. Or, you could, you know, try to understand your users without experimenting on them, like countless of others have managed to do before, and still shipped "great products".
- ryandrake 7mo agoI know this is a salty take, but reliance on A/B testing to design products is indicative of product deciders who don't know what they are doing and don't know what their product should be. It's like a chef saying, I want to make a pancake, but trying 50 different combinations of ingredients until one of them ends up being a pancake. If you have to test whether a product works / is good / is profitable, then you didn't know what you were doing in the first place. Using A/B tests to safely deploy and test bug fixes and change requests? Totally different story.
- coldtea 7mo agoA/B testing is the child of profit maximization, engagement farming, and enshittification. Not of "great product building".
- wavefunction 7mo agoA responsible company develops an informed user group they can test new changes with and receive direct feedback they can take action on.
- londons_explore 7mo agoA big tech company has ~10k experiments running at once. Some engineers will be kicking off a few experiments every day. Some will be minor things like font sizes or wording of buttons, whilst others will be entirely new features or changes in rules. Focus groups have their place, but cannot collect nearly the same scale of information.
- rkomorn 7mo agoI think a lot of people (myself included) would just like to not be constantly part of some sort of revenue optimization effort. I don't care, at all, about the "scale of information" for the company's sake.
- londons_explore 7mo agoOften the experiments are not for revenue - many of them will be optimizing user experience metrics - ie. Load time or user dropoff rate. They are clearly good for both user satisfaction and the companies bottom line.
- jjj123 7mo agoAs someone who works in these orgs, only a small fraction are about user experience metrics. 90+% are extracting more short term value with unknown second order effects on usability.
- franktankbank 7mo agoIf you have 10k experiments running then you are probably p-hacking.
- wavefunction 7mo agoBig tech companies are not serving their "users" but advertisers, it's a common mistake.