3 ms·

IMHO, I think everyone here is missing the big point: all these "features" are being pushed to the web because A/B tests say so. The usual way things go is: -

IMHO, I think everyone here is missing the big point: all these "features" are being pushed to the web because A/B tests say so. The usual way things go is: - option 1: don't add feature X - option 2: add feature X - option 3: add feature X slightly modified So, since option 1 is not an option at all (businesses want to grow; they don't want "stable software", they want to push features live every sprint), then lean product managers say "let's do an A/B test and see what our customers like more: either option 1 or option 2!". The A/B test is done and it appears that option 2 increases conversion slightly more than option 1. The team pushes the feature live and everyone call it a day. The next sprint: the same story. So, the net result is that applications and websites get "features" on of top of each other without any order or purpose, but everyone is happy because metrics look good. I know it's very counterintuitive, but that's how things work these days: no one wants to hear your "common sense" opinion, they only want to listen to what the data says; and data says the more ads the more revenue, the more newsletter pop ups the more user emails store in the db, etc. I know this because I have worked for such companies, and they are not precisely going bankrupt.

gnramires 5y ago
A/B testing seems very greedy in practice -- even in the purely economic sense (not in the user satisfaction sense). For example, say you "A/B tested" product pricing naively. I would expect demand to have a momentum component. If you set your prices to a large value abruptly, you will probably keep getting a similar sales rate for a while, until demand starts dropping over long term (could be years for all I know -- enough time for most opinions around information sources to have shifted, for example). Compare this to a strategy like Optimal Pricing[1] based on more refined methods. Overall, I can locate at least 3 large scale problems here: (1) Short term behavior of websites (maximizing ads, poor experience) (2) Poor industrial coordination: surely this practice brings the entire (web) industry down. If most of what you see is plagued by ads and infinite popups, you will seek alternatives. (3) Regulation failure. Sometimes even industrial coordination is insufficient, because joint efforts are still limited to a single domain, and not all of society. In this case usually regulation is necessary. Privacy is great and all, but clicking a cookie disclaimer for every website seems such unnecessary friction. Most inexperienced users I've met did not know what cookies meant and couldn't make the privacy-convenience decision effectively. [1] https://en.wikipedia.org/wiki/Price_elasticity_of_demand#Optimal_pricing https://en.wikipedia.org/wiki/Price_elasticity_of_demand#Opt...
jrochkind1 5y ago
So you're basically telling us they do it to make money, and they're not wrong, it works to make money? I don't think that actually comes as a shock to anyone! My favorite scene in the movie Sorry To Bother You: - See? It's all just a big misunderstanding. - This ain't no fucking misunderstanding, man! So, you making half-human, half-horse fucking things so you can make more money? - Yeah, basically. I just didn't want you to think I was crazy. That I was doing this for no reason. Because this isn't irrational. - Oh. Cool. Alright. Cool. No, I understand. I just I just got to leave now, man. So, please get the fuck out of my way.
- atemerev 5y ago
  Well it isn't the only way to make money. So far, Apple is not (yet) big on all this stuff, yet they make quite a lot of money. Though even they are starting their descent to hellscape.
throwaway47292 5y ago
I have not worked in any company doing ab tests like that, it is always base vs variant. Option 1 vs option 2/3 and then if the feature wins, you can try option 2 vs option 3, if not they would scrap it, or blame it on the implementation and keep retrying until they either get false positive, or survivor bias makes it positive. I am not defending it as I also believe that ab tests ruin the web, but if anyone does option2/3 at start they are not doing AB test at all.
- Karrot_Kream 5y ago
  I agree. I've seen a lot of AB test abuse, but it's mostly not outright, the kind where you try to re-roll until you get a false positive or sample sizes are so small that you get an effect on the treatment group despite the control group being much larger. It's still all statistical garbage though.
atemerev 5y ago
This is why I cringe every time when I hear about "data-centric approaches". Data-centric gives you this.

gnramires 5y ago

A/B testing seems very greedy in practice -- even in the purely economic sense (not in the user satisfaction sense). For example, say you "A/B tested" product pricing naively. I would expect demand to have a momentum component. If you set your prices to a large value abruptly, you will probably keep getting a similar sales rate for a while, until demand starts dropping over long term (could be years for all I know -- enough time for most opinions around information sources to have shifted, for example). Compare this to a strategy like Optimal Pricing[1] based on more refined methods. Overall, I can locate at least 3 large scale problems here: (1) Short term behavior of websites (maximizing ads, poor experience) (2) Poor industrial coordination: surely this practice brings the entire (web) industry down. If most of what you see is plagued by ads and infinite popups, you will seek alternatives. (3) Regulation failure. Sometimes even industrial coordination is insufficient, because joint efforts are still limited to a single domain, and not all of society. In this case usually regulation is necessary. Privacy is great and all, but clicking a cookie disclaimer for every website seems such unnecessary friction. Most inexperienced users I've met did not know what cookies meant and couldn't make the privacy-convenience decision effectively. [1] https://en.wikipedia.org/wiki/Price_elasticity_of_demand#Optimal_pricing https://en.wikipedia.org/wiki/Price_elasticity_of_demand#Opt...

jrochkind1 5y ago

So you're basically telling us they do it to make money, and they're not wrong, it works to make money? I don't think that actually comes as a shock to anyone! My favorite scene in the movie Sorry To Bother You: - See? It's all just a big misunderstanding. - This ain't no fucking misunderstanding, man! So, you making half-human, half-horse fucking things so you can make more money? - Yeah, basically. I just didn't want you to think I was crazy. That I was doing this for no reason. Because this isn't irrational. - Oh. Cool. Alright. Cool. No, I understand. I just I just got to leave now, man. So, please get the fuck out of my way.

atemerev 5y ago
Well it isn't the only way to make money. So far, Apple is not (yet) big on all this stuff, yet they make quite a lot of money. Though even they are starting their descent to hellscape.

throwaway47292 5y ago

I have not worked in any company doing ab tests like that, it is always base vs variant. Option 1 vs option 2/3 and then if the feature wins, you can try option 2 vs option 3, if not they would scrap it, or blame it on the implementation and keep retrying until they either get false positive, or survivor bias makes it positive. I am not defending it as I also believe that ab tests ruin the web, but if anyone does option2/3 at start they are not doing AB test at all.

Karrot_Kream 5y ago
I agree. I've seen a lot of AB test abuse, but it's mostly not outright, the kind where you try to re-roll until you get a false positive or sample sizes are so small that you get an effect on the treatment group despite the control group being much larger. It's still all statistical garbage though.

atemerev 5y ago

This is why I cringe every time when I hear about "data-centric approaches". Data-centric gives you this.