16 ms·

I've talked about this on HN but I'll say it again. A couple years ago, after being an Android fan for the better part of a decade, I finally bought myself an

by hbn 4y ago

I've talked about this on HN but I'll say it again. A couple years ago, after being an Android fan for the better part of a decade, I finally bought myself an iPhone and pried myself away from Google's ecosystem wherever I could. And Apple didn't even need to do any work for me to make this decision. It was the years of abuse from Google that you experience when you decide to use a Google product or service. And a big part of that was the constant A/B/C/D/E/F testing. I never felt like I was using a complete product, everything felt like a constant beta that could be changed or rearranged at any point, and I was just doing free testing work for them while they harvest all my data. Every app update was a risk of the app rearranging itself, or features appearing/disappearing. Eventually it didn't even come from app updates in the Play Store, and new interfaces would just appear one day when a server somewhere marked your account as being in the group that gets the new UI. This app that you were familiar with could at any point be rearranged when you open it on any given day. Then maybe a week later you open it and it's back to how it was before. A button you thought was here suddenly isn't, and you question whether something actually changed or if you're losing your mind. It's a subtle gaslighting that eventually I couldn't stand any more. To me, A/B testing means you don't respect your users. You see them as just one factor in your money machine that can be poked and prodded to optimize how much money you can squeeze out of them. That's not to say a company like Apple is creating products out of the goodness of their heart, but at least it feels like it was developed by humans who made an opinionated call as to what they thought was the right design decision, and what they would want to use. And in my 2 years of owning an iPhone, I've never opened my reminders app to find out that it's completely unrecognizable, or my messages app has been renamed or rethemed for the umpteenth time.

aceazzameen 4y ago
You nailed it. Google is constantly forcing users to relearn most of their products year after year. Give me Google products from a decade ago and I'd still be happy. Now I'm moving on from Google also. It's an untrustworthy brand.
costcofries 4y ago
"To me, A/B testing means you don't respect your users. You see them as just one factor in your money machine that can be poked and prodded to optimize how much money you can squeeze out of them. " Your perspective is extremely short-sighted. A/B testing can result in this type of behaviour but that's just poor A/B testing. Good A/B testing focuses on removing distractions from the experience and helping users derive more value from the product. Bad A/B testing tries to make things more discoverable, where discoverability is often just noise and distractions. Good A/B testing ensures that the money machine, as you put it, pays its dues to users by making the product experience delightful.
- loriverkutya 4y ago
  I never seen an A/B test where the goal was not to maximise profit for the company.
  - dotecoffee 4y ago
    Have you ever seen "you've already bought this" on Amazon? Highly tested, reduced sales. Overruled by judgement. Stayed on the site. AB test results don't mean you leave your judgement behind.
    - iudqnolq 4y ago
      Maybe. I find that very useful when I'm trying to buy the same thing again. It acts as a trust marker. I do wish they kept better historic purchase data. I assume it's deleted absurdly quickly for compliance?
      - account42 4y ago
        Huh, Amazon keeps years of order history for me and you can't even remove it, just "hide" orders which will then still be available just with an extra step. And I'm in the EU so no idea where else they would have more compliance problems with that.
  - costcofries 4y ago
ChrisMarshallNY 4y ago
> means you don't respect your users. I was just pontif- er, talking about this to someone, a couple of days ago. I love the users of my products. Most of my products are free, and are carefully-crafted, highly-polished, complete deliverables, and I fret over how they are used -even if by a tiny number of end users-, like a nervous hen. I do what I do, out of love for the craft, and out of a genuine desire to make people's lives easier, through the technology I have at my disposal. It is my belief that most tech companies despise their user base. Users are little more than cattle, to be fattened and slaughtered. "Caring about the user" means optimizing for "engagement," or keeping them trapped within their own ecosystem. John Oliver did a rant about this, recently[0]. It has nothing to do with actually caring about the user, or solving their problem. It is about harvesting users. In fact, my discussion about this, came about, because someone wanted to keep users inside the app I'm writing, as opposed to linking them to a more familiar app, on their phone (for the record, it was for videoconferencing). Linking is a "no-brainer," as I can link out to dozens of installed apps, using the simple URL scheme method, built into iOS[1], and "keeping them in the app," would have required several months of extra work, polluting the app with megabytes of junk code, because I'd need to use SDKs, and also kill the ability to easily scale to add new clients (contrary to popular belief, Zoom is not the only videoconferencing option). It would also have possibly put us on the hook, legally, for what happened in those videoconferences. [0] https://youtu.be/jXf04bhcjbg?t=638 https://youtu.be/jXf04bhcjbg?t=638 [1] https://developer.apple.com/documentation/xcode/defining-a-custom-url-scheme-for-your-app https://developer.apple.com/documentation/xcode/defining-a-c...
mgoetzke 4y ago
Recently they remove YouTube PiP on iOS. Then it came back to my device but not others in the family. We pay for YT Premium. This is beyond infurating
- deleted 4y ago
  [deleted]
bratbag 4y ago
Depends on where A/B testing is used. If it's something one-and-done (like different permutations of a signup flow to see what is easier for users), then I don't see the harm in it.
corrral 4y ago
> Every app update was a risk of the app rearranging itself, or features appearing/disappearing. This shit drives my parents insane. Me too, when I have to help them. I've had to spend tens of seconds looking at a major screen in the phone app, of all things, to figure out WTF I'm looking at so I could help them figure out what was up. Re-arranged every update (or new phone) for absolutely no reason, terrible affordances, poor use of their own design language. Ugh. I'd get them on iOS but they need larger screens and the $400 small iPhones (what I have) are already more expensive than they think a phone "should" cost, so they keep buying $200 Android phones about once a year (hoping the next one will be better) and not being able to use them because the UI is garbage.
- bombcar 4y ago
  It's doubly sad because the larger phone or iPad would be perfect, but every year it's another $200 to tell Google they're doing the right thing. At least on iPad/iPhone you can set the apps to access Google mail, etc, which doesn't change as often, but still too often.
- selfhoster11 4y ago
  At one point I was trying to set up my grandma with a popular video calling app on a dedicated device so we could stay in touch. Before I could give her the freshly grandparent-proofed device, said video calling app upgraded on my parents' PC first and changed literally every single element of the UI beyond recognition. To someone the age of my grandma, that would be literally like bricking the device remotely, because none of the buttons would look the same, and she would not be able to work out how to use the new interface. STOP CHANGING THINGS! Even if the new UI is better (debatable), some people just like or rely on a particular layout to operate the device or app. Don't rearrange without giving a ~permanent setting to use the old layout.
  - corrral 4y ago
    > STOP CHANGING THINGS! Even if the new UI is better (debatable), some people just like or rely on a particular layout to operate the device or app. Don't rearrange without giving a ~permanent setting to use the old layout. The fact that we're 15 years into smartphones being popular and that phones & computerized address books were basically fully-solved interfaces long before those took off, and they're still fucking around with phone app UI in big ways, is a sign of some kind of institutional or industry-market failure to me. Or both. Allowed, I suppose, because Google's market position (i.e. monopoly across several markets) both gives them the surplus profit (i.e. rents) to fuck around and waste money on harmful to users crap like phone app redesigns, and insulates them from any actual threat to their profit due to those bad choices.
amelius 4y ago
Yes, I hate Apple, but I'm starting to hate Google even more. One of these days, I might switch too. Why can't we have nice things?
quickthrower2 4y ago
I moved back to iPhone in the 2015s onwards (I think) because of the crap quality of devices available, and the bloatware atop if android. In many cases the hardware was so poor it was hard to make a call due to the touchscreen. Since the primary thing I want the phone to do is make a call I switched to the “it just works” camp and haven’t regretted it. Except getting photos off the phone. Until I realised the best tool for that is … Ubuntu!
Lutger 4y ago
> To me, capitalism (A/B testing) means you don't respect your citizens (users). You see them as just one factor in your money machine that can be poked and prodded to optimize how much money you can squeeze out of them. You just described doing business in todays world. Being a bit more generous towards A/B testing, I would make a counter point: _not_ doing any kind of user testing, of which large scale automated A/B tests are just a subset, means you don't respect your users. Because it means you just assume you know what their experience is like, or worse: you don't even care about it and bother to learn something. Your complaint seems to be more about the scale and aspect of automation honestly, and continuity of the services, which is a valid complaint against Google but not about A/B testing in general.
- saagarjha 4y ago
  > Because it means you just assume you know what their experience is like, or worse: you don't even care about it and bother to learn something. A/B tests are not the only, or even the best, way of collecting user feedback.
bentcorner 4y ago
Google recently shut off Hangouts on me with a flight (since several of my contacts reported that Hangouts worked fine for them). It's kind of mind boggling they'd decide to do that - the replacement they direct me to (Google Chat) doesn't even have feature parity so I just dropped them and moved my social circle using Hangouts to a different app (since at this point they all faced the same problem and we decided on a different platform). I'm really curious how the A/B testing for this went down - Google is willingly throwing customers away because somebody wants to pump numbers for a new app that is objectively worse than the old one. At this point Google Maps is the only product that is keeping me with them, but even that one is beginning to wear thin.
- jklinger410 4y ago
  > Google Chat) doesn't even have feature parity Which features are you missing?
  - cupofpython 4y ago
    check the AB test results
  - bentcorner 4y ago
    Video chat. It redirects me to Google Meet (yet another app!) and the model that Google Meet uses is completely backwards (create a meeting, then invite people) compared to how I use Hangouts (call someone). It's a complete pita and the family-acceptable-factor is low.
adrr 4y ago
We AB tested a performance enhancement to our frontend web app to show that speed had significant benefits to the business. we use the results justify the investment cost. We spent the next six months working on making the site faster because of it. It is a tool. How would you measure things without ab testing?
- fisf 4y ago
  The fact that you need an A/B test to demonstrate that frontend performance has an impact (on user experience first and foremost) in 2022 speaks volumes.
  - jklinger410 4y ago
    Yet it says nothing. Large companies are slow and stubborn.
  - cupofpython 4y ago
    this is the stance I take on AB tests. The are objectively good at things you shouldnt need objective evidence for
  - ajmurmann 4y ago
    But how big is the impact and does that return compare to that of other investments you aren't making because you are improving performance instead?
dakial1 4y ago
The problem is not the AB testing, as it can be a good thing to improve the experience, the problem is poorly set, short sighted, OKRs. The author points that in the text, as he mentions many times that the leadership never asks the right questions, mainly how it will impact the client in the long run (being NPS evolution, lifetime value, etc..)
brynjolf 4y ago
Ironically the place I'm getting forced A/B testing now is the Playstore. They moved the update apps section so it is now 2-3 clicks more, making me have to make a seperate shortcut just to reach the app update quickly. The point is of course to make it annoying to manually update apps and enable auto update. I have been burned too many times with a auto update so I refuse. This wasn't enough, they really want to force me enable auto updates to the point of the update section of the app having 50% of the visible space on my big screen being covered with a message to enable auto update over WiFi. [0] Whoever is doing this at Google... Stop. Just stop. It is cringe. [0] https://i.imgur.com/MbXn9gy.jpg https://i.imgur.com/MbXn9gy.jpg
fouric 4y ago
A few months ago, I came to the belief that Google is the ADHD toddler of user-facing software development - absolutely unable to sit still and concentrate on anything, hence the constant UI/UX churn, half-baked products, and graveyard[1] of shiny things that they worked on for a few years before abandoning. Google seems to be really good at making developer tools like Borg and Blaze - however, I think that as an organization they have some deficit that makes them not responsible enough to develop user-facing software (like, uh, an operating system). Maybe Google would be better as a B2B company. [1] https://killedbygoogle.com/ https://killedbygoogle.com/

by hbn 4y ago

aceazzameen 4y ago

You nailed it. Google is constantly forcing users to relearn most of their products year after year. Give me Google products from a decade ago and I'd still be happy. Now I'm moving on from Google also. It's an untrustworthy brand.

costcofries 4y ago

"To me, A/B testing means you don't respect your users. You see them as just one factor in your money machine that can be poked and prodded to optimize how much money you can squeeze out of them. " Your perspective is extremely short-sighted. A/B testing can result in this type of behaviour but that's just poor A/B testing. Good A/B testing focuses on removing distractions from the experience and helping users derive more value from the product. Bad A/B testing tries to make things more discoverable, where discoverability is often just noise and distractions. Good A/B testing ensures that the money machine, as you put it, pays its dues to users by making the product experience delightful.

loriverkutya 4y ago
I never seen an A/B test where the goal was not to maximise profit for the company.
- dotecoffee 4y ago
  Have you ever seen "you've already bought this" on Amazon? Highly tested, reduced sales. Overruled by judgement. Stayed on the site. AB test results don't mean you leave your judgement behind.
  - iudqnolq 4y ago
    Maybe. I find that very useful when I'm trying to buy the same thing again. It acts as a trust marker. I do wish they kept better historic purchase data. I assume it's deleted absurdly quickly for compliance?
    - account42 4y ago
      Huh, Amazon keeps years of order history for me and you can't even remove it, just "hide" orders which will then still be available just with an extra step. And I'm in the EU so no idea where else they would have more compliance problems with that.
- costcofries 4y ago

ChrisMarshallNY 4y ago

> means you don't respect your users. I was just pontif- er, talking about this to someone, a couple of days ago. I love the users of my products. Most of my products are free, and are carefully-crafted, highly-polished, complete deliverables, and I fret over how they are used -even if by a tiny number of end users-, like a nervous hen. I do what I do, out of love for the craft, and out of a genuine desire to make people's lives easier, through the technology I have at my disposal. It is my belief that most tech companies despise their user base. Users are little more than cattle, to be fattened and slaughtered. "Caring about the user" means optimizing for "engagement," or keeping them trapped within their own ecosystem. John Oliver did a rant about this, recently[0]. It has nothing to do with actually caring about the user, or solving their problem. It is about harvesting users. In fact, my discussion about this, came about, because someone wanted to keep users inside the app I'm writing, as opposed to linking them to a more familiar app, on their phone (for the record, it was for videoconferencing). Linking is a "no-brainer," as I can link out to dozens of installed apps, using the simple URL scheme method, built into iOS[1], and "keeping them in the app," would have required several months of extra work, polluting the app with megabytes of junk code, because I'd need to use SDKs, and also kill the ability to easily scale to add new clients (contrary to popular belief, Zoom is not the only videoconferencing option). It would also have possibly put us on the hook, legally, for what happened in those videoconferences. [0] https://youtu.be/jXf04bhcjbg?t=638 https://youtu.be/jXf04bhcjbg?t=638 [1] https://developer.apple.com/documentation/xcode/defining-a-custom-url-scheme-for-your-app https://developer.apple.com/documentation/xcode/defining-a-c...

mgoetzke 4y ago

Recently they remove YouTube PiP on iOS. Then it came back to my device but not others in the family. We pay for YT Premium. This is beyond infurating

deleted 4y ago
[deleted]

bratbag 4y ago

Depends on where A/B testing is used. If it's something one-and-done (like different permutations of a signup flow to see what is easier for users), then I don't see the harm in it.

corrral 4y ago

> Every app update was a risk of the app rearranging itself, or features appearing/disappearing. This shit drives my parents insane. Me too, when I have to help them. I've had to spend tens of seconds looking at a major screen in the phone app, of all things, to figure out WTF I'm looking at so I could help them figure out what was up. Re-arranged every update (or new phone) for absolutely no reason, terrible affordances, poor use of their own design language. Ugh. I'd get them on iOS but they need larger screens and the $400 small iPhones (what I have) are already more expensive than they think a phone "should" cost, so they keep buying $200 Android phones about once a year (hoping the next one will be better) and not being able to use them because the UI is garbage.

bombcar 4y ago
It's doubly sad because the larger phone or iPad would be perfect, but every year it's another $200 to tell Google they're doing the right thing. At least on iPad/iPhone you can set the apps to access Google mail, etc, which doesn't change as often, but still too often.
selfhoster11 4y ago
At one point I was trying to set up my grandma with a popular video calling app on a dedicated device so we could stay in touch. Before I could give her the freshly grandparent-proofed device, said video calling app upgraded on my parents' PC first and changed literally every single element of the UI beyond recognition. To someone the age of my grandma, that would be literally like bricking the device remotely, because none of the buttons would look the same, and she would not be able to work out how to use the new interface. STOP CHANGING THINGS! Even if the new UI is better (debatable), some people just like or rely on a particular layout to operate the device or app. Don't rearrange without giving a ~permanent setting to use the old layout.
- corrral 4y ago
  > STOP CHANGING THINGS! Even if the new UI is better (debatable), some people just like or rely on a particular layout to operate the device or app. Don't rearrange without giving a ~permanent setting to use the old layout. The fact that we're 15 years into smartphones being popular and that phones & computerized address books were basically fully-solved interfaces long before those took off, and they're still fucking around with phone app UI in big ways, is a sign of some kind of institutional or industry-market failure to me. Or both. Allowed, I suppose, because Google's market position (i.e. monopoly across several markets) both gives them the surplus profit (i.e. rents) to fuck around and waste money on harmful to users crap like phone app redesigns, and insulates them from any actual threat to their profit due to those bad choices.

amelius 4y ago

Yes, I hate Apple, but I'm starting to hate Google even more. One of these days, I might switch too. Why can't we have nice things?

quickthrower2 4y ago

I moved back to iPhone in the 2015s onwards (I think) because of the crap quality of devices available, and the bloatware atop if android. In many cases the hardware was so poor it was hard to make a call due to the touchscreen. Since the primary thing I want the phone to do is make a call I switched to the “it just works” camp and haven’t regretted it. Except getting photos off the phone. Until I realised the best tool for that is … Ubuntu!

Lutger 4y ago

> To me, capitalism (A/B testing) means you don't respect your citizens (users). You see them as just one factor in your money machine that can be poked and prodded to optimize how much money you can squeeze out of them. You just described doing business in todays world. Being a bit more generous towards A/B testing, I would make a counter point: _not_ doing any kind of user testing, of which large scale automated A/B tests are just a subset, means you don't respect your users. Because it means you just assume you know what their experience is like, or worse: you don't even care about it and bother to learn something. Your complaint seems to be more about the scale and aspect of automation honestly, and continuity of the services, which is a valid complaint against Google but not about A/B testing in general.

saagarjha 4y ago
> Because it means you just assume you know what their experience is like, or worse: you don't even care about it and bother to learn something. A/B tests are not the only, or even the best, way of collecting user feedback.

bentcorner 4y ago

Google recently shut off Hangouts on me with a flight (since several of my contacts reported that Hangouts worked fine for them). It's kind of mind boggling they'd decide to do that - the replacement they direct me to (Google Chat) doesn't even have feature parity so I just dropped them and moved my social circle using Hangouts to a different app (since at this point they all faced the same problem and we decided on a different platform). I'm really curious how the A/B testing for this went down - Google is willingly throwing customers away because somebody wants to pump numbers for a new app that is objectively worse than the old one. At this point Google Maps is the only product that is keeping me with them, but even that one is beginning to wear thin.

jklinger410 4y ago
> Google Chat) doesn't even have feature parity Which features are you missing?
- cupofpython 4y ago
  check the AB test results
- bentcorner 4y ago
  Video chat. It redirects me to Google Meet (yet another app!) and the model that Google Meet uses is completely backwards (create a meeting, then invite people) compared to how I use Hangouts (call someone). It's a complete pita and the family-acceptable-factor is low.

adrr 4y ago

We AB tested a performance enhancement to our frontend web app to show that speed had significant benefits to the business. we use the results justify the investment cost. We spent the next six months working on making the site faster because of it. It is a tool. How would you measure things without ab testing?

fisf 4y ago
The fact that you need an A/B test to demonstrate that frontend performance has an impact (on user experience first and foremost) in 2022 speaks volumes.
- jklinger410 4y ago
  Yet it says nothing. Large companies are slow and stubborn.
- cupofpython 4y ago
  this is the stance I take on AB tests. The are objectively good at things you shouldnt need objective evidence for
- ajmurmann 4y ago
  But how big is the impact and does that return compare to that of other investments you aren't making because you are improving performance instead?

dakial1 4y ago

The problem is not the AB testing, as it can be a good thing to improve the experience, the problem is poorly set, short sighted, OKRs. The author points that in the text, as he mentions many times that the leadership never asks the right questions, mainly how it will impact the client in the long run (being NPS evolution, lifetime value, etc..)

brynjolf 4y ago

Ironically the place I'm getting forced A/B testing now is the Playstore. They moved the update apps section so it is now 2-3 clicks more, making me have to make a seperate shortcut just to reach the app update quickly. The point is of course to make it annoying to manually update apps and enable auto update. I have been burned too many times with a auto update so I refuse. This wasn't enough, they really want to force me enable auto updates to the point of the update section of the app having 50% of the visible space on my big screen being covered with a message to enable auto update over WiFi. [0] Whoever is doing this at Google... Stop. Just stop. It is cringe. [0] https://i.imgur.com/MbXn9gy.jpg https://i.imgur.com/MbXn9gy.jpg

fouric 4y ago

A few months ago, I came to the belief that Google is the ADHD toddler of user-facing software development - absolutely unable to sit still and concentrate on anything, hence the constant UI/UX churn, half-baked products, and graveyard[1] of shiny things that they worked on for a few years before abandoning. Google seems to be really good at making developer tools like Borg and Blaze - however, I think that as an organization they have some deficit that makes them not responsible enough to develop user-facing software (like, uh, an operating system). Maybe Google would be better as a B2B company. [1] https://killedbygoogle.com/ https://killedbygoogle.com/