4 ms·

I don't see the purpose. There seems to be an assumption that A/B testing is inherently bad, therefor should be opt-in. A/B testing is necessary for stable rol

I don't see the purpose. There seems to be an assumption that A/B testing is inherently bad, therefor should be opt-in. A/B testing is necessary for stable rollouts, that much seems self evident. So what is the evidence that A/B testing is harmful to users?

badsectoracula 7y ago
It isn't A/B testing that is bad, it is running and/or modifying stuff in the user's computer without their knowledge that is bad. If a site modifies some part of their layout or whatever isn't bad, but a browser installing addons or changing settings to see if it will crash is bad.
- staticassertion 7y ago
  Well, it's not like they're hoping it will crash. Presumably they're hoping it won't. Wouldn't it be more irresponsible to deploy to every user without first testing it out?
  - chii 7y ago
    that's what the stable/experimental channels are for - a user who don't want to be tested on (such as a business user) can easily be sure that they are getting a stable release and their settings won't update under them. Experimental/beta channels can have these changes pushed to it, since that's implied by the name, and those people who like living on the edge of new features do so knowingly. Let the users control their destiny, let the users decide when. That's the hallmark of a trustworthy company.
    - zlynx 7y ago
      Users who intentionally run beta / nightly are not normal users. How many of them run on cheap "desktop" computers they bought on sale at Target, do you think?
- scarejunba 7y ago
  Wait, so running and modifying stuff on all users' computers is good, but a random sample isn't? That's what A/B testing deals with. The discussion isn't "modify stuff" vs. "don't modify stuff". The discussion is "some users" vs. "all users". And obviously if you have an auto-update mechanism you're modifying stuff on the user's computer. Chrome updates are near invisible on modern machines. That's a good thing. It keeps the browser evergreen and protects users. If everything were opt-in because "Google shouldn't modify stuff on my computer" that would be an absolute disaster for everyone. No way.
  - Crinus 7y ago
    This is a technicality, users do not care about technicalities, they have their own expectations and if you are on -say- a stable branch with autoupdates enabled your expectation is that you'll get the latest version that everyone else is using. A/B testing through that branch breaks that expectation (and no, mentioning it somewhere hidden in some EULA or equally obfuscated place does not mean people will not have the expectation - hell, even mentioning it right below the download wont make a difference).

badsectoracula 7y ago

It isn't A/B testing that is bad, it is running and/or modifying stuff in the user's computer without their knowledge that is bad. If a site modifies some part of their layout or whatever isn't bad, but a browser installing addons or changing settings to see if it will crash is bad.

staticassertion 7y ago
Well, it's not like they're hoping it will crash. Presumably they're hoping it won't. Wouldn't it be more irresponsible to deploy to every user without first testing it out?
- chii 7y ago
  that's what the stable/experimental channels are for - a user who don't want to be tested on (such as a business user) can easily be sure that they are getting a stable release and their settings won't update under them. Experimental/beta channels can have these changes pushed to it, since that's implied by the name, and those people who like living on the edge of new features do so knowingly. Let the users control their destiny, let the users decide when. That's the hallmark of a trustworthy company.
  - zlynx 7y ago
    Users who intentionally run beta / nightly are not normal users. How many of them run on cheap "desktop" computers they bought on sale at Target, do you think?
scarejunba 7y ago
Wait, so running and modifying stuff on all users' computers is good, but a random sample isn't? That's what A/B testing deals with. The discussion isn't "modify stuff" vs. "don't modify stuff". The discussion is "some users" vs. "all users". And obviously if you have an auto-update mechanism you're modifying stuff on the user's computer. Chrome updates are near invisible on modern machines. That's a good thing. It keeps the browser evergreen and protects users. If everything were opt-in because "Google shouldn't modify stuff on my computer" that would be an absolute disaster for everyone. No way.
- Crinus 7y ago
  This is a technicality, users do not care about technicalities, they have their own expectations and if you are on -say- a stable branch with autoupdates enabled your expectation is that you'll get the latest version that everyone else is using. A/B testing through that branch breaks that expectation (and no, mentioning it somewhere hidden in some EULA or equally obfuscated place does not mean people will not have the expectation - hell, even mentioning it right below the download wont make a difference).