5 ms·

> There was language that might have implied this was the case, but it was not clear or I was oblivious, probably the latter. For what it's worth, the language

> There was language that might have implied this was the case, but it was not clear or I was oblivious, probably the latter. For what it's worth, the language and layout was probably A/B tested and painstakingly tuned to maximise the probability of you making that mistake. Don't blame yourself when there's this unfair power imbalance, blame them for using their power against you maliciously.

PaulDavisThe1st 5y ago
> For what it's worth, the language and layout was probably A/B tested and painstakingly tuned to maximise the probability This sort of belief (that corporations do extensive testing and tuning of language) seems quite widely held, but I've never seen much support for it. I've frequently seen claims that Amazon must have "A/B tested and painstakingly tuned" the company name, whereas it was really just the founder's whim after watching a documentary, and never received any testing of any kind.
- tux3 5y ago
  Sometimes this happens as a sort of 'evolutionary' process. If a dodgy company has better wording for a misleading product, its deception will work better, so you can have highly tuned wording without anyone having to run an explicit A/B test.
  - patcon 5y ago
    Good point :) I'm glad to be reminded of that framing: a/b testing is just evolutionary selection within the same company-shaped container, but it's happening in the larger ecology even if you're not trying to run it.
- rp1 5y ago
  Have you ever worked at a large corporation like Facebook or Amazon? They have extensive internal A/B testing frameworks that record a wide variety of user metrics for arbitrary control and treatment exposures. It's very easy to set up an A/B test and pretty much necessary for almost all changes. Additionally, employees are expected to have concrete artifacts when they go up for promotion, and A/B test results are often a large part of that.
  - PaulDavisThe1st 5y ago
    > Have you ever worked at a large corporation like Facebook or Amazon? Employee #2 at amzn. But sure, that doesn't count because I got out before the company hit 20 employees. I don't doubt that large corporations with heavy technology dependence engage in what you're describing to cover some aspects of their operations. I do doubt that is covers all aspects, and in particular, I very much doubt if amzn A/B tested the language described in the GP for the purposes claimed by the parent of my comment. EDIT: Given the comment @ https://news.ycombinator.com/item?id=29374358 https://news.ycombinator.com/item?id=29374358 I should probably just accept that for amzn at least, I'm just wrong.
    - rp1 5y ago
      That's pretty impressive! Having worked at Amazon much later, I do think that most copy changes do go through an A/B test. I remember an instance where we tested 20 different versions of a minor UI change. Interestingly, there was a wide spread in the performance of the different treatments.
      - vhgyu75e6u 5y ago
        Current amzn employee working on the retail website. Unless a team is jumping/ignoring all standard operation precedures, everything must go through an A/B test to prove that it works at increasing the revenue or whatever the KPI target is. This includes text that might need to go through legal in some cases
        PaulDavisThe1st 5y ago
        Fair enough then. I'm wrong, for amzn anyway.
        version_five 5y ago
        I think to some extent you and the original post you replied to are both probably right. There is not some painstaking language optimization designed to trick users. But there is not a filter for clarity either, so the A/B testing naturally favors "dark patterns" - it's not malicious design, it's just unintended consequences, and gives the impression that Amazon et al are deliberately hostile when they are really just ignorant
- standardUser 5y ago
  Even the small startups I have worked for A/B test everything they can within inches of its life to eke out an extra penny and a half. This is standard operating procedure.
  - ericd 5y ago
    Every time I've seen this (and I agree that it's ubiquitous), it seems to make a ton of work that could be much better spent on making a better product that people actually want to use.
- puddingforears 5y ago
  The last five companies I’ve worked for, three large, one tiny start up, and two msb’s, extensively used A/B and split testing to carefully fine tune every funnel and product to maximize conversion. Most of my job for the last decade has been moving split points around form flows and capturing metrics to show what caused a split to occur. I talk to less than more people that don’t use this practice. I’ve run splits for everything from background colors to copy changes. If product management anticipates some facet could influence outcome, then it will be tested.
- anonymousab 5y ago
  > This sort of belief (that corporations do extensive testing and tuning of language) seems quite widely held, but I've never seen much support for it. I recall integrating with a beta AWS A/B testing sdk back in ~2015 (may be off on the year) and I'm certain that some the examples in the docs were along the lines of, say, testing two different wordings for a label in your app and measuring the change in engagement. But that's not inherently nefarious, it's just analytics. They ended up shutting down that SDK and the web dashboarding for it IIRC. Now, A/B testing for the purpose of tuning languages towards deceptive dark patterns? It's harder to prove, but I think it follows that if they're A/B testing other changes, there's no reason that deceptive changes wouldn't also be A/B tested as a matter of course.
- tata71 5y ago
  Bet your sweet ass they tested that smile in the logo.
  - PaulDavisThe1st 5y ago
    By the time they switched to that logo, they may have been large enough and "focused" enough to do so. But the initial amzn logo was picked merely on the whims of the founder and first two programmers. It was never focus-grouped or evaluated by anyone else.
- upbeat_general 5y ago
  I've personally worked at a large tech company and saw widespread A/B testing for things exactly as minute as this type of wording. It wasn't a constant dynamic A/B test that automatically selects the best results but rather a trial period and analysis for changes like this. I obviously can't speak to this case in particular but for pieces of text that are part of the core user flow (not necessarily true in this case), I'd be shocked if A/B tests weren't going on at any large company.

PaulDavisThe1st 5y ago

> For what it's worth, the language and layout was probably A/B tested and painstakingly tuned to maximise the probability This sort of belief (that corporations do extensive testing and tuning of language) seems quite widely held, but I've never seen much support for it. I've frequently seen claims that Amazon must have "A/B tested and painstakingly tuned" the company name, whereas it was really just the founder's whim after watching a documentary, and never received any testing of any kind.

tux3 5y ago
Sometimes this happens as a sort of 'evolutionary' process. If a dodgy company has better wording for a misleading product, its deception will work better, so you can have highly tuned wording without anyone having to run an explicit A/B test.
- patcon 5y ago
  Good point :) I'm glad to be reminded of that framing: a/b testing is just evolutionary selection within the same company-shaped container, but it's happening in the larger ecology even if you're not trying to run it.
rp1 5y ago
Have you ever worked at a large corporation like Facebook or Amazon? They have extensive internal A/B testing frameworks that record a wide variety of user metrics for arbitrary control and treatment exposures. It's very easy to set up an A/B test and pretty much necessary for almost all changes. Additionally, employees are expected to have concrete artifacts when they go up for promotion, and A/B test results are often a large part of that.
- PaulDavisThe1st 5y ago
  > Have you ever worked at a large corporation like Facebook or Amazon? Employee #2 at amzn. But sure, that doesn't count because I got out before the company hit 20 employees. I don't doubt that large corporations with heavy technology dependence engage in what you're describing to cover some aspects of their operations. I do doubt that is covers all aspects, and in particular, I very much doubt if amzn A/B tested the language described in the GP for the purposes claimed by the parent of my comment. EDIT: Given the comment @ https://news.ycombinator.com/item?id=29374358 https://news.ycombinator.com/item?id=29374358 I should probably just accept that for amzn at least, I'm just wrong.
  - rp1 5y ago
    That's pretty impressive! Having worked at Amazon much later, I do think that most copy changes do go through an A/B test. I remember an instance where we tested 20 different versions of a minor UI change. Interestingly, there was a wide spread in the performance of the different treatments.
    - vhgyu75e6u 5y ago
      Current amzn employee working on the retail website. Unless a team is jumping/ignoring all standard operation precedures, everything must go through an A/B test to prove that it works at increasing the revenue or whatever the KPI target is. This includes text that might need to go through legal in some cases
      - PaulDavisThe1st 5y ago
        Fair enough then. I'm wrong, for amzn anyway.
        version_five 5y ago
        I think to some extent you and the original post you replied to are both probably right. There is not some painstaking language optimization designed to trick users. But there is not a filter for clarity either, so the A/B testing naturally favors "dark patterns" - it's not malicious design, it's just unintended consequences, and gives the impression that Amazon et al are deliberately hostile when they are really just ignorant
standardUser 5y ago
Even the small startups I have worked for A/B test everything they can within inches of its life to eke out an extra penny and a half. This is standard operating procedure.
- ericd 5y ago
  Every time I've seen this (and I agree that it's ubiquitous), it seems to make a ton of work that could be much better spent on making a better product that people actually want to use.
puddingforears 5y ago
The last five companies I’ve worked for, three large, one tiny start up, and two msb’s, extensively used A/B and split testing to carefully fine tune every funnel and product to maximize conversion. Most of my job for the last decade has been moving split points around form flows and capturing metrics to show what caused a split to occur. I talk to less than more people that don’t use this practice. I’ve run splits for everything from background colors to copy changes. If product management anticipates some facet could influence outcome, then it will be tested.
anonymousab 5y ago
> This sort of belief (that corporations do extensive testing and tuning of language) seems quite widely held, but I've never seen much support for it. I recall integrating with a beta AWS A/B testing sdk back in ~2015 (may be off on the year) and I'm certain that some the examples in the docs were along the lines of, say, testing two different wordings for a label in your app and measuring the change in engagement. But that's not inherently nefarious, it's just analytics. They ended up shutting down that SDK and the web dashboarding for it IIRC. Now, A/B testing for the purpose of tuning languages towards deceptive dark patterns? It's harder to prove, but I think it follows that if they're A/B testing other changes, there's no reason that deceptive changes wouldn't also be A/B tested as a matter of course.
tata71 5y ago
Bet your sweet ass they tested that smile in the logo.
- PaulDavisThe1st 5y ago
  By the time they switched to that logo, they may have been large enough and "focused" enough to do so. But the initial amzn logo was picked merely on the whims of the founder and first two programmers. It was never focus-grouped or evaluated by anyone else.
upbeat_general 5y ago
I've personally worked at a large tech company and saw widespread A/B testing for things exactly as minute as this type of wording. It wasn't a constant dynamic A/B test that automatically selects the best results but rather a trial period and analysis for changes like this. I obviously can't speak to this case in particular but for pieces of text that are part of the core user flow (not necessarily true in this case), I'd be shocked if A/B tests weren't going on at any large company.