3 ms·

One thing that unnerves me about A/B testing is that you can't test for evilness. How much of this glossary[1] do you think is intentionally designed? I doubt t

One thing that unnerves me about A/B testing is that you can't test for evilness. How much of this glossary[1] do you think is intentionally designed? I doubt that all of these changes were intentional, it's just that those changes showed objective results. A/B testing can't test for morality, and you may very well be implying something with your B design that you didn't mean to imply, which none the less rates higher in your test. In a business setting, it becomes awfully hard to argue morality against objective numbers. It's hard to do that anyway. Many business operate with a profit first motive. So once a dark pattern is in, how are you going to get it out? Don't get me wrong, A/B testing is a tool, and like all good tools it can be used for good or evil. It just worries me that A/B testing, despite good intentions, can lead to evil results. I don't see anything on this page about how to avoid evil results. 1. http://darkpatterns.org/ http://darkpatterns.org/

wikwocket 13y ago
I can see how you don't want to be in a situation where you are holding in your hand evidence that Shady Marketing Tactic X is proven to convert better. But I think the solution is simple: Don't A/B test shady features. If something makes you uncomfortable or seems like a dark pattern, don't test it, because you would never want it on your site anyway. I don't think this is a failing of A/B testing. That someone can get numbers that show "evil" tactics can make money is not materially different from someone coding "evil" HTML in Notepad and getting results. As you said, it is just a tool.
- GhotiFish 13y ago
  The problem is you might not be aware you're making a shady feature. As an example, if you are making an ad for your app fooer. You A/B test some banners on an ad distribution network. You find that a simpler ad with a "download now" hyperlink graphic proves effective. No big deal right? People are going to be seeing these banners on places like news sites right? It's not tricking people. It's also pretty reasonable right? People are used to clicking on hyperlinks, it overcomes the resistance to click on images. A reasonable move for a banner ad to make. Except the ad distribution network serves content sites like softpedia as well. You saw the spike because you emulated the download button right under the mirror list, and people were confused. They downloaded your application under the pretext of getting something else. Or what about if I want a "Hey you should sign up for notifications for other job offerings in this field" message box? I'd love to have one of those in my application. I A/B test to see that this message box drastically increases signups. Except it was because people thought it was a paywall. The road to hell is paved with good intentions. Consider this case about the Weebly blogging service: http://minimaxir.com/2013/05/overly-attached-startup/ http://minimaxir.com/2013/05/overly-attached-startup/ They were A/B testing messaging for first week signups. This hyper aggressive and insane route was actually considered. Do you think they're black hat? I don't think so, but what if this had become standard practice?

wikwocket 13y ago

I can see how you don't want to be in a situation where you are holding in your hand evidence that Shady Marketing Tactic X is proven to convert better. But I think the solution is simple: Don't A/B test shady features. If something makes you uncomfortable or seems like a dark pattern, don't test it, because you would never want it on your site anyway. I don't think this is a failing of A/B testing. That someone can get numbers that show "evil" tactics can make money is not materially different from someone coding "evil" HTML in Notepad and getting results. As you said, it is just a tool.

GhotiFish 13y ago
The problem is you might not be aware you're making a shady feature. As an example, if you are making an ad for your app fooer. You A/B test some banners on an ad distribution network. You find that a simpler ad with a "download now" hyperlink graphic proves effective. No big deal right? People are going to be seeing these banners on places like news sites right? It's not tricking people. It's also pretty reasonable right? People are used to clicking on hyperlinks, it overcomes the resistance to click on images. A reasonable move for a banner ad to make. Except the ad distribution network serves content sites like softpedia as well. You saw the spike because you emulated the download button right under the mirror list, and people were confused. They downloaded your application under the pretext of getting something else. Or what about if I want a "Hey you should sign up for notifications for other job offerings in this field" message box? I'd love to have one of those in my application. I A/B test to see that this message box drastically increases signups. Except it was because people thought it was a paywall. The road to hell is paved with good intentions. Consider this case about the Weebly blogging service: http://minimaxir.com/2013/05/overly-attached-startup/ http://minimaxir.com/2013/05/overly-attached-startup/ They were A/B testing messaging for first week signups. This hyper aggressive and insane route was actually considered. Do you think they're black hat? I don't think so, but what if this had become standard practice?