6 ms·

> Wait, does this refer to A/B testing? I hope so. Experimenting on your customers is about as user hostile as you can get.

by flukus 8y ago

onlyrealcuzzo 8y ago
Are you even serious? We're not talking about A/B testing prescription drugs with placebos. We're talking about testing different images. Different colors for buttons. Settle down.
- ixtli 8y ago
  Saying that A/B testing is just different colors for buttons is intentionally ignoring the past 10 years of facebooks development process. Every single aspect of the platform is AB tested and that platform has a big effect on peoples lives.
  - hackinthebochs 8y ago
    I'm still confused about why I'm supposed to be upset that Facebook A/B tests their features on their users. It seems to me that if they're allowed to do either A or B, they're allowed to measure the influence of A vs B. I don't see where the outrage is.
    - steve76 8y ago
      Open access to the default phonebook, and everyone's can be made to look different. If it is in the Senate, someone is doing it. They hear about active intelligence. Target the psycho demographic, and get them to drive by your house. Get striking workers madder. Dump a few thousand in vice ads, and destroy a city to get one person. It's used to go after threats they know, not to trip up good people.
    - ixtli 8y ago
      As I said in another comment this is about consent. It's not that the testing the conversion rates of button a versus button b is in and of itself immoral, it's that experimenting on people without their informed consent, under any circumstances, is. I'm intimately familiar with FBs platform as a developer and a user and its my intuition that 9/10 people aren't aware of the degree to which they are being experimented on via multivariate testing and I think a reasonable person would say they have a right to be informed of this. Another note is that after years of using the platform I can tell that when non-technical people DO become aware of the fact that their experience using the application is sometimes fundamentally different from others because they're in a non-control bucket they generally react pretty negatively to the notion. Sure, some of this is the standard "users always hate every UI change no matter what it is" syndrome but I've noted a lot of "this is creepy and i wonder how much it's been happening before" which is, imo, a super legitimate response, and shouldn't be disregarded because its inconvenient for fb to get consent.
      - sdenton4 8y ago
        Feature experiments are also a thing that exists. I want to deploy a new widget, and need to check that it works, and hasn't done something unexpected that drives users away. Experiments are how you do it. How about these: Are corner stores allowed to experiment with pricing? Are restaurants allowed to experiment with new menus? These are experiments involving humans. Are you just asking for poorly designed experiments? What you're asking for is companies to launch once and never know if it worked. And indeed, software used to be like that, and it sucked...
        flukus 8y ago
        > Feature experiments are also a thing that exists. I want to deploy a new widget, and need to check that it works, and hasn't done something unexpected that drives users away. Experiments are how you do it. Experiments and experiments on live non-consenting users are two different things. > How about these: Are corner stores allowed to experiment with pricing? Are restaurants allowed to experiment with new menus? These are experiments involving humans. Are you just asking for poorly designed experiments? Let a corner store charge different people different prices and let me know how far you get. The also have to deal with consequences fro their experiments, if a customer sees the price of an item has double in an experiment they're unlikely to come back, there's an asymmetry issue and not coming back is often not an option you have in an environment with lock-in and network effects. > What you're asking for is companies to launch once and never know if it worked. And indeed, software used to be like that, and it sucked... Yes, developers had to think through design decisions, stick to well defined HIG's and use controlled test groups, truly a dark age.
        deleted 8y ago
        [deleted]
        antidesitter 8y ago
        > Let a corner store charge different people different prices and let me know how far you get. Well, under your proposal they can’t know how far they’ll get. > Yes, developers had to think through design decisions, stick to well defined HIG's and use controlled test groups Well, did they? To a greater extent than today?
    - tremon 8y ago
      You shouldn't be upset about that. You should be upset that Facebook is performing tests on its users to optimize against the interests of those same users, without letting the user know what they're doing.
  - baroffoos 8y ago
    These mega websites should probably be held to a different standard to the rest. No ones life is changing when I try out different colors but some of the stuff facebook is testing is very unethical.
    - ken 8y ago
      I think the standard shouldn't be size, but type of software. Facebook is a platform. People expect (reasonably or not) some element of stability in a platform. I don't want even a small platform doing tests on me and my data. But if it's a game, or a blog? Knock yourself out, no matter how big it is.
      - ixtli 8y ago
        I agree with you but personally I dont see how its so onerous for blizzard or rockstar to tell me in plain language what it intends to do with its behavior tracking (or really that its tracking my behavior at all.) For me this is about consent, and I'm willing to consent to things that I'm made aware of. I mean, I'm a software developer too, I know there are legitimate use-cases here.
- ocdtrekkie 8y ago
  This: https://www.nytimes.com/2014/06/30/technology/facebook-tinkers-with-users-emotions-in-news-feed-experiment-stirring-outcry.html https://www.nytimes.com/2014/06/30/technology/facebook-tinke... Which is to say, some tech companies think that A/B experiments that might lead someone to commit suicide is okay.
  - hackinthebochs 8y ago
    If their feature A is OK to do in isolation, and could conceivably lead someone to commit suicide, then I don't see the problem with measuring that impact vs some feature B.
  - malandrew 8y ago
    The alternative is to not know at all that this is possible and accidentally design something that makes people more negative. Knowing this is possible and how to measure the effect, lets them detect when they accidentally do it and reverse course. Making it illegal to figure out the negative impacts of your decisions will make it harder to avoid them. It would be much better to require disclosure when these negative impacts are detected and require that this information can only ever be used in the best interest of the user.
    - ocdtrekkie 8y ago
      Please find me one real scientist who would argue it's ethical to encourage some people to commit suicide to find out how to avoid encouraging people to commit suicide. Just because the outcome of the research is potentially valuable doesn't mean it's ethical to conduct it on people, especially without their consent.
      - hackinthebochs 8y ago
        More accurately, expose people in a controlled setting to what they're already being exposed to, to find out its impact. But this sounds pretty neutral.
- vuln 8y ago
  Very serious. Where does it stop? "It has published details of a vast experiment in which it manipulated information posted on 689,000 users' home pages and found it could make people feel more positive or negative through a process of "emotional contagion". https://www.theguardian.com/technology/2014/jun/29/facebook-users-emotions-news-feeds https://www.theguardian.com/technology/2014/jun/29/facebook-...
  - gotocake 8y ago
    That’s not A/B testing, or at least not just A/B testing, that’s a full blown psych experiment.
    - rapind 8y ago
      But it's really not a one-off. This has become modern day marketing tactics. Guarantee someone gave a presentation today to a bunch of execs about how to manipulate a percentage of your users to achieve [x] goal by lightly "nudging" them.
ztratar 8y ago
Absolutely everything in business is an experiment. Experimenting with customers if what gives you the information to make a better product. It's the least user hostile thing you can get.
- ixtli 8y ago
  This seems to be ignoring consent. I'd bet you if I went around asking people whether or not they realized their FB app experience was being consistently multivariate tested i'd be on the street a while before someone said yes. This is FAR different from product testing, say in the hardware world, where you tell people you want them to come test a product, or in the design world where you show them various things and quiz them on their feelings. In these situations they all know they're being tested on. So no, this isn't "the least" user hostile thing you can do. Doing things without content is basically the prerequisite for hostility here.
  - phailhaus 8y ago
    > Doing things without content is basically the prerequisite for hostility here. Do you understand how often websites change without asking the user? Websites are constantly being updated, algorithms tweaked, features being added and taken away. You seem to be taking offense to the fact that they're providing a different experience to different subsets of the userbase? Is that what you're trying to ban? What could that possibly accomplish? If you don't have A/B testing, then websites are just going to do it the old fashioned way: collect data, make the feature change, compare the data. What does this solve?
    - ixtli 8y ago
      Look this clearly is not in the spirit of the argument I'm making. Why would I advocate for rules that restrict a websites ability to change? Again, im talking about consent when it comes to how your behavior is going to be used. Further I would say that the parent post isn't even about this. It's about protecting the consumer and yes, I would go so far as to say that if the "change" that the websites want to do has violates the rights of the user then yeah they should be restricted in their ability to do so!
sodosopa 8y ago
The goal is to improve their experience, that's the complete opposite of hostility. Edit. My response was about website a/b testing.
- gruez 8y ago
  >The goal is to improve their experience And how is that measured by most social media companies? "engagement".
- flukus 8y ago
  Changing random stuff all the time is not improving my experience, giving me a consistent interface is. And the goal is rarely to "improve my experience", it's to more efficiently manipulate me into spending more money, directly or indirectly.
- gcp123 8y ago
  For these companies "improve their experience" = optimizing for maximum time spent with the product even when it's not in the best interest of the people using the product. i.e. Netflix autoplay, algorithmic newsfeed that tends to show more outrage-inducing content, and artificial notifications that aren't from people you know, but are engineered to get you back into the product unnecessarily. They just want to capture as much of your attention as possible because it increases the number of ads they can show/sell. That's not improving user experience, that's hijacking attention to maximize profit. Considering attention is the primary instrument nature has given us for crafting our lives, that's a pretty user-hostile thing to do.
kadendogthing 8y ago
I hope everyone down voting this realizes that A/B testing has an "infinite" resolution. Most people when they hear A/B testing think of something like switching the color of a button, or what have you. But when you start A/B testing a complex series of permutations of components in aggregate, specifically designed to target certain psychological profiles, you can actually start to learn a lot about the group you're testing. To pretend that A/B testing is some simple "what image do you like more?" game is entirely disingenuous and is so characteristic of the attitude people in tech have when dealing with people or any kind of social side effects of systems they build. We absolutely do need to be very careful with this. Careful with how we test people, careful with how the information is recorded, and careful with what we do with the data. I'm going to start calling A/B testing "psychological side channel attacks" and maybe then places like HN will appreciate what's happening more.
- Zak 8y ago
  I think people on HN generally have a good understanding of what A/B testing is: "if I change X, will users do more of Y?" Of course Y is usually something that makes the site owner money. Sometimes A, B, both, or the entire business model of the site is unethical. In such cases, they would still be unethical if a comparison test was not run. In cases where none of them are unethical, I have trouble imagining a realistic scenario in which the act of running a comparison test makes it unethical.
  - kadendogthing 8y ago
    >I think people on HN generally have a good understanding of what A/B testing is Not only do I not believe this, as most people on HN tend to have a very superficial understanding of whatever tech is being discussed, there is always a dismissal of any social consequences that may happen as a result of using any kind of technology. >I have trouble imagining a realistic scenario in which the act of running a comparison test makes it unethical. I can't tell if you're being serious or not. You don't need any kind of testing framework for a few famous examples to satisfy your "realistic" qualification. I see a lot of bland contrarian stuff on HN, but I'm kind of speechless right now.
JulianMorrison 8y ago
Basically it depends on the experiment. Does the homepage work better? Good experiment. Can we manipulate people into feeling depressed or happy? Bad experiment.
- flukus 8y ago
  > Does the homepage work better? Good experiment. "Better" for who? The website owners interests are rarely in alignment with my interests. They want increased sales, higher user engagement, etc. I want less engagement and the ability to make informed choices on products. Let's say there's a product listing with it's list of features and this list has been extensively tweaked to maximize sales. As a result of that tweaking they took away a line item, that would have caused me to not buy it, say an annoying LED status indicator. This is good for the website owner but bad for me because I've lost the ability to make an informed decision. It's asymmetric manipulation and I'd regard it as immoral.

> Wait, does this refer to A/B testing? I hope so. Experimenting on your customers is about as user hostile as you can get.

by flukus 8y ago

> Wait, does this refer to A/B testing? I hope so. Experimenting on your customers is about as user hostile as you can get.

onlyrealcuzzo 8y ago

Are you even serious? We're not talking about A/B testing prescription drugs with placebos. We're talking about testing different images. Different colors for buttons. Settle down.

ixtli 8y ago
Saying that A/B testing is just different colors for buttons is intentionally ignoring the past 10 years of facebooks development process. Every single aspect of the platform is AB tested and that platform has a big effect on peoples lives.
- hackinthebochs 8y ago
  I'm still confused about why I'm supposed to be upset that Facebook A/B tests their features on their users. It seems to me that if they're allowed to do either A or B, they're allowed to measure the influence of A vs B. I don't see where the outrage is.
  - steve76 8y ago
    Open access to the default phonebook, and everyone's can be made to look different. If it is in the Senate, someone is doing it. They hear about active intelligence. Target the psycho demographic, and get them to drive by your house. Get striking workers madder. Dump a few thousand in vice ads, and destroy a city to get one person. It's used to go after threats they know, not to trip up good people.
  - ixtli 8y ago
    As I said in another comment this is about consent. It's not that the testing the conversion rates of button a versus button b is in and of itself immoral, it's that experimenting on people without their informed consent, under any circumstances, is. I'm intimately familiar with FBs platform as a developer and a user and its my intuition that 9/10 people aren't aware of the degree to which they are being experimented on via multivariate testing and I think a reasonable person would say they have a right to be informed of this. Another note is that after years of using the platform I can tell that when non-technical people DO become aware of the fact that their experience using the application is sometimes fundamentally different from others because they're in a non-control bucket they generally react pretty negatively to the notion. Sure, some of this is the standard "users always hate every UI change no matter what it is" syndrome but I've noted a lot of "this is creepy and i wonder how much it's been happening before" which is, imo, a super legitimate response, and shouldn't be disregarded because its inconvenient for fb to get consent.
    - sdenton4 8y ago
      Feature experiments are also a thing that exists. I want to deploy a new widget, and need to check that it works, and hasn't done something unexpected that drives users away. Experiments are how you do it. How about these: Are corner stores allowed to experiment with pricing? Are restaurants allowed to experiment with new menus? These are experiments involving humans. Are you just asking for poorly designed experiments? What you're asking for is companies to launch once and never know if it worked. And indeed, software used to be like that, and it sucked...
      - flukus 8y ago
        > Feature experiments are also a thing that exists. I want to deploy a new widget, and need to check that it works, and hasn't done something unexpected that drives users away. Experiments are how you do it. Experiments and experiments on live non-consenting users are two different things. > How about these: Are corner stores allowed to experiment with pricing? Are restaurants allowed to experiment with new menus? These are experiments involving humans. Are you just asking for poorly designed experiments? Let a corner store charge different people different prices and let me know how far you get. The also have to deal with consequences fro their experiments, if a customer sees the price of an item has double in an experiment they're unlikely to come back, there's an asymmetry issue and not coming back is often not an option you have in an environment with lock-in and network effects. > What you're asking for is companies to launch once and never know if it worked. And indeed, software used to be like that, and it sucked... Yes, developers had to think through design decisions, stick to well defined HIG's and use controlled test groups, truly a dark age.
        deleted 8y ago
        [deleted]
        antidesitter 8y ago
        > Let a corner store charge different people different prices and let me know how far you get. Well, under your proposal they can’t know how far they’ll get. > Yes, developers had to think through design decisions, stick to well defined HIG's and use controlled test groups Well, did they? To a greater extent than today?
  - tremon 8y ago
    You shouldn't be upset about that. You should be upset that Facebook is performing tests on its users to optimize against the interests of those same users, without letting the user know what they're doing.
- baroffoos 8y ago
  These mega websites should probably be held to a different standard to the rest. No ones life is changing when I try out different colors but some of the stuff facebook is testing is very unethical.
  - ken 8y ago
    I think the standard shouldn't be size, but type of software. Facebook is a platform. People expect (reasonably or not) some element of stability in a platform. I don't want even a small platform doing tests on me and my data. But if it's a game, or a blog? Knock yourself out, no matter how big it is.
    - ixtli 8y ago
      I agree with you but personally I dont see how its so onerous for blizzard or rockstar to tell me in plain language what it intends to do with its behavior tracking (or really that its tracking my behavior at all.) For me this is about consent, and I'm willing to consent to things that I'm made aware of. I mean, I'm a software developer too, I know there are legitimate use-cases here.
ocdtrekkie 8y ago
This: https://www.nytimes.com/2014/06/30/technology/facebook-tinkers-with-users-emotions-in-news-feed-experiment-stirring-outcry.html https://www.nytimes.com/2014/06/30/technology/facebook-tinke... Which is to say, some tech companies think that A/B experiments that might lead someone to commit suicide is okay.
- hackinthebochs 8y ago
  If their feature A is OK to do in isolation, and could conceivably lead someone to commit suicide, then I don't see the problem with measuring that impact vs some feature B.
- malandrew 8y ago
  The alternative is to not know at all that this is possible and accidentally design something that makes people more negative. Knowing this is possible and how to measure the effect, lets them detect when they accidentally do it and reverse course. Making it illegal to figure out the negative impacts of your decisions will make it harder to avoid them. It would be much better to require disclosure when these negative impacts are detected and require that this information can only ever be used in the best interest of the user.
  - ocdtrekkie 8y ago
    Please find me one real scientist who would argue it's ethical to encourage some people to commit suicide to find out how to avoid encouraging people to commit suicide. Just because the outcome of the research is potentially valuable doesn't mean it's ethical to conduct it on people, especially without their consent.
    - hackinthebochs 8y ago
      More accurately, expose people in a controlled setting to what they're already being exposed to, to find out its impact. But this sounds pretty neutral.
vuln 8y ago
Very serious. Where does it stop? "It has published details of a vast experiment in which it manipulated information posted on 689,000 users' home pages and found it could make people feel more positive or negative through a process of "emotional contagion". https://www.theguardian.com/technology/2014/jun/29/facebook-users-emotions-news-feeds https://www.theguardian.com/technology/2014/jun/29/facebook-...
- gotocake 8y ago
  That’s not A/B testing, or at least not just A/B testing, that’s a full blown psych experiment.
  - rapind 8y ago
    But it's really not a one-off. This has become modern day marketing tactics. Guarantee someone gave a presentation today to a bunch of execs about how to manipulate a percentage of your users to achieve [x] goal by lightly "nudging" them.

ztratar 8y ago

Absolutely everything in business is an experiment. Experimenting with customers if what gives you the information to make a better product. It's the least user hostile thing you can get.

ixtli 8y ago
This seems to be ignoring consent. I'd bet you if I went around asking people whether or not they realized their FB app experience was being consistently multivariate tested i'd be on the street a while before someone said yes. This is FAR different from product testing, say in the hardware world, where you tell people you want them to come test a product, or in the design world where you show them various things and quiz them on their feelings. In these situations they all know they're being tested on. So no, this isn't "the least" user hostile thing you can do. Doing things without content is basically the prerequisite for hostility here.
- phailhaus 8y ago
  > Doing things without content is basically the prerequisite for hostility here. Do you understand how often websites change without asking the user? Websites are constantly being updated, algorithms tweaked, features being added and taken away. You seem to be taking offense to the fact that they're providing a different experience to different subsets of the userbase? Is that what you're trying to ban? What could that possibly accomplish? If you don't have A/B testing, then websites are just going to do it the old fashioned way: collect data, make the feature change, compare the data. What does this solve?
  - ixtli 8y ago
    Look this clearly is not in the spirit of the argument I'm making. Why would I advocate for rules that restrict a websites ability to change? Again, im talking about consent when it comes to how your behavior is going to be used. Further I would say that the parent post isn't even about this. It's about protecting the consumer and yes, I would go so far as to say that if the "change" that the websites want to do has violates the rights of the user then yeah they should be restricted in their ability to do so!

sodosopa 8y ago

The goal is to improve their experience, that's the complete opposite of hostility. Edit. My response was about website a/b testing.

gruez 8y ago
>The goal is to improve their experience And how is that measured by most social media companies? "engagement".
flukus 8y ago
Changing random stuff all the time is not improving my experience, giving me a consistent interface is. And the goal is rarely to "improve my experience", it's to more efficiently manipulate me into spending more money, directly or indirectly.
gcp123 8y ago
For these companies "improve their experience" = optimizing for maximum time spent with the product even when it's not in the best interest of the people using the product. i.e. Netflix autoplay, algorithmic newsfeed that tends to show more outrage-inducing content, and artificial notifications that aren't from people you know, but are engineered to get you back into the product unnecessarily. They just want to capture as much of your attention as possible because it increases the number of ads they can show/sell. That's not improving user experience, that's hijacking attention to maximize profit. Considering attention is the primary instrument nature has given us for crafting our lives, that's a pretty user-hostile thing to do.

kadendogthing 8y ago

I hope everyone down voting this realizes that A/B testing has an "infinite" resolution. Most people when they hear A/B testing think of something like switching the color of a button, or what have you. But when you start A/B testing a complex series of permutations of components in aggregate, specifically designed to target certain psychological profiles, you can actually start to learn a lot about the group you're testing. To pretend that A/B testing is some simple "what image do you like more?" game is entirely disingenuous and is so characteristic of the attitude people in tech have when dealing with people or any kind of social side effects of systems they build. We absolutely do need to be very careful with this. Careful with how we test people, careful with how the information is recorded, and careful with what we do with the data. I'm going to start calling A/B testing "psychological side channel attacks" and maybe then places like HN will appreciate what's happening more.

Zak 8y ago
I think people on HN generally have a good understanding of what A/B testing is: "if I change X, will users do more of Y?" Of course Y is usually something that makes the site owner money. Sometimes A, B, both, or the entire business model of the site is unethical. In such cases, they would still be unethical if a comparison test was not run. In cases where none of them are unethical, I have trouble imagining a realistic scenario in which the act of running a comparison test makes it unethical.
- kadendogthing 8y ago
  >I think people on HN generally have a good understanding of what A/B testing is Not only do I not believe this, as most people on HN tend to have a very superficial understanding of whatever tech is being discussed, there is always a dismissal of any social consequences that may happen as a result of using any kind of technology. >I have trouble imagining a realistic scenario in which the act of running a comparison test makes it unethical. I can't tell if you're being serious or not. You don't need any kind of testing framework for a few famous examples to satisfy your "realistic" qualification. I see a lot of bland contrarian stuff on HN, but I'm kind of speechless right now.

JulianMorrison 8y ago

Basically it depends on the experiment. Does the homepage work better? Good experiment. Can we manipulate people into feeling depressed or happy? Bad experiment.

flukus 8y ago
> Does the homepage work better? Good experiment. "Better" for who? The website owners interests are rarely in alignment with my interests. They want increased sales, higher user engagement, etc. I want less engagement and the ability to make informed choices on products. Let's say there's a product listing with it's list of features and this list has been extensively tweaked to maximize sales. As a result of that tweaking they took away a line item, that would have caused me to not buy it, say an annoying LED status indicator. This is good for the website owner but bad for me because I've lost the ability to make an informed decision. It's asymmetric manipulation and I'd regard it as immoral.