5 ms·

How do you distinguish between unethical science and A/B testing, or even just 'We changed our product, then didn't like the results, and changed it back?'

zzzeek 8y ago
I would assume a core tenet of ethics and deciding if something is unethical is if it has the potential to harm people. Inducing depression to indiscriminate masses is pretty high on the list. It's not valid to compare the emotional impact of Facebook on millions of its addicted users to something like the color of a box of laundry detergent.
- deleted 8y ago
  [deleted]
- TulliusCicero 8y ago
  Selling cake has the potential to harm people, a moral framework needs a bit more subtlety than that to be sensible.
  - deleted 8y ago
    [deleted]
  - zzzeek 8y ago
    do folks "sell cake" to see if it worsens people's depression? this is more like selling cake that secretly has nuts in it which are known to cause allergic reactions without disclosing it.
    - staticassertion 8y ago
      Well, it's more like if the nuts had an unknown reaction, and the idea was to figure out what the reaction was by slipping it into some cake, and then observing.
      - zzzeek 8y ago
        which would be massively unethical if not illegal to do as a food manufacturer
  - ddingus 8y ago
    Well, including laxative to judge how much people can tolerate before they get the shits isn't far off the mark from inducing depression. FB - A little negative = meh FB - More negative = major downer FB - Totally negative = woah, look at the sorry state they are in now. Cake - A little laxative = mmm this cake is special somehow Cake - More laxative = nice cake, special, where is the roll of extra TP? Cake - Dump it in = people home for days, the shits. In both cases, not telling them is the ethics problem. Basically the public trust boils down to people expecting others not to harm them. Depression can become chronic. The shits could damage someone requiring medical help. Both are clear risks people would very likely avoid, if they knew. FB - We want to run a depression test. Plz volunteer your feed and see if you get depressed. Cake - We have a laxative that tastes great, but you might get the shits, plz have some cake. See the problem?
jacobush 8y ago
Medical trials are also just A/B testing. waves hands
- jethro_tell 8y ago
  true, but most of them have opt in. . .. Like people know they are in the test even if they don't know if they are the control or not.
  - jacobush 8y ago
    That was my point, apparently that was not obvious at all. That is what I meant by the "hand waving" part. Being apologetic about FB just because what they did was some kind of A/B testing does not redeem them at all.
794CD01 8y ago
The terms don't apply to the same context. A/B testing is a specific technique (well, not that specific) which can be used to do ethical science or unethical science.
euyyn 8y ago
I think one good discriminant is: Would you want to explain it candidly? E.g.: "Starting next week, we'll try to affect your mood as part of an experiment". Or "the new version of our product purposefully tries to depress you a bit" followed by "we didn't like the results of it so we've changed it back".
rhizome 8y ago
There is at least a dense 50 years of research and writing on the topic of scientific ethics, it's a bit naive to pop into a thread and ask, "so what's the deeeeaaaal with human experimentation?"
- smallnamespace 8y ago
  To be clear, I think the question is where the line is between 'experimentation' and 'just doing what people do normally'. Concrete example: I am in the process of trying to hire people right now. I'm not sure what the exact best way to do that is, so I vary how I interview over time and see how well it seems to work. I record what happens in a spreadsheet, and then later on I look at the 'data' to make some judgments. Am I experimenting on my interviewees? Am I potentially harming them? Well certainly if I ask question A in one interview, and question A' in another, and question A' is harder and I don't pass that person, then one might be tempted to argue that maybe that person would've done better on question A; therefore, I've harmed someone by 'experimenting' and making it harder for them to get a job. Rejecting someone can certainly make them depressed. In principle this is no different from A/B testing an interview process, using real live humans no less. So do I personally have an ethical obligation to disclose to everyone I interview that I am conducting human experimentation on them? Or is the only difference in power and scale—I am just one person, but Facebook can A/B test on millions of people at once?
  - secabeen 8y ago
    > Well certainly if I ask question A in one interview, and question A' in another, and question A' is harder and I don't pass that person, then one might be tempted to argue that maybe that person would've done better on question A; therefore, I've harmed someone by 'experimenting' and making it harder for them to get a job. Rejecting someone can certainly make them depressed. > In principle this is no different from A/B testing an interview process, using real live humans no less. > So do I personally have an ethical obligation to disclose to everyone I interview that I am conducting human experimentation on them? Or is the only difference in power and scale—I am just one person, but Facebook can A/B test on millions of people at once? Generally, when interviewing, we ask the same questions of each applicant. We adjust the questions from A to A' as positions are hired, and pools are assembled, but for a specific job, each candidate gets the same questions. It would have to be different for open recruitment not tied to a specific job, but I haven't had to do that.
sizzle 8y ago
Well if your study involves depressed people unknowingly being manipulated with negative newsfeed items that they didn't know they consented to being subjected to in a secret study, then that's unethical. The scientists aren't stupid, they knew beforehand that in a sufficiently sized population, there will be a segment of depressed people that they are influencing to commit suicide by being subjected to increased negative newsfeed material. That is unethical to me any way you cut it.

zzzeek 8y ago

I would assume a core tenet of ethics and deciding if something is unethical is if it has the potential to harm people. Inducing depression to indiscriminate masses is pretty high on the list. It's not valid to compare the emotional impact of Facebook on millions of its addicted users to something like the color of a box of laundry detergent.

deleted 8y ago
[deleted]
TulliusCicero 8y ago
Selling cake has the potential to harm people, a moral framework needs a bit more subtlety than that to be sensible.
- deleted 8y ago
  [deleted]
- zzzeek 8y ago
  do folks "sell cake" to see if it worsens people's depression? this is more like selling cake that secretly has nuts in it which are known to cause allergic reactions without disclosing it.
  - staticassertion 8y ago
    Well, it's more like if the nuts had an unknown reaction, and the idea was to figure out what the reaction was by slipping it into some cake, and then observing.
    - zzzeek 8y ago
      which would be massively unethical if not illegal to do as a food manufacturer
- ddingus 8y ago
  Well, including laxative to judge how much people can tolerate before they get the shits isn't far off the mark from inducing depression. FB - A little negative = meh FB - More negative = major downer FB - Totally negative = woah, look at the sorry state they are in now. Cake - A little laxative = mmm this cake is special somehow Cake - More laxative = nice cake, special, where is the roll of extra TP? Cake - Dump it in = people home for days, the shits. In both cases, not telling them is the ethics problem. Basically the public trust boils down to people expecting others not to harm them. Depression can become chronic. The shits could damage someone requiring medical help. Both are clear risks people would very likely avoid, if they knew. FB - We want to run a depression test. Plz volunteer your feed and see if you get depressed. Cake - We have a laxative that tastes great, but you might get the shits, plz have some cake. See the problem?

jacobush 8y ago

Medical trials are also just A/B testing. waves hands

jethro_tell 8y ago
true, but most of them have opt in. . .. Like people know they are in the test even if they don't know if they are the control or not.
- jacobush 8y ago
  That was my point, apparently that was not obvious at all. That is what I meant by the "hand waving" part. Being apologetic about FB just because what they did was some kind of A/B testing does not redeem them at all.

794CD01 8y ago

The terms don't apply to the same context. A/B testing is a specific technique (well, not that specific) which can be used to do ethical science or unethical science.

euyyn 8y ago

I think one good discriminant is: Would you want to explain it candidly? E.g.: "Starting next week, we'll try to affect your mood as part of an experiment". Or "the new version of our product purposefully tries to depress you a bit" followed by "we didn't like the results of it so we've changed it back".

rhizome 8y ago

There is at least a dense 50 years of research and writing on the topic of scientific ethics, it's a bit naive to pop into a thread and ask, "so what's the deeeeaaaal with human experimentation?"

smallnamespace 8y ago
To be clear, I think the question is where the line is between 'experimentation' and 'just doing what people do normally'. Concrete example: I am in the process of trying to hire people right now. I'm not sure what the exact best way to do that is, so I vary how I interview over time and see how well it seems to work. I record what happens in a spreadsheet, and then later on I look at the 'data' to make some judgments. Am I experimenting on my interviewees? Am I potentially harming them? Well certainly if I ask question A in one interview, and question A' in another, and question A' is harder and I don't pass that person, then one might be tempted to argue that maybe that person would've done better on question A; therefore, I've harmed someone by 'experimenting' and making it harder for them to get a job. Rejecting someone can certainly make them depressed. In principle this is no different from A/B testing an interview process, using real live humans no less. So do I personally have an ethical obligation to disclose to everyone I interview that I am conducting human experimentation on them? Or is the only difference in power and scale—I am just one person, but Facebook can A/B test on millions of people at once?
- secabeen 8y ago
  > Well certainly if I ask question A in one interview, and question A' in another, and question A' is harder and I don't pass that person, then one might be tempted to argue that maybe that person would've done better on question A; therefore, I've harmed someone by 'experimenting' and making it harder for them to get a job. Rejecting someone can certainly make them depressed. > In principle this is no different from A/B testing an interview process, using real live humans no less. > So do I personally have an ethical obligation to disclose to everyone I interview that I am conducting human experimentation on them? Or is the only difference in power and scale—I am just one person, but Facebook can A/B test on millions of people at once? Generally, when interviewing, we ask the same questions of each applicant. We adjust the questions from A to A' as positions are hired, and pools are assembled, but for a specific job, each candidate gets the same questions. It would have to be different for open recruitment not tied to a specific job, but I haven't had to do that.

sizzle 8y ago

Well if your study involves depressed people unknowingly being manipulated with negative newsfeed items that they didn't know they consented to being subjected to in a secret study, then that's unethical. The scientists aren't stupid, they knew beforehand that in a sufficiently sized population, there will be a segment of depressed people that they are influencing to commit suicide by being subjected to increased negative newsfeed material. That is unethical to me any way you cut it.