3 ms·

It's also ridiculous because it presupposes an objectively correct way to rank results. The way Facebook places items on the page isn't the result of some natu

It's also ridiculous because it presupposes an objectively correct way to rank results. The way Facebook places items on the page isn't the result of some natural law, it's the result of code they wrote. Nobody can seriously say that That Way of ranking results is superior to This Way, and certainly not that This Way is somehow illegal.

scarmig 12y ago
I wonder if a retailer hiking or cutting prices would fall afoul of the legal idea being pushed here. After all, it amounts to an experiment on people to discover the ideal price of a good.
- colbyh 12y ago
  This blog post's interpretation of "illegal" would include any change to any product with the intent to stimulate user behavior. It's a very slippery slope.
pdkl95 12y ago
Sure there is - you provide the service that customers signed up for. To most people that would be showing them all the results they expect. The research at Facebook selectively left some things out, which is clearly not what the customer signed up for. Even better, the entire issue can be avoided by placing the decision in the customer's hands. Let them decide what kind of results they want. You can provide all the tools you want when it is the customer that is voluntarily choosing to use them. You might even learn something interesting from that choice, too. /* now, to the thread in general, and not specifically the above comment / What so many here seem to be missing is that this is about respecting the customer and not trying to go behind their back. They chose to use a service for some reason(s), and suddenly changing that without notifying them or asking for their input* is at a minimum highly unethical (and plain rude). As this article points out, sometime it is also illegal. Depending on the laws, contracts, and/or monetary transactions involved (if any) this kind of unilateral modification to the relationship may conflict with contract law or various business laws as well. As some may people seem to have not read the article, a key quote: "The central pillar of modern research ethics is that in most cases researchers don’t get to decide for themselves whether an experiment is worth it. Unless the risks are minimal or nonexistent, that decision belongs to the participants, not to the researchers. That’s what the Common Rule does: it systematically takes these decisions out of researchers’ hands, and gives them to participants and IRBs." Yes, whenever humans are involved, the decision to modify ("experiment") on humans in any way cannot be made by the experimenters. Which bring me to the thing that I find most damning about this whole situation: the Facebook[1] and OkCupid experiments was fairly trivial. In in my not-an-IRB opinion, there is little risk to any of the participants; it's trivial compared to experiments like drug trials or other medical research. If they asked an IRB first, they could have gotten their approval quite easily. If, due to the specifics of the experiment, proper Informed Consent would have made the experiment useless, the IRB could have easily waived some of those requirements. So no, various testing is not "illegal". You can try new things and see how it affects your customers. We have simply decided as a society (and try and ensure with various federal and state laws) that if it involves humans, you need a neutral 3rd party to make that decision. Yes, this slows[2] you down a bit, but again approval for an experiment like those done by Facebook and OkCupid should be fast and easy. Be glad you're not doing medical research with nasty risk. Traditionally, if there is a question as to if approval is needed you should assume you do need it. If you are going behind the customers back to change something from the expectations[3] they initially signed up for without their approval, you absolutely need IRB approval. [1] As I understand it, it may be Cornell that was the initially at fault for not getting IRB approval, so they deserve some of the blame in that case. Regardless, as the researchers Facebook has a duty (and sometimes, a legal requirement) to make sure that approval was done properly before they proceeded with the experiment. As an analogy, consider that before you do any penetration testing it is important make damn sure you have all your contracts and approvals properly signed. You don't rely on "but the CEO said so" or "We just tried some common passwords and did a few nmap scans - they shouldn't even notice", you get explicit approval first. [2] If getting IRB approval too slow for the tech industry, they could try working with existing IRBs to try and come up with some new way of fast-tracking some types of experiments or they could create some sort of new mechanism that better fits the types of experiments would like to do regularly. The important part is that a 3rd party - not just the researchers - has vetted the experiment. The methods to get that approval can probably be modified. [3] While it's not related at all to the topic of human testing and the IRB, HN readers my find the talk given by Joe Peacock at NOTACON 8 about how they (fark.com) screwed up their website redesign in the infamous "you'll get over it" incident. The point of the talk is about the dangers of violating the expectations of your users. Not only is it an ethical/legal issue in some cases, the attitude where the customer was not even involved in the change (even trivial) can cause them to leave you surprisingly fast. Simply asking or otherwise involving you users is the only real way to keep them in the long run. (it's fark, so mild NSFW language warning) https://www.youtube.com/watch?v=YnVeysllPDI https://www.youtube.com/watch?v=YnVeysllPDI

scarmig 12y ago

I wonder if a retailer hiking or cutting prices would fall afoul of the legal idea being pushed here. After all, it amounts to an experiment on people to discover the ideal price of a good.

colbyh 12y ago
This blog post's interpretation of "illegal" would include any change to any product with the intent to stimulate user behavior. It's a very slippery slope.

pdkl95 12y ago

Sure there is - you provide the service that customers signed up for. To most people that would be showing them all the results they expect. The research at Facebook selectively left some things out, which is clearly not what the customer signed up for. Even better, the entire issue can be avoided by placing the decision in the customer's hands. Let them decide what kind of results they want. You can provide all the tools you want when it is the customer that is voluntarily choosing to use them. You might even learn something interesting from that choice, too. /* now, to the thread in general, and not specifically the above comment / What so many here seem to be missing is that this is about respecting the customer and not trying to go behind their back. They chose to use a service for some reason(s), and suddenly changing that without notifying them or asking for their input* is at a minimum highly unethical (and plain rude). As this article points out, sometime it is also illegal. Depending on the laws, contracts, and/or monetary transactions involved (if any) this kind of unilateral modification to the relationship may conflict with contract law or various business laws as well. As some may people seem to have not read the article, a key quote: "The central pillar of modern research ethics is that in most cases researchers don’t get to decide for themselves whether an experiment is worth it. Unless the risks are minimal or nonexistent, that decision belongs to the participants, not to the researchers. That’s what the Common Rule does: it systematically takes these decisions out of researchers’ hands, and gives them to participants and IRBs." Yes, whenever humans are involved, the decision to modify ("experiment") on humans in any way cannot be made by the experimenters. Which bring me to the thing that I find most damning about this whole situation: the Facebook[1] and OkCupid experiments was fairly trivial. In in my not-an-IRB opinion, there is little risk to any of the participants; it's trivial compared to experiments like drug trials or other medical research. If they asked an IRB first, they could have gotten their approval quite easily. If, due to the specifics of the experiment, proper Informed Consent would have made the experiment useless, the IRB could have easily waived some of those requirements. So no, various testing is not "illegal". You can try new things and see how it affects your customers. We have simply decided as a society (and try and ensure with various federal and state laws) that if it involves humans, you need a neutral 3rd party to make that decision. Yes, this slows[2] you down a bit, but again approval for an experiment like those done by Facebook and OkCupid should be fast and easy. Be glad you're not doing medical research with nasty risk. Traditionally, if there is a question as to if approval is needed you should assume you do need it. If you are going behind the customers back to change something from the expectations[3] they initially signed up for without their approval, you absolutely need IRB approval. [1] As I understand it, it may be Cornell that was the initially at fault for not getting IRB approval, so they deserve some of the blame in that case. Regardless, as the researchers Facebook has a duty (and sometimes, a legal requirement) to make sure that approval was done properly before they proceeded with the experiment. As an analogy, consider that before you do any penetration testing it is important make damn sure you have all your contracts and approvals properly signed. You don't rely on "but the CEO said so" or "We just tried some common passwords and did a few nmap scans - they shouldn't even notice", you get explicit approval first. [2] If getting IRB approval too slow for the tech industry, they could try working with existing IRBs to try and come up with some new way of fast-tracking some types of experiments or they could create some sort of new mechanism that better fits the types of experiments would like to do regularly. The important part is that a 3rd party - not just the researchers - has vetted the experiment. The methods to get that approval can probably be modified. [3] While it's not related at all to the topic of human testing and the IRB, HN readers my find the talk given by Joe Peacock at NOTACON 8 about how they (fark.com) screwed up their website redesign in the infamous "you'll get over it" incident. The point of the talk is about the dangers of violating the expectations of your users. Not only is it an ethical/legal issue in some cases, the attitude where the customer was not even involved in the change (even trivial) can cause them to leave you surprisingly fast. Simply asking or otherwise involving you users is the only real way to keep them in the long run. (it's fark, so mild NSFW language warning) https://www.youtube.com/watch?v=YnVeysllPDI https://www.youtube.com/watch?v=YnVeysllPDI