3 ms·

There may be cases where negative consequences can be predicted, but this sure isn't it. Of all the predictions I heard about Facebook when it was new, "fake n

There may be cases where negative consequences can be predicted, but this sure isn't it. Of all the predictions I heard about Facebook when it was new, "fake news" and "depression from comparing yourself to friends out having fun" and "an increasingly clickbaity and outrage-stoking newsmedia" were not among them. Not even close. Facebook is a great example of how programmers are not good at predicting negative consequences. The predictions I heard were mostly based on comparisons with Myspace and Livejournal. We are not good at this. Plumbers shouldn't try to do your electrical wiring, doctors shouldn't try to manage your database, legislators shouldn't try to pick your programming language, and programmers should not try to determine how society will react to technology.

ksdale 8y ago
I'm not sure it's so much a matter of picking negative consequences out of the blue sky of possible consequences, but rather wondering, "What happens if our software works exactly how we intend and we gain the ability to really manipulate people in a certain way?" I agree completely that it's hard or impossible to just predict in general how society will react to technology. But in many cases, there are people designing systems specifically for the purpose of getting people to behave a certain way. This implicitly acknowledges that they believe that they can affect behavior in a predictable way. If they can make predictions one way and be correct enough to profit from it, they should be capable of making predictions about things that could go wrong. I agree completely that predicting something like what Facebook will look like 10 years in the future is impossible, and there's surely a lot of research where the possible consequences are infinite. A lot of results in CS research are like someone developing a new material - "Like, sure, this could be used to make a new, extra deadly tank or something, but it could be used for literally anything." In that sense, it's not useful to make predictions, but I think there's also a lot of research going on at Facebook and Google that looks closer to what I mentioned above, where they absolutely have an intention to change people's behavior and so probably also have the ability to predict the consequences of their complete success.
- sdrinf 8y ago
  These are two different set of people: computer scientists develop algos / ml / etc, and want to publish them. UX designers , entrepreneurs, and other customer-focused people in the industry observe human behavior, and (amongst other activities) design traps around them. It is not useful to expect good predictions from the former group, and the later group isn't really interested in publishing in CS.
  - 0xdeadbeefbabe 8y ago
    The Challenger disaster is a good example of the engineer vs group dynamic https://www.npr.org/sections/thetwo-way/2016/03/21/470870426/challenger-engineer-who-warned-of-shuttle-disaster-dies https://www.npr.org/sections/thetwo-way/2016/03/21/470870426...
- nostrademons 8y ago
  "What happens if our software works exactly how we intend and we gain the ability to really manipulate people in a certain way?" This is still really difficult, and quite ethically fraught. When I was in high school and college (late 90s & early 2000s), I was one of the techno-utopians who believed that the Internet would usher in a new era of censorship-resistant free speech, free information, and free exchange of ideas with people across the globe, regardless of whatever walk of life you happened to come from. It would give a voice to marginalized people regardless of how niche their views are. By and large, it succeeded in this, and this site (HackerNews) is an example of that. I have no idea who you are or where you come from, and it could be literally half a world away from me, but I can still post this comment, and probably hundreds to thousands of people will read it. Yet nowhere on our radar screen was the idea that maybe we wished some of those marginalized groups would stay marginalized. That in addition to giving a voice to dissidents and hobbyists and nerds and subcultures, we would also give a voice to racists and militants and pedophiles and drug traffickers. Why would we even consider them? The definition of "marginalized" involves being "out of sight of the mainstream". Yet now, one of the main complaints being levied against Silicon Valley companies is that they aren't doing enough to police hate speech or kick undesirable groups off their platforms. And then this is hugely ethically fraught, because I'm sure that from the perspective of people in those groups, I'm the evil powermonger, and they are just exercising their human right to believe what they want. Silicon Valley has remained largely libertarian in outlook, but what if people get their wish and it starts cracking down on undesirables? Who's to say that your group would actually be one of the desirables? The new boss starts to look an awful lot like the old boss.
techbio 8y ago
Perhaps not predictable at the outset, but surely FB developers had insights into the usage/sentiment data long before reporters got onto it?
- deleted 8y ago
  [deleted]
rosser 8y ago
The idea that how we organize ourselves socially online might lead to echo chambers and deepening division was around long before the phrase "social media" had ever been used. That this phenomenon came about through the heretofore unprecedented (sarcasm) exploitation of human credulity and vanity, that dopamine spikes are the very mechanism of addiction, and that we react more strongly to things that trip our amygdalae than our cortices, should surprise precisely no-one — least of all the people who were building tools predicated upon those things. None of this stuff is new. None of this stuff is news. You don't get to play ignorant of the implications of the things you're building. Not while you're selling your time on the basis of how smart you are, anyway.
inimino 8y ago
I think you (and some other commenters here) are making the mistake of saying that because we cannot predict the future perfectly, that we are absolved of any responsibility for thinking about negative outcomes. > programmers should not try to determine how society will react to technology That is exactly one of the most important parts of our job. Do you think doctors should not think about the wider impact of treatment decisions? Legislators should not think about the second-order effects of the laws they pass? Yes, prediction is difficult (especially about the future, as Yogi Berra said). We all know this. Most of the things that most programmers are doing have known and predictable effects, because most of what we are doing is not new. If we had no idea how society would react to technology, why would we even develop it? Of course thinking about the effect that your work has on the world is a basic responsibility of any human being, how it could possibly be otherwise?
- ratacat 8y ago
  Respect.

ksdale 8y ago

I'm not sure it's so much a matter of picking negative consequences out of the blue sky of possible consequences, but rather wondering, "What happens if our software works exactly how we intend and we gain the ability to really manipulate people in a certain way?" I agree completely that it's hard or impossible to just predict in general how society will react to technology. But in many cases, there are people designing systems specifically for the purpose of getting people to behave a certain way. This implicitly acknowledges that they believe that they can affect behavior in a predictable way. If they can make predictions one way and be correct enough to profit from it, they should be capable of making predictions about things that could go wrong. I agree completely that predicting something like what Facebook will look like 10 years in the future is impossible, and there's surely a lot of research where the possible consequences are infinite. A lot of results in CS research are like someone developing a new material - "Like, sure, this could be used to make a new, extra deadly tank or something, but it could be used for literally anything." In that sense, it's not useful to make predictions, but I think there's also a lot of research going on at Facebook and Google that looks closer to what I mentioned above, where they absolutely have an intention to change people's behavior and so probably also have the ability to predict the consequences of their complete success.

sdrinf 8y ago
These are two different set of people: computer scientists develop algos / ml / etc, and want to publish them. UX designers , entrepreneurs, and other customer-focused people in the industry observe human behavior, and (amongst other activities) design traps around them. It is not useful to expect good predictions from the former group, and the later group isn't really interested in publishing in CS.
- 0xdeadbeefbabe 8y ago
  The Challenger disaster is a good example of the engineer vs group dynamic https://www.npr.org/sections/thetwo-way/2016/03/21/470870426/challenger-engineer-who-warned-of-shuttle-disaster-dies https://www.npr.org/sections/thetwo-way/2016/03/21/470870426...
nostrademons 8y ago
"What happens if our software works exactly how we intend and we gain the ability to really manipulate people in a certain way?" This is still really difficult, and quite ethically fraught. When I was in high school and college (late 90s & early 2000s), I was one of the techno-utopians who believed that the Internet would usher in a new era of censorship-resistant free speech, free information, and free exchange of ideas with people across the globe, regardless of whatever walk of life you happened to come from. It would give a voice to marginalized people regardless of how niche their views are. By and large, it succeeded in this, and this site (HackerNews) is an example of that. I have no idea who you are or where you come from, and it could be literally half a world away from me, but I can still post this comment, and probably hundreds to thousands of people will read it. Yet nowhere on our radar screen was the idea that maybe we wished some of those marginalized groups would stay marginalized. That in addition to giving a voice to dissidents and hobbyists and nerds and subcultures, we would also give a voice to racists and militants and pedophiles and drug traffickers. Why would we even consider them? The definition of "marginalized" involves being "out of sight of the mainstream". Yet now, one of the main complaints being levied against Silicon Valley companies is that they aren't doing enough to police hate speech or kick undesirable groups off their platforms. And then this is hugely ethically fraught, because I'm sure that from the perspective of people in those groups, I'm the evil powermonger, and they are just exercising their human right to believe what they want. Silicon Valley has remained largely libertarian in outlook, but what if people get their wish and it starts cracking down on undesirables? Who's to say that your group would actually be one of the desirables? The new boss starts to look an awful lot like the old boss.

techbio 8y ago

Perhaps not predictable at the outset, but surely FB developers had insights into the usage/sentiment data long before reporters got onto it?

deleted 8y ago
[deleted]

rosser 8y ago

The idea that how we organize ourselves socially online might lead to echo chambers and deepening division was around long before the phrase "social media" had ever been used. That this phenomenon came about through the heretofore unprecedented (sarcasm) exploitation of human credulity and vanity, that dopamine spikes are the very mechanism of addiction, and that we react more strongly to things that trip our amygdalae than our cortices, should surprise precisely no-one — least of all the people who were building tools predicated upon those things. None of this stuff is new. None of this stuff is news. You don't get to play ignorant of the implications of the things you're building. Not while you're selling your time on the basis of how smart you are, anyway.

inimino 8y ago

I think you (and some other commenters here) are making the mistake of saying that because we cannot predict the future perfectly, that we are absolved of any responsibility for thinking about negative outcomes. > programmers should not try to determine how society will react to technology That is exactly one of the most important parts of our job. Do you think doctors should not think about the wider impact of treatment decisions? Legislators should not think about the second-order effects of the laws they pass? Yes, prediction is difficult (especially about the future, as Yogi Berra said). We all know this. Most of the things that most programmers are doing have known and predictable effects, because most of what we are doing is not new. If we had no idea how society would react to technology, why would we even develop it? Of course thinking about the effect that your work has on the world is a basic responsibility of any human being, how it could possibly be otherwise?

ratacat 8y ago
Respect.