4 ms·

I agree that it's unlikely that anyone can predict broad, long-term consequences of any particular technology. On the other hand, the people who design Facebook

by ksdale 8y ago

I agree that it's unlikely that anyone can predict broad, long-term consequences of any particular technology. On the other hand, the people who design Facebook or mobile games to be maximally addicting definitely understand something about how their software is affecting society. It's not a huge leap to go from "I'm pretty sure that certain people will become obsessed with this," to "It's probably harmful for lots of people to become obsessed with this." If a person can predict the consequences of design decisions well enough to get paid for it, they'll probably have some idea of potential negative consequences as well.

throwawayjava 8y ago
More-over, I don't think parent is giving enough credit to certain types of computer scientists and certain companies. Facebook, Google, and some game companies definitely do hire lots of human-computer interaction Ph.D.s who definitely are trained in exactly this sort of thing. There's a reason the title is "computer science researches" and not "computer programmers".
throwawaymath 8y ago
> On the other hand, the people who design Facebook or mobile games to be maximally addicting definitely understand something about how their software is affecting society. It's not a huge leap to go from "I'm pretty sure that certain people will become obsessed with this," to "It's probably harmful for lots of people to become obsessed with this." I don't agree. I think this causal relationship makes sense in hindsight, but I don't think most people capable of designing or implementing software can reliably extrapolate what its impact on society will be. That goes for both positive and negative impacts. Researchers and engineers should still put the effort into questioning the nature of their work and its impact on society, because that's a good exercise in general. But I'm skeptical any given individual would be good at arriving to the right answer.
rossdavidh 8y ago
There may be cases where negative consequences can be predicted, but this sure isn't it. Of all the predictions I heard about Facebook when it was new, "fake news" and "depression from comparing yourself to friends out having fun" and "an increasingly clickbaity and outrage-stoking newsmedia" were not among them. Not even close. Facebook is a great example of how programmers are not good at predicting negative consequences. The predictions I heard were mostly based on comparisons with Myspace and Livejournal. We are not good at this. Plumbers shouldn't try to do your electrical wiring, doctors shouldn't try to manage your database, legislators shouldn't try to pick your programming language, and programmers should not try to determine how society will react to technology.
- ksdale 8y ago
  I'm not sure it's so much a matter of picking negative consequences out of the blue sky of possible consequences, but rather wondering, "What happens if our software works exactly how we intend and we gain the ability to really manipulate people in a certain way?" I agree completely that it's hard or impossible to just predict in general how society will react to technology. But in many cases, there are people designing systems specifically for the purpose of getting people to behave a certain way. This implicitly acknowledges that they believe that they can affect behavior in a predictable way. If they can make predictions one way and be correct enough to profit from it, they should be capable of making predictions about things that could go wrong. I agree completely that predicting something like what Facebook will look like 10 years in the future is impossible, and there's surely a lot of research where the possible consequences are infinite. A lot of results in CS research are like someone developing a new material - "Like, sure, this could be used to make a new, extra deadly tank or something, but it could be used for literally anything." In that sense, it's not useful to make predictions, but I think there's also a lot of research going on at Facebook and Google that looks closer to what I mentioned above, where they absolutely have an intention to change people's behavior and so probably also have the ability to predict the consequences of their complete success.
  - sdrinf 8y ago
    These are two different set of people: computer scientists develop algos / ml / etc, and want to publish them. UX designers , entrepreneurs, and other customer-focused people in the industry observe human behavior, and (amongst other activities) design traps around them. It is not useful to expect good predictions from the former group, and the later group isn't really interested in publishing in CS.
    - 0xdeadbeefbabe 8y ago
      The Challenger disaster is a good example of the engineer vs group dynamic https://www.npr.org/sections/thetwo-way/2016/03/21/470870426/challenger-engineer-who-warned-of-shuttle-disaster-dies https://www.npr.org/sections/thetwo-way/2016/03/21/470870426...
freeflight 8y ago
> On the other hand, the people who design Facebook or mobile games to be maximally addicting definitely understand something about how their software is affecting society. This is an interesting point, a whole lot of companies do actually hire soft-sciences talent specifically for that purpose, has been quite common in the gaming sector for years now (Blizzard, Valve, Bungie, just from the top of my head).
- mywittyname 8y ago
  This point backs up the statement of ~this parent~ rossdavidh's reply: programmers and engineers are woefully unequipped to perform any significant social science research. If they were, then tech companies would have little to no need to hire research psychologists or human factors specialists. They'd instead hire programmers and engineers that specialize in getting people addicted to technology.
  - rosser 8y ago
    So because the employer hires specialists in weaponizing people's brains against them (via manipulating addiction psychology, &c), the programmer is somehow off the hook for implementing things that he now knows to be designed thusly, on the basis that it's the work-product of those specialists? I'm reminded of the Tom Lehrer lyric: Once the rockets are up Who cares where they come down That's not my department Says Wernher von Braun
    - mywittyname 8y ago
      You're counter-argument is disingenuous, almost to the point of willful ignorance. It's obvious that a missile can be used to send bombs across the world just as well as it can be used to send satellites into space. Minecraft seems like a great form of entertainment that can be used as an educational tool. The fact that it might also be destructive to the development of children is non-obvious and takes some expertise outside the domain of software engineering to uncover.
      - 8y ago
LeifCarrotson 8y ago
It's much difficult to differentiate between "many people will enjoy and be entertained by this game" and "people will be harmfully obsessed with this game".
- ksdale 8y ago
  I agree it's hard to pick which games will be which, but it's not hard to predict that the more addicting a game is, the more people will be harmfully obsessed with it.
mywittyname 8y ago
> If a person can predict the consequences of design decisions well enough to get paid for it, they'll probably have some idea of potential negative consequences as well. Being able to hypothesize about potential risks and being able to perform experiments to verify the hypothesis are totally different skills. You can guess that maybe your game will be unhealthy for teenagers, but constructing an experiment to verify this is a whole different ball game.
zdragnar 8y ago
I think you're way over-estimating what a realistic ability to predict outcomes will be. I'm reminded of back when I worked in agency land, and clients would ask us to build something to "go viral" back when going viral was a new concept. They assumed it was as easy as just saying "make it so". Of course, it's never that easy to know that it will catch on. How exactly do you make something that is fun, engaging, and rewarding enough to encourage regular use, but not fun, engaging or rewarding enough to abuse? Seriously, if you can answer that question, you're sitting on a gold mine of knowledge.

by ksdale 8y ago

throwawayjava 8y ago

More-over, I don't think parent is giving enough credit to certain types of computer scientists and certain companies. Facebook, Google, and some game companies definitely do hire lots of human-computer interaction Ph.D.s who definitely are trained in exactly this sort of thing. There's a reason the title is "computer science researches" and not "computer programmers".

throwawaymath 8y ago

> On the other hand, the people who design Facebook or mobile games to be maximally addicting definitely understand something about how their software is affecting society. It's not a huge leap to go from "I'm pretty sure that certain people will become obsessed with this," to "It's probably harmful for lots of people to become obsessed with this." I don't agree. I think this causal relationship makes sense in hindsight, but I don't think most people capable of designing or implementing software can reliably extrapolate what its impact on society will be. That goes for both positive and negative impacts. Researchers and engineers should still put the effort into questioning the nature of their work and its impact on society, because that's a good exercise in general. But I'm skeptical any given individual would be good at arriving to the right answer.

rossdavidh 8y ago

There may be cases where negative consequences can be predicted, but this sure isn't it. Of all the predictions I heard about Facebook when it was new, "fake news" and "depression from comparing yourself to friends out having fun" and "an increasingly clickbaity and outrage-stoking newsmedia" were not among them. Not even close. Facebook is a great example of how programmers are not good at predicting negative consequences. The predictions I heard were mostly based on comparisons with Myspace and Livejournal. We are not good at this. Plumbers shouldn't try to do your electrical wiring, doctors shouldn't try to manage your database, legislators shouldn't try to pick your programming language, and programmers should not try to determine how society will react to technology.

ksdale 8y ago
I'm not sure it's so much a matter of picking negative consequences out of the blue sky of possible consequences, but rather wondering, "What happens if our software works exactly how we intend and we gain the ability to really manipulate people in a certain way?" I agree completely that it's hard or impossible to just predict in general how society will react to technology. But in many cases, there are people designing systems specifically for the purpose of getting people to behave a certain way. This implicitly acknowledges that they believe that they can affect behavior in a predictable way. If they can make predictions one way and be correct enough to profit from it, they should be capable of making predictions about things that could go wrong. I agree completely that predicting something like what Facebook will look like 10 years in the future is impossible, and there's surely a lot of research where the possible consequences are infinite. A lot of results in CS research are like someone developing a new material - "Like, sure, this could be used to make a new, extra deadly tank or something, but it could be used for literally anything." In that sense, it's not useful to make predictions, but I think there's also a lot of research going on at Facebook and Google that looks closer to what I mentioned above, where they absolutely have an intention to change people's behavior and so probably also have the ability to predict the consequences of their complete success.
- sdrinf 8y ago
  These are two different set of people: computer scientists develop algos / ml / etc, and want to publish them. UX designers , entrepreneurs, and other customer-focused people in the industry observe human behavior, and (amongst other activities) design traps around them. It is not useful to expect good predictions from the former group, and the later group isn't really interested in publishing in CS.
  - 0xdeadbeefbabe 8y ago
    The Challenger disaster is a good example of the engineer vs group dynamic https://www.npr.org/sections/thetwo-way/2016/03/21/470870426/challenger-engineer-who-warned-of-shuttle-disaster-dies https://www.npr.org/sections/thetwo-way/2016/03/21/470870426...

freeflight 8y ago

> On the other hand, the people who design Facebook or mobile games to be maximally addicting definitely understand something about how their software is affecting society. This is an interesting point, a whole lot of companies do actually hire soft-sciences talent specifically for that purpose, has been quite common in the gaming sector for years now (Blizzard, Valve, Bungie, just from the top of my head).

mywittyname 8y ago
This point backs up the statement of ~this parent~ rossdavidh's reply: programmers and engineers are woefully unequipped to perform any significant social science research. If they were, then tech companies would have little to no need to hire research psychologists or human factors specialists. They'd instead hire programmers and engineers that specialize in getting people addicted to technology.
- rosser 8y ago
  So because the employer hires specialists in weaponizing people's brains against them (via manipulating addiction psychology, &c), the programmer is somehow off the hook for implementing things that he now knows to be designed thusly, on the basis that it's the work-product of those specialists? I'm reminded of the Tom Lehrer lyric: Once the rockets are up Who cares where they come down That's not my department Says Wernher von Braun
  - mywittyname 8y ago
    You're counter-argument is disingenuous, almost to the point of willful ignorance. It's obvious that a missile can be used to send bombs across the world just as well as it can be used to send satellites into space. Minecraft seems like a great form of entertainment that can be used as an educational tool. The fact that it might also be destructive to the development of children is non-obvious and takes some expertise outside the domain of software engineering to uncover.
    - 8y ago

LeifCarrotson 8y ago

It's much difficult to differentiate between "many people will enjoy and be entertained by this game" and "people will be harmfully obsessed with this game".

ksdale 8y ago
I agree it's hard to pick which games will be which, but it's not hard to predict that the more addicting a game is, the more people will be harmfully obsessed with it.

mywittyname 8y ago

> If a person can predict the consequences of design decisions well enough to get paid for it, they'll probably have some idea of potential negative consequences as well. Being able to hypothesize about potential risks and being able to perform experiments to verify the hypothesis are totally different skills. You can guess that maybe your game will be unhealthy for teenagers, but constructing an experiment to verify this is a whole different ball game.

zdragnar 8y ago

I think you're way over-estimating what a realistic ability to predict outcomes will be. I'm reminded of back when I worked in agency land, and clients would ask us to build something to "go viral" back when going viral was a new concept. They assumed it was as easy as just saying "make it so". Of course, it's never that easy to know that it will catch on. How exactly do you make something that is fun, engaging, and rewarding enough to encourage regular use, but not fun, engaging or rewarding enough to abuse? Seriously, if you can answer that question, you're sitting on a gold mine of knowledge.