2 ms·

I think a lot of people are missing the point of this post, especially because it offers an opportunity to nit pick on specifics. But the point here isn't "don'

I think a lot of people are missing the point of this post, especially because it offers an opportunity to nit pick on specifics. But the point here isn't "don't write code for the military" or "don't write code for killing" or anything else, it's that we need to think about if we're okay with the code we're writing. We need to think about the implications of the things we build. We're never going to find a hard line we all can agree on, aka "don't work on missiles, but airplanes are okay" or something similar. But we can all stop to think about what that line is for us personally, and if what we're working on crosses it. What would we do if asked to violate that boundary we've created for ourselves? When I was in grad school, we had to take an ethics in engineering course. We split into groups and discussed the ethics of building certain things, in my group's case building predator drones. While I didn't agree with the person who said they were okay building them because we need weapons, I was far less concerned with that individual than the one who looked at me, straight faced, and said "I don't get what airplanes have to do with morals".

CamperBob2 8y ago
I was far less concerned with that individual than the one who looked at me, straight faced, and said "I don't get what airplanes have to do with morals". They're arguing that morality attaches to the user, not the tool. See any gun-control argument in history, basically.
- sullyj3 8y ago
  Let's accept for the moment that tools are morally neutral. Notwithstanding that, when you're thinking in a consequentialist fashion, you don't just ignore the knowledge you have of how the tools you build are going to be used. You can say "well it's not up to me, it's not my responsibility to think about how they'll be used", but you have to be upfront with yourself about the fact that you're not taking moral responsibility for your actions.
  - CamperBob2 8y ago
    but you have to be upfront with yourself about the fact that you're not taking moral responsibility for your actions. My concern isn't so much the nature or purpose of the tool I'm building, as it is with whose hands the tool will ultimately end up in. I'd be OK working on military drones, for instance, if I knew they wouldn't end up in the hands of avaricious neoconservatives, LBJ-school paleoliberals, religious nuts, or basically anyone who isn't someone whose judgment I personally trust. So in practice I wouldn't want to work on a military weapons project with no valid civilian uses. But it's not because I believe drones or land mines or atomic bombs are inherently evil. It's because I have no idea who will eventually end up in control of those products, and because history suggests that they will be used in ways I don't approve of. I have the same attitude towards patents, basically, but that's a different debate.
cmroanirgo 8y ago
> it's that we need to think about if we're okay with the code we're writing I don't disagree, but feel that it's rather difficult to actually discern this. For me, the stand-out problem that most coders face is that of compartmentalization. That is, you are a small part of a bigger churning wheel. You are told to do something, so you do it. Unless you are working completely for yourself, or a small start up with specific goals in mind, you will not have the oversight necessary to actually objectively determine whether your code is going to be used for the greater good or not. The majority of bigger commercial ventures (whether it be defense related or fb) are heavily comparmentalized. In my experience, the more deliberate and the more legalistic the compartmentalization is, the more intrinsically morally corrupt (or even illegal) it possibly will end up being. For instance, look at Google's eroding "don't be evil" construct [0], or FB's experiments on mass psychology [1] This is why I left the corporate world. Too often, my moral compass was being desensitized by sociopaths. [0] https://www.businessinsider.com.au/google-downgrades-dont-be-evil-2018-5?r=US&IR=T https://www.businessinsider.com.au/google-downgrades-dont-be... [1] https://www.theguardian.com/technology/2014/jul/02/facebook-apologises-psychological-experiments-on-users https://www.theguardian.com/technology/2014/jul/02/facebook-...

CamperBob2 8y ago

I was far less concerned with that individual than the one who looked at me, straight faced, and said "I don't get what airplanes have to do with morals". They're arguing that morality attaches to the user, not the tool. See any gun-control argument in history, basically.

sullyj3 8y ago
Let's accept for the moment that tools are morally neutral. Notwithstanding that, when you're thinking in a consequentialist fashion, you don't just ignore the knowledge you have of how the tools you build are going to be used. You can say "well it's not up to me, it's not my responsibility to think about how they'll be used", but you have to be upfront with yourself about the fact that you're not taking moral responsibility for your actions.
- CamperBob2 8y ago
  but you have to be upfront with yourself about the fact that you're not taking moral responsibility for your actions. My concern isn't so much the nature or purpose of the tool I'm building, as it is with whose hands the tool will ultimately end up in. I'd be OK working on military drones, for instance, if I knew they wouldn't end up in the hands of avaricious neoconservatives, LBJ-school paleoliberals, religious nuts, or basically anyone who isn't someone whose judgment I personally trust. So in practice I wouldn't want to work on a military weapons project with no valid civilian uses. But it's not because I believe drones or land mines or atomic bombs are inherently evil. It's because I have no idea who will eventually end up in control of those products, and because history suggests that they will be used in ways I don't approve of. I have the same attitude towards patents, basically, but that's a different debate.

cmroanirgo 8y ago

> it's that we need to think about if we're okay with the code we're writing I don't disagree, but feel that it's rather difficult to actually discern this. For me, the stand-out problem that most coders face is that of compartmentalization. That is, you are a small part of a bigger churning wheel. You are told to do something, so you do it. Unless you are working completely for yourself, or a small start up with specific goals in mind, you will not have the oversight necessary to actually objectively determine whether your code is going to be used for the greater good or not. The majority of bigger commercial ventures (whether it be defense related or fb) are heavily comparmentalized. In my experience, the more deliberate and the more legalistic the compartmentalization is, the more intrinsically morally corrupt (or even illegal) it possibly will end up being. For instance, look at Google's eroding "don't be evil" construct [0], or FB's experiments on mass psychology [1] This is why I left the corporate world. Too often, my moral compass was being desensitized by sociopaths. [0] https://www.businessinsider.com.au/google-downgrades-dont-be-evil-2018-5?r=US&IR=T https://www.businessinsider.com.au/google-downgrades-dont-be... [1] https://www.theguardian.com/technology/2014/jul/02/facebook-apologises-psychological-experiments-on-users https://www.theguardian.com/technology/2014/jul/02/facebook-...