4 ms·

In the vein of Kumail's critique, if it's useful to anyone, I wrote some thoughts about software engineering and ethics from my own experiences: A serial tech

by nemild 9y ago

In the vein of Kumail's critique, if it's useful to anyone, I wrote some thoughts about software engineering and ethics from my own experiences: A serial tech entrepreneur in Silicon Valley once asked me to design a “social stockade” for his financial services customers. It would lock people out of their social media accounts and tweet out/FB share to their friends when they hadn’t paid a loan. He pitched it to prospective employees as meaningful work that would reduce the cost of loans for the needy. I was horrified that his product was being built and that many others would likely take the role I was turning down. And he was hardly the first to pitch his “innovation” as providing only good. Every software engineer I’ve worked with has had a strong sense of personal values and ethics, but the organizations we work for can take actions that are at odds with these. I’d like to highlight a few of the key challenges you’ll face and provide feedback for living your personal values. Most importantly, it’s critical that you think about the impact of your work and consciously set your personal values in advance of inevitable future challenges. (Always available to discuss if valuable to anyone, feel free to email me encrypted messages, see my HN profile) https://www.nemil.com/musings/software-engineers-and-ethics.html https://www.nemil.com/musings/software-engineers-and-ethics....

tomjen3 9y ago
I am curious as to what your issue is with the loan setup. Inn my mind out is an alternative to a traditional loan, how is that bad?
- nemild 9y ago
  It felt like a modern day scarlet letter, instead of a innovative new way to deliver cheaper loans. There are other techniques to deliver cheaper loans, which don't lead to social ostracism (and which don't violate any social network terms of services).
- stale2002 9y ago
  It is bad for the same reason that 100% interest loans are bad. They are predatory on poor people, and they trick people into accepting something when they don't really understand the consequences or "believe" them. A person may incorrectly accept a loan like this, while being under the mistaken assumption that they aren't at risk of the consequences. Sometimes people just make bad decisions. It is really immoral to take advantage of someone like that.
  - CookieMon 9y ago
    Social connections (social capital?) is also how poor people survive everyday setbacks that others would pay their way out of, a “social stockade” would be debilitating. Debilitating your debtor IMHO seems counterproductive for getting a loan repaid. But is the harm it would cause even an unforeseen coincidence - the guy might have heard some of the studies on how crucial social connections are in poverty and gone "aha! Here's something that truly matters to them, it could be wielded as a cheap big stick in loan repayments"
- sharemywin 9y ago
  because your tricking poor investors into thinking the returns would be better. "Most" people that don't pay usually don't pay because they can't pay. And people that don't live up to their obligations and can pay will probably sue you for your predatory lending and cruel loan terms.
aphextron 9y ago
>Every software engineer I’ve worked with has had a strong sense of personal values and ethics, but the organizations we work for can take actions that are at odds with these. I’d like to highlight a few of the key challenges you’ll face and provide feedback for living your personal values. Most importantly, it’s critical that you think about the impact of your work and consciously set your personal values in advance of inevitable future challenges. I'd agree with this assessment of developers. However I've come into these exact moral dilemmas myself in the line of working for various companies doing web programming. How did I resolve them? By making the decision that affected my own bottom line favorably. When looked at in isolation, it's impossible for a single developer to say no to the incentives of doing "evil" work. It won't be until the field of software engineering really matures and develops a set of standards bodies like other professions that engineers will have some sort of protection. Physicians can refuse to perform a procedure they're told to in the name of protecting their medical license. A software engineer should likewise be held to the same standard.
- SomeStupidPoint 9y ago
  Nuclear missiles, land mines, biological weapons, torture research, etc has all been carried out by willing and often motivated engineers or doctors. I'm skeptical something that hasn't worked in other fields is going to fix our woes in software.
  - goatlover 9y ago
    Probably because it was interesting and cutting edge, and so they rationalized it to themselves, like humans are so good at doing.
- bendoernberg 9y ago
  It is possible to say no to the incentives of doing evil work; I would describe those who say no as the people with "a strong sense of personal values and ethics". This phrase has no meaning if you apply it to people who routinely violate their own beliefs in order to make more money, especially in a highly-compensated profession with a booming job market and plenty of available positions that don't require unethical behavior.
mmerlin 9y ago
This sounds like the new govt policy being introduced in China to publicly shame loan defaulters http://fortune.com/2017/10/11/china-debt-evaders-loans-blacklist/ http://fortune.com/2017/10/11/china-debt-evaders-loans-black...
finnthehuman 9y ago
Every software engineer I’ve worked with has had a strong sense of personal values and ethics, but the organizations we work for can take actions that are at odds with these. Do they though? I've noticed a trend where the strength of some developers' beliefs are proportional to how unethical the products they're paid to build are. This makes me suspect that for many people, the ethical stance they take is a coping mechanism for excusing their own actions with the Yuppie Nuremberg Defense.

by nemild 9y ago

tomjen3 9y ago

I am curious as to what your issue is with the loan setup. Inn my mind out is an alternative to a traditional loan, how is that bad?

nemild 9y ago
It felt like a modern day scarlet letter, instead of a innovative new way to deliver cheaper loans. There are other techniques to deliver cheaper loans, which don't lead to social ostracism (and which don't violate any social network terms of services).
stale2002 9y ago
It is bad for the same reason that 100% interest loans are bad. They are predatory on poor people, and they trick people into accepting something when they don't really understand the consequences or "believe" them. A person may incorrectly accept a loan like this, while being under the mistaken assumption that they aren't at risk of the consequences. Sometimes people just make bad decisions. It is really immoral to take advantage of someone like that.
- CookieMon 9y ago
  Social connections (social capital?) is also how poor people survive everyday setbacks that others would pay their way out of, a “social stockade” would be debilitating. Debilitating your debtor IMHO seems counterproductive for getting a loan repaid. But is the harm it would cause even an unforeseen coincidence - the guy might have heard some of the studies on how crucial social connections are in poverty and gone "aha! Here's something that truly matters to them, it could be wielded as a cheap big stick in loan repayments"
sharemywin 9y ago
because your tricking poor investors into thinking the returns would be better. "Most" people that don't pay usually don't pay because they can't pay. And people that don't live up to their obligations and can pay will probably sue you for your predatory lending and cruel loan terms.

aphextron 9y ago

>Every software engineer I’ve worked with has had a strong sense of personal values and ethics, but the organizations we work for can take actions that are at odds with these. I’d like to highlight a few of the key challenges you’ll face and provide feedback for living your personal values. Most importantly, it’s critical that you think about the impact of your work and consciously set your personal values in advance of inevitable future challenges. I'd agree with this assessment of developers. However I've come into these exact moral dilemmas myself in the line of working for various companies doing web programming. How did I resolve them? By making the decision that affected my own bottom line favorably. When looked at in isolation, it's impossible for a single developer to say no to the incentives of doing "evil" work. It won't be until the field of software engineering really matures and develops a set of standards bodies like other professions that engineers will have some sort of protection. Physicians can refuse to perform a procedure they're told to in the name of protecting their medical license. A software engineer should likewise be held to the same standard.

SomeStupidPoint 9y ago
Nuclear missiles, land mines, biological weapons, torture research, etc has all been carried out by willing and often motivated engineers or doctors. I'm skeptical something that hasn't worked in other fields is going to fix our woes in software.
- goatlover 9y ago
  Probably because it was interesting and cutting edge, and so they rationalized it to themselves, like humans are so good at doing.
bendoernberg 9y ago
It is possible to say no to the incentives of doing evil work; I would describe those who say no as the people with "a strong sense of personal values and ethics". This phrase has no meaning if you apply it to people who routinely violate their own beliefs in order to make more money, especially in a highly-compensated profession with a booming job market and plenty of available positions that don't require unethical behavior.

mmerlin 9y ago

This sounds like the new govt policy being introduced in China to publicly shame loan defaulters http://fortune.com/2017/10/11/china-debt-evaders-loans-blacklist/ http://fortune.com/2017/10/11/china-debt-evaders-loans-black...

finnthehuman 9y ago

Every software engineer I’ve worked with has had a strong sense of personal values and ethics, but the organizations we work for can take actions that are at odds with these. Do they though? I've noticed a trend where the strength of some developers' beliefs are proportional to how unethical the products they're paid to build are. This makes me suspect that for many people, the ethical stance they take is a coping mechanism for excusing their own actions with the Yuppie Nuremberg Defense.