6 ms·

Minor point: While I do agree that ethics course should be a part of CS curricula, wouldn’t managers be the ones who likely pushed for the pattern you’re remark

Minor point: While I do agree that ethics course should be a part of CS curricula, wouldn’t managers be the ones who likely pushed for the pattern you’re remarking upon? (Ie ethics classes would be more useful for them?)

dogecoinbase 7y ago
The engineers would know not to do it. And if we had a decent professional organization, we'd know that we'd be supported by our fellows when we refused to perform unethical or unsafe work.
- Alissia17 7y ago
  Rate my young hot ass, pls -- https://ezo.no/kitty17 https://ezo.no/kitty17
arrrg 7y ago
Developers have a lot of power (because they do the actual work and have all the power over how something is actually implemented in the end) and share a lot of responsibility. Additionally they are often the only ones who can be aware of certain ethical issues and concerns. If they don’t bring it up others might not be able to even perceive the ethical issue because it comes down to implementation details. I would agree that that’s not really the case here (since grading Siri using people means hiring people and that’s immediately a bigger fish), but in general everyone should keep ethics on their mind. In the end I would say that everyone has to be made aware of and always keep in mind ethical issues since they can crop up everywhere, also places where a certain person has the most insight and others might overlook the ethical issue. It’s a shared responsibility. I’m sometimes shocked about how careless developers approach ethical issues and since they are the one actually writing the code they can do a lot of harm. Since we live in this capitalist hellscape all of this is unlikely to have any effect anyway …
- AstralStorm 7y ago
  Developers have no power other than quit, which is not always financially possible. And as we are well paid, there are enough replacements. But not necessarily always good job offers. (And word gets out. You can get informally blacklisted.) The only other option, still risking the job, is whistleblowing. That potentially also breaks NDA and employment contracts. We have no power over deadlines (cutting privacy features), budget, priorities or vetoing unethical ideas of product owners or marketing.
  - 9HZZRfNlpR 7y ago
    The vast majority of mass surveillance tech is programmed in Silicon Valley. These are the most privileged and intelligent programmers out there, they absolutely do have power, or at minimum a choice, to quit and get another well paying moral job.
    - AstralStorm 7y ago
      Silicon Valley, where developers live in trailer parks? (San Jose for example.) Silicon Valley, where the ethos is to get rich quick by accepting personal degradation and eschewing morality?
  - germanier 7y ago
    Developers always have to power to just not implement the feature. If you choose to work on such features, remember it's a choice you have made. You may come to the conclusion that your personal gains outweigh the damage you create to other people but it's still your responsibility. You can't delegate that away to sleep better at night.
    - AstralStorm 7y ago
      The feature will get assigned to someone else, be implemented and you will be fined in performance review if not outright fired for not doing your job. If you do other things on that project making it better, you're still complicit. Your only bet is switching projects if you want to stay at the company, which would be harder most of the time. (Because sociopaths rule.) There are more effective ways to fight it, they're not by refusing orders. You're not deciding someone's life here nor playing God and decisions get reversed with damage being mitigated if any.
pluma 7y ago
It's in mangers best business interests to ignore ethics when they stand in the way of profits. They have a massive conflict of interest. That said, of course there are business ethics courses but sadly economics are generally taught as being inherently amoral, especially in the US. Changing this would mean nothing less than abandoning laissez-faire capitalism, which would be a massive cultural shift in not just economics but (US or Western) politics as a whole. Programmers on the other hand, just like engineers or doctors or any other craftsperson, are only conflicted because ethics can get in the way of job security. If businesses were the military, managers are the officers giving orders, programmers are the soldiers pulling the trigger. If your officer tells you to do something extremely immoral, it's your ethical duty to object even if it means you will be punished. Even if it will only be recognised as the right thing to do when your officers are brought before a court by another nation. EDIT: As HN has ratelimited me again, I'll reply here instead: Yes, in the case of doctors the moral duty is in part enforced through licensing. But anyone thinking an absence of a legal duty means the ethical duties are entirely down to personal preferences seems to have a very short memory of history. There's a reason in my military example I explicitly said "even if it will only be recognised by another nation": laws are arbitrary, fundamental ethics are not. The Nuremberg trials are an oft-quoted example because it was a case of people being tried for blatantly unethical acts that were perfectly legal in their own microcosm (mostly because the people supporting those acts were also the ones writing the laws). If you're a soldier and your commanding officer orders you to perform massive human rights violations (even if your country does not consider the UN Human Rights legally binding) it's your duty to refuse, even if you suffer drastic consequences for that refusal. "Just following orders" has become a cliche for non-excuses thanks to people using exactly that justification for committing unspeakable atrocities. Also, fun fact: the German military distinguishes between binding orders, orders you can refuse and orders you must refuse. A typical example of the second category seems to be "go and wash my (private) car". An example of the third category would be a criminal act. But even despite those distinctions a soldier might be legally punished for refusing to do something unethical. Extremely immoral orders are generally also orders to do something illegal (and thus illegitimate in this case) but that mapping is neither guaranteed nor necessary. My point is that "doing the right thing" doesn't mean doing something that'll benefit you and "doing the wrong thing" doesn't mean something you'll suffer direct consequences for. Whether the consequences are legal, personal or financial (e.g. by being fired from your job), sometimes following your ethical duty comes with a cost. Nobody said any of this was fair.
IfOnlyYouKnew 7y ago
Tangentially related: The Technical University of Berlin is considered to have one of Germany's best departments of philosophy. The reason: after World War 2, the university was located in the British-controlled sector, and their administration made philosophy mandatory for all students. Because never again should young people be given the power of an engineering/science education without any regard to the ethical implications of the possibilities.
- ekianjo 7y ago
  > have one of Germany's best departments of philosophy. So what? Philosophy has barely anything to do with ethics. You can study philosophers that can argue for complete opposite sides of the same point at hand, so it does not give you any indication as to what is right or wrong. And Germany has a pretty poor record so far in terms of privacy protections/provisions, so I am not sure there is any indication that post-WW2 or post-RDA experiences have had much impact.
  - sooheon 7y ago
    An education should not hand you a canned answer to what is "right" or "wrong", it should do almost the opposite. Learning to evaluate well founded opposing arguments is certainly more useful than being force fed moralist dogma du jour.
    - ekianjo 7y ago
      You can't evaluate arguments in a vacuum. You need a value system to do that, and Philosophy doesn't give you any, it's merely the art of debating.
      - sooheon 7y ago
        Calling it "merely the art of debating" implies it lacks actual substance. I think philosophy concerns itself with what you debate about, not how you do it. Besides, nothing outside of dogma can "give you" a value system, but philosophy does present candidates. It's up to you to choose your own, the hope being that this is better than unquestioned inheritance of values.
chongli 7y ago
wouldn’t managers be the ones who likely pushed for the pattern you’re remarking upon? A friend of mine is currently studying to be a civil engineer. As part of his program, he's required to take multiple courses in engineering ethics and business ethics, as well as economics and law. He's relayed to me numerous anecdotes from lectures all about engineers who caved to pressure from non-engineer business and political stakeholders. When the bridge collapsed or the building fell over, it was the engineers who were to blame. Again and again it was drilled into the class that you, as an engineer, are the last word when it comes to defending sound engineering principles. They went on to learn about how this is the basis for the structure of engineering firms and that it's not allowed for civil engineers to work in a subordinate role at a company run by non-engineers, in order to prevent a non-engineer from ordering an employee to affix their seal to a building plan. Perhaps we need something similar for software developers (seems a little absurd to call them engineers in light of the above, doesn't it?) I don't see how we'd enforce it though, since any kid can take a laptop into their bedroom and write a program with potentially millions of users.

dogecoinbase 7y ago

The engineers would know not to do it. And if we had a decent professional organization, we'd know that we'd be supported by our fellows when we refused to perform unethical or unsafe work.

Alissia17 7y ago
Rate my young hot ass, pls -- https://ezo.no/kitty17 https://ezo.no/kitty17

arrrg 7y ago

Developers have a lot of power (because they do the actual work and have all the power over how something is actually implemented in the end) and share a lot of responsibility. Additionally they are often the only ones who can be aware of certain ethical issues and concerns. If they don’t bring it up others might not be able to even perceive the ethical issue because it comes down to implementation details. I would agree that that’s not really the case here (since grading Siri using people means hiring people and that’s immediately a bigger fish), but in general everyone should keep ethics on their mind. In the end I would say that everyone has to be made aware of and always keep in mind ethical issues since they can crop up everywhere, also places where a certain person has the most insight and others might overlook the ethical issue. It’s a shared responsibility. I’m sometimes shocked about how careless developers approach ethical issues and since they are the one actually writing the code they can do a lot of harm. Since we live in this capitalist hellscape all of this is unlikely to have any effect anyway …

AstralStorm 7y ago
Developers have no power other than quit, which is not always financially possible. And as we are well paid, there are enough replacements. But not necessarily always good job offers. (And word gets out. You can get informally blacklisted.) The only other option, still risking the job, is whistleblowing. That potentially also breaks NDA and employment contracts. We have no power over deadlines (cutting privacy features), budget, priorities or vetoing unethical ideas of product owners or marketing.
- 9HZZRfNlpR 7y ago
  The vast majority of mass surveillance tech is programmed in Silicon Valley. These are the most privileged and intelligent programmers out there, they absolutely do have power, or at minimum a choice, to quit and get another well paying moral job.
  - AstralStorm 7y ago
    Silicon Valley, where developers live in trailer parks? (San Jose for example.) Silicon Valley, where the ethos is to get rich quick by accepting personal degradation and eschewing morality?
- germanier 7y ago
  Developers always have to power to just not implement the feature. If you choose to work on such features, remember it's a choice you have made. You may come to the conclusion that your personal gains outweigh the damage you create to other people but it's still your responsibility. You can't delegate that away to sleep better at night.
  - AstralStorm 7y ago
    The feature will get assigned to someone else, be implemented and you will be fined in performance review if not outright fired for not doing your job. If you do other things on that project making it better, you're still complicit. Your only bet is switching projects if you want to stay at the company, which would be harder most of the time. (Because sociopaths rule.) There are more effective ways to fight it, they're not by refusing orders. You're not deciding someone's life here nor playing God and decisions get reversed with damage being mitigated if any.

pluma 7y ago

It's in mangers best business interests to ignore ethics when they stand in the way of profits. They have a massive conflict of interest. That said, of course there are business ethics courses but sadly economics are generally taught as being inherently amoral, especially in the US. Changing this would mean nothing less than abandoning laissez-faire capitalism, which would be a massive cultural shift in not just economics but (US or Western) politics as a whole. Programmers on the other hand, just like engineers or doctors or any other craftsperson, are only conflicted because ethics can get in the way of job security. If businesses were the military, managers are the officers giving orders, programmers are the soldiers pulling the trigger. If your officer tells you to do something extremely immoral, it's your ethical duty to object even if it means you will be punished. Even if it will only be recognised as the right thing to do when your officers are brought before a court by another nation. EDIT: As HN has ratelimited me again, I'll reply here instead: Yes, in the case of doctors the moral duty is in part enforced through licensing. But anyone thinking an absence of a legal duty means the ethical duties are entirely down to personal preferences seems to have a very short memory of history. There's a reason in my military example I explicitly said "even if it will only be recognised by another nation": laws are arbitrary, fundamental ethics are not. The Nuremberg trials are an oft-quoted example because it was a case of people being tried for blatantly unethical acts that were perfectly legal in their own microcosm (mostly because the people supporting those acts were also the ones writing the laws). If you're a soldier and your commanding officer orders you to perform massive human rights violations (even if your country does not consider the UN Human Rights legally binding) it's your duty to refuse, even if you suffer drastic consequences for that refusal. "Just following orders" has become a cliche for non-excuses thanks to people using exactly that justification for committing unspeakable atrocities. Also, fun fact: the German military distinguishes between binding orders, orders you can refuse and orders you must refuse. A typical example of the second category seems to be "go and wash my (private) car". An example of the third category would be a criminal act. But even despite those distinctions a soldier might be legally punished for refusing to do something unethical. Extremely immoral orders are generally also orders to do something illegal (and thus illegitimate in this case) but that mapping is neither guaranteed nor necessary. My point is that "doing the right thing" doesn't mean doing something that'll benefit you and "doing the wrong thing" doesn't mean something you'll suffer direct consequences for. Whether the consequences are legal, personal or financial (e.g. by being fired from your job), sometimes following your ethical duty comes with a cost. Nobody said any of this was fair.

IfOnlyYouKnew 7y ago

Tangentially related: The Technical University of Berlin is considered to have one of Germany's best departments of philosophy. The reason: after World War 2, the university was located in the British-controlled sector, and their administration made philosophy mandatory for all students. Because never again should young people be given the power of an engineering/science education without any regard to the ethical implications of the possibilities.

ekianjo 7y ago
> have one of Germany's best departments of philosophy. So what? Philosophy has barely anything to do with ethics. You can study philosophers that can argue for complete opposite sides of the same point at hand, so it does not give you any indication as to what is right or wrong. And Germany has a pretty poor record so far in terms of privacy protections/provisions, so I am not sure there is any indication that post-WW2 or post-RDA experiences have had much impact.
- sooheon 7y ago
  An education should not hand you a canned answer to what is "right" or "wrong", it should do almost the opposite. Learning to evaluate well founded opposing arguments is certainly more useful than being force fed moralist dogma du jour.
  - ekianjo 7y ago
    You can't evaluate arguments in a vacuum. You need a value system to do that, and Philosophy doesn't give you any, it's merely the art of debating.
    - sooheon 7y ago
      Calling it "merely the art of debating" implies it lacks actual substance. I think philosophy concerns itself with what you debate about, not how you do it. Besides, nothing outside of dogma can "give you" a value system, but philosophy does present candidates. It's up to you to choose your own, the hope being that this is better than unquestioned inheritance of values.

chongli 7y ago

wouldn’t managers be the ones who likely pushed for the pattern you’re remarking upon? A friend of mine is currently studying to be a civil engineer. As part of his program, he's required to take multiple courses in engineering ethics and business ethics, as well as economics and law. He's relayed to me numerous anecdotes from lectures all about engineers who caved to pressure from non-engineer business and political stakeholders. When the bridge collapsed or the building fell over, it was the engineers who were to blame. Again and again it was drilled into the class that you, as an engineer, are the last word when it comes to defending sound engineering principles. They went on to learn about how this is the basis for the structure of engineering firms and that it's not allowed for civil engineers to work in a subordinate role at a company run by non-engineers, in order to prevent a non-engineer from ordering an employee to affix their seal to a building plan. Perhaps we need something similar for software developers (seems a little absurd to call them engineers in light of the above, doesn't it?) I don't see how we'd enforce it though, since any kid can take a laptop into their bedroom and write a program with potentially millions of users.