3 ms·

> why technologists don't think about the consequences of what they do before they do it? They have bills to pay, and people are willing to pay for that. Techn

by pyrale 3y ago

> why technologists don't think about the consequences of what they do before they do it? They have bills to pay, and people are willing to pay for that. Technologists with strongly held opinions also exist, but they've been marginalized in every possible area where money gets you leverage. Just look at how open-source became a corporate breeding ground. When you have a question like "Do N people have no ethics?", the answer is very often: some of N do, some of N don't. And the decisions to promote N1 and sack N2 are rarely made within the N group

Eddy_Viscosity2 3y ago
> why technologists don't think about the consequences of what they do before they do it? Imagine if they had opinions held so strongly they'd be willing to camp out in the CEOs office in protest. I'm sure that the executives would re-examine their positions on the subject and work with the technologists to address their concerns.
- palata 3y ago
  I will assume this was sarcasm. So think about this: say your CEO asks you to make a missile specifically to target and kill children. Would you consider that it is not your job to question it, or would you actually think about it? I will assume that you would at least think about it. You would have some choices, then, for example: - Accept it and keep doing your job - Raise concerns internally (to your CEO) or externally (to the media), more or less vehemently - Quit and go do something else And I assume that for such an extreme case, all of this seems absolutely obvious to you. So let me ask: why does it feel like you can't imagine questioning something less extreme, like working on LLMs or autonomous cars or a connected fridge? Engineers are paid to think (among others of course, but it is usually a necessary skill). It feels like they could reuse that skill and sometimes think about ethics, too.
  - pyrale 3y ago
    > So think about this: say your CEO asks you to make a missile specifically to target and kill children. Well, I wouldn't and I would get fired, and the CEO would find someone with less scruples. And you would say "Damn, these technologists have no mercy" even if 99 engineers have refused before the one that accepted. You're simply falling for survivorship bias.
    - palata 3y ago
      > You're simply falling for survivorship bias. What if I told you that I worked in fields that raise such ethical questions, and I have seen with my own eyes how there was never a single instance with 99 engineers who refuse for 1 who accepts? For what I have seen, there was no survivorship bias. And then regarding LLMs in particular, I struggle to find people around me who actually question the ethical problems they bring. They just don't seem to even imagine that they could think about ethics. So unless you don't ask them to do something that is extreme (design a weapon to kill children), they just don't question anything at all.
  - Eddy_Viscosity2 3y ago
    It was sarcasm yes, but engineers are not paid to think. They are paid to do work (which may involve thinking) toward a given goal decided by the executives. People are regularly faced with such situations (with both small and large stakes) and all three of those choices are taken by different people. We know that 1. happens because controversial stuff still gets made. 2. happens because we hear about it in the media. And 3. I assume happens as well. But in the end the company can always find new people that choose 1. "Do it or I'll find someone who will." is both true and powerful. If you quit you lose your livelihood and potentially could be blacklisted from the industry, and yet the bad thing still happens anyway.
    - atmavatar 3y ago
      > "Do it or I'll find someone who will." is both true and powerful. If you quit you lose your livelihood and potentially could be blacklisted from the industry, and yet the bad thing still happens anyway. That's merely a rationalization to remain a member of group 1. The truth is that 2 and 3 are not mutually exclusive, and together, they can and do occasionally prevent "bad thing" from happening. In some cases, it's because group 1 is small enough that the company cannot hire a toady to do "bad thing" in time to retain their contract for doing so. It may also be because the combination of external publicity coupled with a high-profile resignation is enough of a PR hit that the company reconsiders. There's another element to consider: other, unrelated teams within the company who catch wind of "bad thing" can also opt for 2 and/or 3, further disincentivizing the company and perhaps even its competitors from proceeding, as losing a leading team in a highly-contested market segment could mean ceding significant ground to a competitor. Granted, such moral stands don't affect change as often as one might like, but parroting "bad thing will always happen regardless what you try" everywhere certainly doesn't help things.
    - red-iron-pine 3y ago
      no engineers are paid to think. that's why they're not technicians. and it's why they usually have to have a decent amount of education -- which usually includes ethics -- and in many cases (like civil or mechanical engineers) have to have a formal certification from a governing body. programming doesn't have that, but probably should. regardless, it's a knowledge field, not a fork-and-spoon operator gig. > "Do it or I'll find someone who will." is both true and powerful. it's only powerful when there are high salaries attached. most people would swallow their objections for a steady 300k/yr

by pyrale 3y ago

Eddy_Viscosity2 3y ago

> why technologists don't think about the consequences of what they do before they do it? Imagine if they had opinions held so strongly they'd be willing to camp out in the CEOs office in protest. I'm sure that the executives would re-examine their positions on the subject and work with the technologists to address their concerns.

palata 3y ago
I will assume this was sarcasm. So think about this: say your CEO asks you to make a missile specifically to target and kill children. Would you consider that it is not your job to question it, or would you actually think about it? I will assume that you would at least think about it. You would have some choices, then, for example: - Accept it and keep doing your job - Raise concerns internally (to your CEO) or externally (to the media), more or less vehemently - Quit and go do something else And I assume that for such an extreme case, all of this seems absolutely obvious to you. So let me ask: why does it feel like you can't imagine questioning something less extreme, like working on LLMs or autonomous cars or a connected fridge? Engineers are paid to think (among others of course, but it is usually a necessary skill). It feels like they could reuse that skill and sometimes think about ethics, too.
- pyrale 3y ago
  > So think about this: say your CEO asks you to make a missile specifically to target and kill children. Well, I wouldn't and I would get fired, and the CEO would find someone with less scruples. And you would say "Damn, these technologists have no mercy" even if 99 engineers have refused before the one that accepted. You're simply falling for survivorship bias.
  - palata 3y ago
    > You're simply falling for survivorship bias. What if I told you that I worked in fields that raise such ethical questions, and I have seen with my own eyes how there was never a single instance with 99 engineers who refuse for 1 who accepts? For what I have seen, there was no survivorship bias. And then regarding LLMs in particular, I struggle to find people around me who actually question the ethical problems they bring. They just don't seem to even imagine that they could think about ethics. So unless you don't ask them to do something that is extreme (design a weapon to kill children), they just don't question anything at all.
- Eddy_Viscosity2 3y ago
  It was sarcasm yes, but engineers are not paid to think. They are paid to do work (which may involve thinking) toward a given goal decided by the executives. People are regularly faced with such situations (with both small and large stakes) and all three of those choices are taken by different people. We know that 1. happens because controversial stuff still gets made. 2. happens because we hear about it in the media. And 3. I assume happens as well. But in the end the company can always find new people that choose 1. "Do it or I'll find someone who will." is both true and powerful. If you quit you lose your livelihood and potentially could be blacklisted from the industry, and yet the bad thing still happens anyway.
  - atmavatar 3y ago
    > "Do it or I'll find someone who will." is both true and powerful. If you quit you lose your livelihood and potentially could be blacklisted from the industry, and yet the bad thing still happens anyway. That's merely a rationalization to remain a member of group 1. The truth is that 2 and 3 are not mutually exclusive, and together, they can and do occasionally prevent "bad thing" from happening. In some cases, it's because group 1 is small enough that the company cannot hire a toady to do "bad thing" in time to retain their contract for doing so. It may also be because the combination of external publicity coupled with a high-profile resignation is enough of a PR hit that the company reconsiders. There's another element to consider: other, unrelated teams within the company who catch wind of "bad thing" can also opt for 2 and/or 3, further disincentivizing the company and perhaps even its competitors from proceeding, as losing a leading team in a highly-contested market segment could mean ceding significant ground to a competitor. Granted, such moral stands don't affect change as often as one might like, but parroting "bad thing will always happen regardless what you try" everywhere certainly doesn't help things.
  - red-iron-pine 3y ago
    no engineers are paid to think. that's why they're not technicians. and it's why they usually have to have a decent amount of education -- which usually includes ethics -- and in many cases (like civil or mechanical engineers) have to have a formal certification from a governing body. programming doesn't have that, but probably should. regardless, it's a knowledge field, not a fork-and-spoon operator gig. > "Do it or I'll find someone who will." is both true and powerful. it's only powerful when there are high salaries attached. most people would swallow their objections for a steady 300k/yr