5 ms·

> This really has to stop. "We", the totally homogeneous group of software professionals could make this stop. "We" don't.

by junga 2y ago

ryandrake 2y ago
Yea, every time I read one of these articles, I can’t help but think: “A software engineer sat down and wrote this remote kill switch." We, as a profession are responsible for this shit, or at the very least, complicit. Regulation is one thing, but also, software engineering as a profession is in dire need of ethical standards. Just because we can code something doesn’t mean we should.
- zahlman 2y ago
  I remember one of Asimov's stories involved a human defeating the Laws of Robotics by distributing work among multiple robots with imperfect information. I wonder if something analogous doesn't happen with software engineers nowadays. When it comes to something like a "remote kill switch" for software, it's hard to imagine any alternate beneficial use. But generally I assign the blame to the users of software who put it to a malicious use, not to authors.
  - immibis 2y ago
    It's become quite normal in malicious organizations of all types, such as most large businesses and governments.
  - akoboldfrying 2y ago
    >When it comes to something like a "remote kill switch" for software, it's hard to imagine any alternate beneficial use. The obvious alternate beneficial use is the ability to immediately disable the hardware in case a serious safety issue (the kind that triggers product recall) is discovered.
  - immibis 2y ago
    There was an anecdote shared on HN of someone claiming to be ethically employed by the military to develop a system for planes to automatically land on unmarked runways without engine power. They didn't tell her the planes were elongated spherical and filled with powerful explosives and the runways weren't flat - at least not before the plane landed on them.
- GauntletWizard 2y ago
  One of the great and terrible things about the software industry is that there's no certifying body, no professional ethics code to sign and adhere to, no government regulation around how you can sell your services. This is one of the best parts: many software people have gotten in through circuitous routes, have no formal training, and have done great things despite that. On the other hand, because of that, we don't have any consensus and ability to shun or disposess companies that act unethically. Quite frankly, I don't think any board of ethics would step in here. I don't see anything in the IEEE code of ethics that would be clear here. I don't think that professional licensing or better professional organizations are the way to stop this behavior.
  - ocschwar 2y ago
    I think disabling the firmware in circumstances that are clearly defined but not disclosed to the customer is very much outside existing IEEE ethical rules. And making a Professional Engineer sign on to the software release before the release would be a good way to prevent shit like this.
- Teever 2y ago
  To have what you're asking for requires transparency. It wasn't just a faceless and nameless software engineer it was a real human being with a name. Until it is mandated that public infrastructure is developed in the open so we know precisely who attempts to add features to render a product defective by design we will not be able to fix this.
immibis 2y ago
I am physically there at C3 right now and one of the prevalent themes this year is "being nice didn't work". You can see it in this year's tagline: "illegal instructions"
teeray 2y ago
“We” is an amorphous blob, not a guild. The company will just shop the job around until they find someone to do it.
jasdi 2y ago
If you want to have a say in how software works, then you have to control how companies run. If Technical folk are not on the Boards or have controlling share in an org, or don't know how to get into such positions then they have very little to no say in how anything works. There are countless examples were technical people object and get replaced, sidelined or fired, cuz they are totally unprepared in how to win such age old political and financial fights. If Oppenheimer, Engelbart and the Google brainiacs who protested recently got pushed aside, then its beyond obvious how the story will end for anyone else. The lesson from history for anyone serious about this stuff is - develop business+finance acumen, or develop alliances with business+finance power.
- ratmeadow 2y ago
  You speak like working at a company exists in a vacuum. It doesn't. You are (generally, in Western society) free to stop working for that company. If everyone refuses to write the code, then the company won't have that code. Easily said than done, of course, because life has many pressures. But I dislike this "oh can't do anything about it individually, let's give up and double down on unethical behaviour" attitude (meanwhile we are conveniently paid a high salary for it). We always have a choice.
  - philosopher1234 2y ago
    Whose interests does it serve to place the blame on individual employees?
    - DirkH 2y ago
      Sounds like a cop out response. Placing blame on individual employees is not mutually exclusive with placing blame on a company's board.
      - philosopher1234 2y ago
        You dont have to either blame one set of people or another. You could put the blame in the right place instead, the design of the system which all but dictates the individuals choices.

> This really has to stop. "We", the totally homogeneous group of software professionals could make this stop. "We" don't.

by junga 2y ago

> This really has to stop. "We", the totally homogeneous group of software professionals could make this stop. "We" don't.

ryandrake 2y ago

Yea, every time I read one of these articles, I can’t help but think: “A software engineer sat down and wrote this remote kill switch." We, as a profession are responsible for this shit, or at the very least, complicit. Regulation is one thing, but also, software engineering as a profession is in dire need of ethical standards. Just because we can code something doesn’t mean we should.

zahlman 2y ago
I remember one of Asimov's stories involved a human defeating the Laws of Robotics by distributing work among multiple robots with imperfect information. I wonder if something analogous doesn't happen with software engineers nowadays. When it comes to something like a "remote kill switch" for software, it's hard to imagine any alternate beneficial use. But generally I assign the blame to the users of software who put it to a malicious use, not to authors.
- immibis 2y ago
  It's become quite normal in malicious organizations of all types, such as most large businesses and governments.
- akoboldfrying 2y ago
  >When it comes to something like a "remote kill switch" for software, it's hard to imagine any alternate beneficial use. The obvious alternate beneficial use is the ability to immediately disable the hardware in case a serious safety issue (the kind that triggers product recall) is discovered.
- immibis 2y ago
  There was an anecdote shared on HN of someone claiming to be ethically employed by the military to develop a system for planes to automatically land on unmarked runways without engine power. They didn't tell her the planes were elongated spherical and filled with powerful explosives and the runways weren't flat - at least not before the plane landed on them.
GauntletWizard 2y ago
One of the great and terrible things about the software industry is that there's no certifying body, no professional ethics code to sign and adhere to, no government regulation around how you can sell your services. This is one of the best parts: many software people have gotten in through circuitous routes, have no formal training, and have done great things despite that. On the other hand, because of that, we don't have any consensus and ability to shun or disposess companies that act unethically. Quite frankly, I don't think any board of ethics would step in here. I don't see anything in the IEEE code of ethics that would be clear here. I don't think that professional licensing or better professional organizations are the way to stop this behavior.
- ocschwar 2y ago
  I think disabling the firmware in circumstances that are clearly defined but not disclosed to the customer is very much outside existing IEEE ethical rules. And making a Professional Engineer sign on to the software release before the release would be a good way to prevent shit like this.
Teever 2y ago
To have what you're asking for requires transparency. It wasn't just a faceless and nameless software engineer it was a real human being with a name. Until it is mandated that public infrastructure is developed in the open so we know precisely who attempts to add features to render a product defective by design we will not be able to fix this.

immibis 2y ago

I am physically there at C3 right now and one of the prevalent themes this year is "being nice didn't work". You can see it in this year's tagline: "illegal instructions"

teeray 2y ago

“We” is an amorphous blob, not a guild. The company will just shop the job around until they find someone to do it.

jasdi 2y ago

If you want to have a say in how software works, then you have to control how companies run. If Technical folk are not on the Boards or have controlling share in an org, or don't know how to get into such positions then they have very little to no say in how anything works. There are countless examples were technical people object and get replaced, sidelined or fired, cuz they are totally unprepared in how to win such age old political and financial fights. If Oppenheimer, Engelbart and the Google brainiacs who protested recently got pushed aside, then its beyond obvious how the story will end for anyone else. The lesson from history for anyone serious about this stuff is - develop business+finance acumen, or develop alliances with business+finance power.

ratmeadow 2y ago
You speak like working at a company exists in a vacuum. It doesn't. You are (generally, in Western society) free to stop working for that company. If everyone refuses to write the code, then the company won't have that code. Easily said than done, of course, because life has many pressures. But I dislike this "oh can't do anything about it individually, let's give up and double down on unethical behaviour" attitude (meanwhile we are conveniently paid a high salary for it). We always have a choice.
- philosopher1234 2y ago
  Whose interests does it serve to place the blame on individual employees?
  - DirkH 2y ago
    Sounds like a cop out response. Placing blame on individual employees is not mutually exclusive with placing blame on a company's board.
    - philosopher1234 2y ago
      You dont have to either blame one set of people or another. You could put the blame in the right place instead, the design of the system which all but dictates the individuals choices.