3 ms·

> In this discourse it is often forgotten that It is also forgotten that we as engineers are accountable as well. Mistakes will happen, and no one is expecting

> In this discourse it is often forgotten that It is also forgotten that we as engineers are accountable as well. Mistakes will happen, and no one is expecting perfection, but effort must be made. Even if we create legal frameworks, individual accountability is critical to maintaining social protection. And with individual accountability we provide protection to novel harms. Legal frameworks are reactive, where the personal accountability is preventative. The legal framework can't prevent things happening (other than through disincentivization), it can only react to what has happened. By "individual accountability" I do not mean jailing engineers, I mean you acting on your own ethical code. You hold yourself and your peers accountable. In general, this is the same way it is done in traditional engineering. The exception is the principle engineer, who has legal responsibility. But it is also highly stressed through engineering classes that "just following orders" is not an excuse. There can be "blood on your hands" (not literal) even if you are not the one who directly did the harm. You enabled it. The question is if you made attempts to prevent harm or not. Adversaries are clever, and will find means of abuse that you never thought of, but you need to try. And in the case here of LLMs, the potential harm has been well known and well discussed for decades.

sokoloff 2y ago
What does that look like in practice, assuming an engineer doesn’t believe that the LLM genie can be put back into the toothpaste tube? “Summarize the best arguments for and against the following proposition: <topic here />. Label the pro/for arguments with a <pro> tag and the con/against a with a <con> tag” seems like it’s going to be a valid prompt and any system that can only give one side is bound to lose to a system that can give both sides. And any system that can give those answers can be pretty easily used to make arguments of varying truthfulness.
- godelski 2y ago
  > What does that look like in practice In action it looks like - You act on your morals. If you find something objectionable, then object. Vocally. - You, yourself, get in the habit of trying to find issues with the things you build. This is an essential part of your job as an engineer. - You can't make things better if you aren't looking for problems. The job of an engineer is to look for flaws and fix them. - Encourage culture where your cohort understands that when someone is saying "but what about" or "how would we handle" isn't saying "no" but "let's get ahead of this problem". That person is doing their job, they're not trying to be a killjoy. They're trying to make the product better.[0] - If your coworkers are doing something unethical, say something. - If your boss does something unethical, say something. It doesn't matter what your job is, you should always be doing that. As engineers the potential for harm is greater. But importantly, as engineers IT IS YOUR JOB. It is your job to find issues, and solve them. You have to think how people will abuse the tools you build. You have to think about how your things will fail. You have to think about what sucks. And most importantly, your job is to then resolve those things. That's what an engineer does. Don't dismiss it because "there's no value". The job of an engineer isn't to determine monetary value, that's the business people (obviously you do to some extent, but it isn't the primary focus). I'm really just asking that people do their jobs and not throw their hands up in the air or just pass on blame or kick the can down the road. [0] I can't express how many times I've seen these people shut down and then passed over for promotion. It creates yes men. But yes men are bad for the business too! You and your actions matter: https://talyarkoni.org/blog/2018/10/02/no-its-not-the-incentives-its-you/ https://talyarkoni.org/blog/2018/10/02/no-its-not-the-incent...
runarberg 2y ago
> I mean you acting on your own ethical code. You hold yourself and your peers accountable Industry wide self regulation is a poor substitute for actual regulation, especially in this capitalistic environment which rewards profitable behavior regardless of morality or ethics. In this environment the best an engineer can do is resign in protest (and I applaud any engineer which does that in fact), however that won’t stop the company from hiring new engineers who value their salary more than their ethical behavior—or have different ethical standards. > And in the case here of LLMs, the potential harm has been well known and well discussed for decades. The harms posed by LLMs are the very same which are caused by any company in the pursue of profits without regulation. In the past the only proven methods to force company to behave ethically has be industry wide regulation, especially consumer protection regulations.
- godelski 2y ago
  > is a poor substitute for actual regulation Substitute? No. But you need both. Sorry, it's not the incentives, it is you[0]. As I said before, regulation is reactionary. This is why you need both. No one is saying no regulation, but I (and [0]) is saying things only happen because people do them. I know this is a wild claim, but it is an indisputable fact. > The harms posed by LLMs are the very same I expect every programmer and HN user to be familiar with scale. Please stop making this argument. You might as well be saying a nuclear bomb is the same as a Pop-Its. The dangers that LLMs pose is still unknown but if we're unwilling to acknowledge that there's any unique harm then there's zero chance of solving them. Stop passing the buck. [0] https://talyarkoni.org/blog/2018/10/02/no-its-not-the-incentives-its-you/ https://talyarkoni.org/blog/2018/10/02/no-its-not-the-incent...
  - runarberg 2y ago
    I‘m sorry, but I’m not buying it. Other industries have had the technology for a long time to mass produce low quality products which harms their customers, or worse, bystanders. And in many of those cases the harmful production was never allowed on the market because the regulator was proactive and prevented the technology to be sold to consumers. I know those stories aren’t as prominent as the story of leaded gasoline, because stories of harm spread much further than stories of prevented harm. But they do exist, and are numerous. I also fail to see why we need both regulation and moral behavior from developers. If the regulation exists, and the regulator is willing to enforce it, any company which goes against the regulation will be breaking the law, and stopped by the regulator. We only need the regulation in this case.

sokoloff 2y ago

What does that look like in practice, assuming an engineer doesn’t believe that the LLM genie can be put back into the toothpaste tube? “Summarize the best arguments for and against the following proposition: <topic here />. Label the pro/for arguments with a <pro> tag and the con/against a with a <con> tag” seems like it’s going to be a valid prompt and any system that can only give one side is bound to lose to a system that can give both sides. And any system that can give those answers can be pretty easily used to make arguments of varying truthfulness.

godelski 2y ago
> What does that look like in practice In action it looks like - You act on your morals. If you find something objectionable, then object. Vocally. - You, yourself, get in the habit of trying to find issues with the things you build. This is an essential part of your job as an engineer. - You can't make things better if you aren't looking for problems. The job of an engineer is to look for flaws and fix them. - Encourage culture where your cohort understands that when someone is saying "but what about" or "how would we handle" isn't saying "no" but "let's get ahead of this problem". That person is doing their job, they're not trying to be a killjoy. They're trying to make the product better.[0] - If your coworkers are doing something unethical, say something. - If your boss does something unethical, say something. It doesn't matter what your job is, you should always be doing that. As engineers the potential for harm is greater. But importantly, as engineers IT IS YOUR JOB. It is your job to find issues, and solve them. You have to think how people will abuse the tools you build. You have to think about how your things will fail. You have to think about what sucks. And most importantly, your job is to then resolve those things. That's what an engineer does. Don't dismiss it because "there's no value". The job of an engineer isn't to determine monetary value, that's the business people (obviously you do to some extent, but it isn't the primary focus). I'm really just asking that people do their jobs and not throw their hands up in the air or just pass on blame or kick the can down the road. [0] I can't express how many times I've seen these people shut down and then passed over for promotion. It creates yes men. But yes men are bad for the business too! You and your actions matter: https://talyarkoni.org/blog/2018/10/02/no-its-not-the-incentives-its-you/ https://talyarkoni.org/blog/2018/10/02/no-its-not-the-incent...

runarberg 2y ago

> I mean you acting on your own ethical code. You hold yourself and your peers accountable Industry wide self regulation is a poor substitute for actual regulation, especially in this capitalistic environment which rewards profitable behavior regardless of morality or ethics. In this environment the best an engineer can do is resign in protest (and I applaud any engineer which does that in fact), however that won’t stop the company from hiring new engineers who value their salary more than their ethical behavior—or have different ethical standards. > And in the case here of LLMs, the potential harm has been well known and well discussed for decades. The harms posed by LLMs are the very same which are caused by any company in the pursue of profits without regulation. In the past the only proven methods to force company to behave ethically has be industry wide regulation, especially consumer protection regulations.

godelski 2y ago
> is a poor substitute for actual regulation Substitute? No. But you need both. Sorry, it's not the incentives, it is you[0]. As I said before, regulation is reactionary. This is why you need both. No one is saying no regulation, but I (and [0]) is saying things only happen because people do them. I know this is a wild claim, but it is an indisputable fact. > The harms posed by LLMs are the very same I expect every programmer and HN user to be familiar with scale. Please stop making this argument. You might as well be saying a nuclear bomb is the same as a Pop-Its. The dangers that LLMs pose is still unknown but if we're unwilling to acknowledge that there's any unique harm then there's zero chance of solving them. Stop passing the buck. [0] https://talyarkoni.org/blog/2018/10/02/no-its-not-the-incentives-its-you/ https://talyarkoni.org/blog/2018/10/02/no-its-not-the-incent...
- runarberg 2y ago
  I‘m sorry, but I’m not buying it. Other industries have had the technology for a long time to mass produce low quality products which harms their customers, or worse, bystanders. And in many of those cases the harmful production was never allowed on the market because the regulator was proactive and prevented the technology to be sold to consumers. I know those stories aren’t as prominent as the story of leaded gasoline, because stories of harm spread much further than stories of prevented harm. But they do exist, and are numerous. I also fail to see why we need both regulation and moral behavior from developers. If the regulation exists, and the regulator is willing to enforce it, any company which goes against the regulation will be breaking the law, and stopped by the regulator. We only need the regulation in this case.