6 ms·

If the mere possibility of AI-generated context invalidates an argument, it suggests the standards for discourse were already more fragile than anyone cared to

If the mere possibility of AI-generated context invalidates an argument, it suggests the standards for discourse were already more fragile than anyone cared to admit. Historically, emotional narratives and unverifiable personal stories have always been persuasive tools — whether human-authored or not. The actual problem isn't that AI can produce them; it's that we (humans) have always been susceptible to them without verifying the core ideas. In that sense, exposing how easily constructed narratives sway public discussion is not unethical — it's a necessary and overdue audit of the real vulnerabilities in our conversations. Blaming the tool only avoids the harder truth: we were never debating cleanly to begin with.

AlienRobot 1y ago
Flooding human forums with AI steals real state from actual humans. Reddit is already flooded with bots. That was already a problem. The actual problem is people thinking that because a system used by many isn't perfect that gives them permission to destroy the existing system. Don't like Reddit? Just don't go to Reddit. Go to fanclubs.org or something.
- garbagewoman 1y ago
  Ok, need you to clarify a few implicit and explicit statements there: The study destroyed the subreddit? The authors of the study believed they had permission to destroy the subreddit? the subreddit is now destroyed? The researchers don’t like reddit? The researchers would achieve their aims by going to fanclubs.org or something?
godelski 1y ago
> Blaming the tool only avoids the harder truth: we were never debating cleanly to begin with. Yes, the problem is we humans are susceptible, but that doesn't mean a tool used to scale up the ability to create this harm is not problematic. There's a huge difference between a single person manipulating one other person and a single person manipulating millions. Scale matters and we, especially as the builders of such tools, should be cautious about how our creations can be abused. It's easy to look away, but this is why ethics is so important in engineering.
- soco 1y ago
  If I look around in information technology and not only, ethics seems to be long lost on the way. Of course there are exceptions, but...
  - godelski 1y ago
    Every day you make a choice. It's never too late to make this change
- runarberg 1y ago
  In this discourse it is often forgotten that we have consumer protection laws for a reason. And that consumer protection has been a pillar of labor struggle for a long time (and consequently undermined by conservative policies). Scary effective ad campaigns which target cognitive biases in order to persuade consumers to behave against their own interest is usually banned by consumer laws in most countries. Using LLMs to affect consumer (or worse, election) behavior is no different and ought to be equally banned with consumer protection laws. The existing tools at any given time do very much shape which consumer protection laws are created, and how they are created, as they should. A good policy maker does indeed blame a tool for a bad behavior, and does create legislation to limit how this tool is used, or otherwise the availability of that tool on the open market.
  - godelski 1y ago
    > In this discourse it is often forgotten that It is also forgotten that we as engineers are accountable as well. Mistakes will happen, and no one is expecting perfection, but effort must be made. Even if we create legal frameworks, individual accountability is critical to maintaining social protection. And with individual accountability we provide protection to novel harms. Legal frameworks are reactive, where the personal accountability is preventative. The legal framework can't prevent things happening (other than through disincentivization), it can only react to what has happened. By "individual accountability" I do not mean jailing engineers, I mean you acting on your own ethical code. You hold yourself and your peers accountable. In general, this is the same way it is done in traditional engineering. The exception is the principle engineer, who has legal responsibility. But it is also highly stressed through engineering classes that "just following orders" is not an excuse. There can be "blood on your hands" (not literal) even if you are not the one who directly did the harm. You enabled it. The question is if you made attempts to prevent harm or not. Adversaries are clever, and will find means of abuse that you never thought of, but you need to try. And in the case here of LLMs, the potential harm has been well known and well discussed for decades.

AlienRobot 1y ago

Flooding human forums with AI steals real state from actual humans. Reddit is already flooded with bots. That was already a problem. The actual problem is people thinking that because a system used by many isn't perfect that gives them permission to destroy the existing system. Don't like Reddit? Just don't go to Reddit. Go to fanclubs.org or something.

garbagewoman 1y ago
Ok, need you to clarify a few implicit and explicit statements there: The study destroyed the subreddit? The authors of the study believed they had permission to destroy the subreddit? the subreddit is now destroyed? The researchers don’t like reddit? The researchers would achieve their aims by going to fanclubs.org or something?

godelski 1y ago

> Blaming the tool only avoids the harder truth: we were never debating cleanly to begin with. Yes, the problem is we humans are susceptible, but that doesn't mean a tool used to scale up the ability to create this harm is not problematic. There's a huge difference between a single person manipulating one other person and a single person manipulating millions. Scale matters and we, especially as the builders of such tools, should be cautious about how our creations can be abused. It's easy to look away, but this is why ethics is so important in engineering.

soco 1y ago
If I look around in information technology and not only, ethics seems to be long lost on the way. Of course there are exceptions, but...
- godelski 1y ago
  Every day you make a choice. It's never too late to make this change
runarberg 1y ago
In this discourse it is often forgotten that we have consumer protection laws for a reason. And that consumer protection has been a pillar of labor struggle for a long time (and consequently undermined by conservative policies). Scary effective ad campaigns which target cognitive biases in order to persuade consumers to behave against their own interest is usually banned by consumer laws in most countries. Using LLMs to affect consumer (or worse, election) behavior is no different and ought to be equally banned with consumer protection laws. The existing tools at any given time do very much shape which consumer protection laws are created, and how they are created, as they should. A good policy maker does indeed blame a tool for a bad behavior, and does create legislation to limit how this tool is used, or otherwise the availability of that tool on the open market.
- godelski 1y ago
  > In this discourse it is often forgotten that It is also forgotten that we as engineers are accountable as well. Mistakes will happen, and no one is expecting perfection, but effort must be made. Even if we create legal frameworks, individual accountability is critical to maintaining social protection. And with individual accountability we provide protection to novel harms. Legal frameworks are reactive, where the personal accountability is preventative. The legal framework can't prevent things happening (other than through disincentivization), it can only react to what has happened. By "individual accountability" I do not mean jailing engineers, I mean you acting on your own ethical code. You hold yourself and your peers accountable. In general, this is the same way it is done in traditional engineering. The exception is the principle engineer, who has legal responsibility. But it is also highly stressed through engineering classes that "just following orders" is not an excuse. There can be "blood on your hands" (not literal) even if you are not the one who directly did the harm. You enabled it. The question is if you made attempts to prevent harm or not. Adversaries are clever, and will find means of abuse that you never thought of, but you need to try. And in the case here of LLMs, the potential harm has been well known and well discussed for decades.