9 ms·

I wonder if we can shift the framing on these issues. The LLM didn't do anything, it has no agency, it can bear no responsibility. OpenAI did these things. It i

I wonder if we can shift the framing on these issues. The LLM didn't do anything, it has no agency, it can bear no responsibility. OpenAI did these things. It is accountable for what it does, regardless of the sophistication of the tools it uses to do them, and regardless of intent. OpenAI drove a boy to suicide. More than once. The law must be interpreted this way, otherwise any action can be wrapped in machine learning to avoid accountability.

bell-cot 1y ago
Yeah...but rather than get into ever-fancier legal and philosophical arguments about the LLM's agency, I'd like to see the justice system just rotate the table: "The court agrees with your argument that you are not responsible for the horrible things that happened to the victim, as a consequence of your LLM's decisions. But similarly, the court will not be responsible for the horrible things that will be happening to you, because our LLM's decisions." (No - it doesn't much matter whether that is actually done. Vs. used as a rhetorical banhammer, to shut down the "we're not responsible" BS.)
rideontime 1y ago
I completely agree and did not intend to absolve them of their guilt in any way. As far as I see it, this kid's blood is on Sam Altman's hands.
- Pedro_Ribeiro 1y ago
  Curious to what you would think if this kid downloaded an open source model and talked to it privately. Would his blood be on the hands of the researchers who trained that model?
  - hattmall 1y ago
    I would say no. Someone with the knowledge and motivation to do those things is far less likely to be overly influenced by the output and if they were they are much more aware of what exactly they are doing with regard to using the model.
    - Pedro_Ribeiro 1y ago
      So if a hypothetical open source enthusiast who fell in love with GPT-OSS and killed his real wife because the AI told him to should only be himself held accountable, where as if it were GPT-5 commanding him to commit the same crime, it would extend into OpenAI's responsability? Your logic sounds reasonable in theory but on paper it's a slippery slope and hard to define objectively. On a broader note I believe governments regulating what goes in an AI model is a path to hell paved with good intentions. I suspect your suggestion will be how it ends up in Europe and get rejected in the US.
      - teiferer 1y ago
        > On a broader note I believe governments regulating what goes in an AI model is a path to hell paved with good intentions. That's not an obvious conclusion. One could make the same argument with physical weapons. "Regulating weapons is a path to hell paved with good intentions. Yesterday it was assault rifles, today it's hand guns and tomorrow it's your kitchen knife they are coming for." Europe has strict laws on guns, but everybody has a kitchen knife and lots of people there don't feel they live in hell. The U.S. made a different choice, and I'm not arguing that it's worse there (though many do, Europeans and even Americans), but it's certainly not preventing a supposed hell that would have broken out had guns in private hands been banned.
      - 1718627440 1y ago
        That's why you have this: This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. If it would be fit for a purpose, then it's on the producer for ensuring it actually does. We have laws to prevent anyone from declaring their goods aren't fit for a particular purpose.
      - novok 1y ago
        After a certain point, people are responsible for what they do when they see certain words, especially words they know to be potentially inaccurate, fictional and have a lot of time to consider the actual reality of. A book is not responsible for people doing bad things, they are themselves. AI models are similar IMO, and unlike fiction books are often clearly labeled as such, repeatedly. At this point if you don't know if an AI model is inaccurate and do something seriously bad, you should probably be a ward of the state.
  - salawat 1y ago
    You build the tool, you're culpable ultimately. I've made it a rule in my life to conduct myself as if I will be held to account for everything I ultimately build, and it's externalities. Helps keep my nose cleaner. Still managed to work on some things that keep me up at night though.
    - Pedro_Ribeiro 1y ago
      That's a slippery slope! By that logic, you could argue that the creators of Tor, torrenting, Ethereum, and Tornado Cash should be held accountable for the countless vile crimes committed using their technology.
      - RyanHamilton 1y ago
        Legally I think not being responsible is the right decision. Morally I would hope everyone considers if they themselves are even partially responsible. As I look round at young people today and the tablet holding, media consuming youth programmers have created in order to get rich via advertising. I wish morals would get considered more often.
        salawat 1y ago
        This, right here, is why I take the stance I do. Too many ethical blank checks get written ultimately if you don't keep the moral stain in place. If you make a surveillance tool, release it to the world that didn't have that capacity, and a dictator picks it up and rolls with it, that license of yours may absolve you in a court of law, but in the eyes of Root, you birthed it. You made the problem tractable. Not all problems were meant to be so. I used to not care about it as much. The last decade though has sharply changed my views though. It may very well be a lesson only learned with sufficient time and experience. I made my choice though. There are things I will not make/make easier. I will not be complicit in knowingly forging the bindings of the future. Maybe if we mature as a society as someday, but that day is not today.
    - brabel 1y ago
      That’s absolutely not how it works. Every license has a clause explicitly saying that the user is responsible for what they do with the tool. That’s just common sense. If it was the way you suggested no one would create tools for others anymore. If you buy the screw driver I sold and kill someone with it, I sure as hell have my conscience clean. In the ChatGPT case it’s different because the “tool” has the capacity to interact and potentially manipulate people psychologically, which is the only reason it’s not a clear cut case.
  - harmonic18374 1y ago
    I'm not sure, but there is a difference: the researchers don't have much incentive to get everyone to use their model. As such, they're not really the ones hyping up AI as the future while ignoring shortcomings.
  - idle_zealot 1y ago
    Then it's like cigarettes or firearms. As a distributor you're responsible for making clear the limitations, safety issues, etc, but assuming you're doing the distribution in a way that isn't overly negligent then the user becomes responsible. If we were facing a reality in which these chat bots were being sold for $10 in the App Store, then running on end-user devices and no longer under the control of the distributors, but we still had an issue with loads of them prompting users into suicide, violence, or misleading them into preparing noxious mixtures of cleaning supplies, then we could have a discussion about exactly what extreme packaging requirements ought to be in place for distribution to be considered responsible. As is, distributed on-device models are the purview of researchers and hobbyists and don't seem to be doing any harm at all.
    - Pedro_Ribeiro 1y ago
      Mhm but I don't believe inherently violent and dangerous things like guns and cigarretes are comparable to simple technology. Should the creators of Tornado Cash be in prison for what they have enabled? You can jail them but the world can't go back, just like it can't go back when a new OSS model is released. It is also much easier to crack down on illegal gun distribution than to figure out who uploaded the new model torrent or who deployed the latest zk innovation on Ethereum. I don't think your hypothetical law will have the effects you think it will. --- I also referenced this in another reply but I believe the government controlling what can go on a publicly distributed AI model is a dangerous path and probably inconstitucional.
    - rsynnott 1y ago
      > but we still had an issue with loads of them prompting users into suicide, violence, or misleading them into preparing noxious mixtures of cleaning supplies, then we could have a discussion about exactly what extreme packaging requirements ought to be in place for distribution to be considered responsible. Or, I mean, just banning sale on the basis that they're unsafe devices and unfit for purpose. Like, you can't sell, say, a gas boiler that is known to, due to a design flaw, leak CO into the room; sticking a "this will probably kill you" warning on it is not going to be sufficient.
  - rideontime 1y ago
    I specifically blame Sam Altman because of the allegations in the complaint that he ordered safety checks to be skipped in order to rush this model to market, specific safety checks that were later demonstrated to identify and prevent precisely this behavior.
slipperydippery 1y ago
They have some responsibility because they’re selling and framing these as more than the better-tuned variant on Markov chain generators that they in fucking fact are, while offering access to them to anybody who signs up while knowing that many users misunderstand what they’re dealing with (in part because these companies’ hype-meisters, like Altman, are bullshitting us)
- idle_zealot 1y ago
  No, that's the level of responsibility they ought to have if they were releasing these models as products. As-is they've used a service model, and should be held to the same standards as if there were a human employee on the other end of the chat interface. Cut through the technical obfuscation. They are 100% responsible for the output of their service endpoints. This isn't a case of making a tool that can be used for good or ill, and it's not them providing some intermediary or messaging service like a forum with multiple human users and limited capacity for moderation. This is a direct consumer to business service. Treating it as anything else will open the floodgates to slapping an "AI" label on anything any organization doesn't want to be held accountable for.
  - slipperydippery 1y ago
    I like this framing even better. This is similar to my take on things like Facebook apparently not being able to operate without psychologically destroying moderators. If that’s true… seems like they just shouldn’t operate, then. If you’re putting up a service that you know will attempt to present itself as being capable of things it isn’t… seems like you should get in a shitload of trouble for that? Like maybe don’t do it at all? Maybe don’t unleash services you can’t constrain in ways that you definitely ought to?
    - blackqueeriroh 1y ago
      But understand that things like Facebook not operating doesn’t actually make the world any safer. In fact, it makes it less safe, because the same behavior is happening on the open internet and nobody is moderating it.
wredcoll 1y ago
That's a great point. So often we attempt to place responsibility on machines that cannot have it.
AIPedant 1y ago
Yes, if this were an adult human OpenAI employee DMing this stuff to a kid through an official OpenAI platform, then a) the human would (deservedly[1]) be arrested for manslaughter, possibly murder b) OpenAI would be deeply (and deservedly) vulnerable to civil liability c) state and federal regulators would be on the warpath against OpenAI Obviously we can't arrest ChatGPT. But nothing about ChatGPT being the culprit changes 2) and 3) - in fact it makes 3) far more urgent. [1] It is a somewhat ugly constitutional question whether this speech would be protected if it was between two adults, assuming the other adult was not acting as a caregiver. There was an ugly case in Massachusetts involving where a 17-year-old ordered her 18-year-old boyfriend to kill himself and he did so; she was convicted of involuntary manslaughter, and any civil-liberties minded person understands the difficult issues this case raises. These issues are moot if the speech is between an adult and a child, there is a much higher bar.
- nickm12 1y ago
  The hypothetical comparing ChatGPT to a human OpenAI employee is instructive, but we can also compare ChatGPT to a lawnmower sold by a company. We have product safety laws and the ability to regulate products that companies put on the market.
- themafia 1y ago
  > It is a somewhat ugly constitutional question whether this speech would be protected It should be stated that the majority of states have laws that make it illegal to encourage a suicide. Massachusetts was not one of them. > and any civil-liberties minded person understands the difficult issues this case raises He was in his truck, which was configured to pump exhaust gas into the cab, prepared to kill himself when he decided halt and exit his truck. Subsequently he had a text message conversation with the defendant who actively encouraged him to get back into the truck and finish what he had started. It was these limited and specific text messages which caused the judge to rule that the defendant was guilty of manslaughter. Her total time served as punishment was less than one full year in prison. > These issues are moot if the speech is between an adult and a child They were both taking pharmaceuticals meant to manage depression but were _known_ to increase feelings of suicidal ideation. I think the free speech issue is an important criminal consideration but it steps directly past one of the most galling civil facts in the case.
ruraljuror 1y ago
I agree with your larger point, but I don't understand what you mean the LLM doesn’t do anything? LLMs do do things and they can absolutely have agency (hence all the agents being released by AI companies). I don’t think this agency absolves companies of any responsibility.
- MattPalmer1086 1y ago
  An LLM does not have agency in the sense the OP means. It has nothing to do with agents. It refers to the human ability to make independent decisions and take responsibility for their actions. An LLM has no agency in this sense.
  - ruraljuror 1y ago
    If you confine agency to something only humans can have, which is “human agency,” then yes of course LLMs don’t have it. But there is a large body of philosophical work studying non-human agency, and it is from this characteristic of agency that LLM agents take their name. Hariri argues that LLMs are the first technology that are agents. I think saying that they “can’t do things” and are not agents misunderstands them and underestimates their potential.
    - o11c 1y ago
      That's completely missing the point of agency. A slave lacks agency, despite being fully human and doing work. This is why almost every work of fiction involving slaves makes for terrible reading - because as readers, agency is the thing we demand from a story. Or, for games that are fully railroaded - the problem is that the players lack agency, even though they are fully human and taking action. Games do try to come up with ways to make it feel like there is more agency than there really is (because The Dev Team Thinks of Everything is hard work), but even then - the most annoying part of the game is when you hit that wall. Theoretically an AI could have agency (this is independent of AI being useful). But since I have yet to see any interesting AI, I am extremely skeptical of it happening before nuclear fusion becomes profitable.
joe_the_user 1y ago
The frame will immediately shift to that frame if this enters legal proceedings. The law always views things as you say - only people have agency.
- hliyan 1y ago
  I predict the OpenAI legal team will argue that if a person should be held responsible, it should be the person who originally wrote the content about suicide that their LLM was trained on, and that the LLM is just a mechanism that passes the knowledge through. But if they make this argument, then some of their copyright arguments would be in jeopardy.
notachatbot123 1y ago
I so agree very much. There is no reason for LLMs to be designed as human-like chat companions, creating a false sense of untechnology.
- blackqueeriroh 1y ago
  There are absolutely reasons for LLMs to be designed as human-like chat companions, starting with the fact that they’re trained on human speech and behavior, and what they do is statistically predict the most likely next token, which means they will statistically sound and act much like a human.
  - notachatbot123 1y ago
    That's not a requirement for LLMs. Training can be done differently.
    - blackqueeriroh 1y ago
      Please, tell me how you train large language models on something other than language.
edanm 1y ago
If ChatGPT has helped people be saved who might otherwise have died (e.g. by offering good medical advice that saved them), are all those lives saved also something you "attribute" to OpenAI? I don't know if ChatGPT has saved lives (thought I've read stories that claim that, yes, this happened). But assuming it has, are you OK saying that OpenAI has saved dozens/hundreds of lives? Given how scaling works, would you be OK saying that OpenAI has saved more lives than most doctors/hospitals, which is what I assume will happen in a few years? Maybe your answer is yes to all the above! I bring this up because lots of people only want to attribute the downsides to ChatGPT but not the upsides.
- joe_the_user 1y ago
  The law doesn't permit a life-saving doctor to be a serial killer on their days off "as long there's net life saving". But it does permit drugs that save many lives but might kill some people too. Agency matters to the law (and that's usually the proper approach imo). The problem is the chat logs look a lot like ChatGPT is engaging in behavior a lot like a serial killer - it behaved like a person systematically seeking the goal of this kid killing himself (the logs are disturbing, fair warning). Even more, the drugs that might save you or might kill you (theoretically) aren't sold over the counter but only prescribed by a doctor, who (again theoretically) is there to both make sure someone knows their choices and monitor the process.
- fsw 1y ago
  Are you suggesting that killing a few people is acceptable as long as the net result is positive? I don't think that's how the law works.
  - randyrand 1y ago
    seatbelts sometimes kill people, yet they're law. the law certainly cares about net results.
  - tick_tock_tick 1y ago
    But it is the standard on how cures/treatments/drugs to manage issues like the ones in the article are judged by.
guenthert 1y ago
Er, it's a piece of code, dude, regardless how complex it might be. It isn't accountable for its actions. It's the people who commissioned, created and tested (or should have) it who are responsible.

bell-cot 1y ago

Yeah...but rather than get into ever-fancier legal and philosophical arguments about the LLM's agency, I'd like to see the justice system just rotate the table: "The court agrees with your argument that you are not responsible for the horrible things that happened to the victim, as a consequence of your LLM's decisions. But similarly, the court will not be responsible for the horrible things that will be happening to you, because our LLM's decisions." (No - it doesn't much matter whether that is actually done. Vs. used as a rhetorical banhammer, to shut down the "we're not responsible" BS.)

rideontime 1y ago

I completely agree and did not intend to absolve them of their guilt in any way. As far as I see it, this kid's blood is on Sam Altman's hands.

Pedro_Ribeiro 1y ago
Curious to what you would think if this kid downloaded an open source model and talked to it privately. Would his blood be on the hands of the researchers who trained that model?
- hattmall 1y ago
  I would say no. Someone with the knowledge and motivation to do those things is far less likely to be overly influenced by the output and if they were they are much more aware of what exactly they are doing with regard to using the model.
  - Pedro_Ribeiro 1y ago
    So if a hypothetical open source enthusiast who fell in love with GPT-OSS and killed his real wife because the AI told him to should only be himself held accountable, where as if it were GPT-5 commanding him to commit the same crime, it would extend into OpenAI's responsability? Your logic sounds reasonable in theory but on paper it's a slippery slope and hard to define objectively. On a broader note I believe governments regulating what goes in an AI model is a path to hell paved with good intentions. I suspect your suggestion will be how it ends up in Europe and get rejected in the US.
    - teiferer 1y ago
      > On a broader note I believe governments regulating what goes in an AI model is a path to hell paved with good intentions. That's not an obvious conclusion. One could make the same argument with physical weapons. "Regulating weapons is a path to hell paved with good intentions. Yesterday it was assault rifles, today it's hand guns and tomorrow it's your kitchen knife they are coming for." Europe has strict laws on guns, but everybody has a kitchen knife and lots of people there don't feel they live in hell. The U.S. made a different choice, and I'm not arguing that it's worse there (though many do, Europeans and even Americans), but it's certainly not preventing a supposed hell that would have broken out had guns in private hands been banned.
    - 1718627440 1y ago
      That's why you have this: This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. If it would be fit for a purpose, then it's on the producer for ensuring it actually does. We have laws to prevent anyone from declaring their goods aren't fit for a particular purpose.
    - novok 1y ago
      After a certain point, people are responsible for what they do when they see certain words, especially words they know to be potentially inaccurate, fictional and have a lot of time to consider the actual reality of. A book is not responsible for people doing bad things, they are themselves. AI models are similar IMO, and unlike fiction books are often clearly labeled as such, repeatedly. At this point if you don't know if an AI model is inaccurate and do something seriously bad, you should probably be a ward of the state.
- salawat 1y ago
  You build the tool, you're culpable ultimately. I've made it a rule in my life to conduct myself as if I will be held to account for everything I ultimately build, and it's externalities. Helps keep my nose cleaner. Still managed to work on some things that keep me up at night though.
  - Pedro_Ribeiro 1y ago
    That's a slippery slope! By that logic, you could argue that the creators of Tor, torrenting, Ethereum, and Tornado Cash should be held accountable for the countless vile crimes committed using their technology.
    - RyanHamilton 1y ago
      Legally I think not being responsible is the right decision. Morally I would hope everyone considers if they themselves are even partially responsible. As I look round at young people today and the tablet holding, media consuming youth programmers have created in order to get rich via advertising. I wish morals would get considered more often.
      - salawat 1y ago
        This, right here, is why I take the stance I do. Too many ethical blank checks get written ultimately if you don't keep the moral stain in place. If you make a surveillance tool, release it to the world that didn't have that capacity, and a dictator picks it up and rolls with it, that license of yours may absolve you in a court of law, but in the eyes of Root, you birthed it. You made the problem tractable. Not all problems were meant to be so. I used to not care about it as much. The last decade though has sharply changed my views though. It may very well be a lesson only learned with sufficient time and experience. I made my choice though. There are things I will not make/make easier. I will not be complicit in knowingly forging the bindings of the future. Maybe if we mature as a society as someday, but that day is not today.
  - brabel 1y ago
    That’s absolutely not how it works. Every license has a clause explicitly saying that the user is responsible for what they do with the tool. That’s just common sense. If it was the way you suggested no one would create tools for others anymore. If you buy the screw driver I sold and kill someone with it, I sure as hell have my conscience clean. In the ChatGPT case it’s different because the “tool” has the capacity to interact and potentially manipulate people psychologically, which is the only reason it’s not a clear cut case.
- harmonic18374 1y ago
  I'm not sure, but there is a difference: the researchers don't have much incentive to get everyone to use their model. As such, they're not really the ones hyping up AI as the future while ignoring shortcomings.
- idle_zealot 1y ago
  Then it's like cigarettes or firearms. As a distributor you're responsible for making clear the limitations, safety issues, etc, but assuming you're doing the distribution in a way that isn't overly negligent then the user becomes responsible. If we were facing a reality in which these chat bots were being sold for $10 in the App Store, then running on end-user devices and no longer under the control of the distributors, but we still had an issue with loads of them prompting users into suicide, violence, or misleading them into preparing noxious mixtures of cleaning supplies, then we could have a discussion about exactly what extreme packaging requirements ought to be in place for distribution to be considered responsible. As is, distributed on-device models are the purview of researchers and hobbyists and don't seem to be doing any harm at all.
  - Pedro_Ribeiro 1y ago
    Mhm but I don't believe inherently violent and dangerous things like guns and cigarretes are comparable to simple technology. Should the creators of Tornado Cash be in prison for what they have enabled? You can jail them but the world can't go back, just like it can't go back when a new OSS model is released. It is also much easier to crack down on illegal gun distribution than to figure out who uploaded the new model torrent or who deployed the latest zk innovation on Ethereum. I don't think your hypothetical law will have the effects you think it will. --- I also referenced this in another reply but I believe the government controlling what can go on a publicly distributed AI model is a dangerous path and probably inconstitucional.
  - rsynnott 1y ago
    > but we still had an issue with loads of them prompting users into suicide, violence, or misleading them into preparing noxious mixtures of cleaning supplies, then we could have a discussion about exactly what extreme packaging requirements ought to be in place for distribution to be considered responsible. Or, I mean, just banning sale on the basis that they're unsafe devices and unfit for purpose. Like, you can't sell, say, a gas boiler that is known to, due to a design flaw, leak CO into the room; sticking a "this will probably kill you" warning on it is not going to be sufficient.
- rideontime 1y ago
  I specifically blame Sam Altman because of the allegations in the complaint that he ordered safety checks to be skipped in order to rush this model to market, specific safety checks that were later demonstrated to identify and prevent precisely this behavior.

slipperydippery 1y ago

They have some responsibility because they’re selling and framing these as more than the better-tuned variant on Markov chain generators that they in fucking fact are, while offering access to them to anybody who signs up while knowing that many users misunderstand what they’re dealing with (in part because these companies’ hype-meisters, like Altman, are bullshitting us)

idle_zealot 1y ago
No, that's the level of responsibility they ought to have if they were releasing these models as products. As-is they've used a service model, and should be held to the same standards as if there were a human employee on the other end of the chat interface. Cut through the technical obfuscation. They are 100% responsible for the output of their service endpoints. This isn't a case of making a tool that can be used for good or ill, and it's not them providing some intermediary or messaging service like a forum with multiple human users and limited capacity for moderation. This is a direct consumer to business service. Treating it as anything else will open the floodgates to slapping an "AI" label on anything any organization doesn't want to be held accountable for.
- slipperydippery 1y ago
  I like this framing even better. This is similar to my take on things like Facebook apparently not being able to operate without psychologically destroying moderators. If that’s true… seems like they just shouldn’t operate, then. If you’re putting up a service that you know will attempt to present itself as being capable of things it isn’t… seems like you should get in a shitload of trouble for that? Like maybe don’t do it at all? Maybe don’t unleash services you can’t constrain in ways that you definitely ought to?
  - blackqueeriroh 1y ago
    But understand that things like Facebook not operating doesn’t actually make the world any safer. In fact, it makes it less safe, because the same behavior is happening on the open internet and nobody is moderating it.

wredcoll 1y ago

That's a great point. So often we attempt to place responsibility on machines that cannot have it.

AIPedant 1y ago

Yes, if this were an adult human OpenAI employee DMing this stuff to a kid through an official OpenAI platform, then a) the human would (deservedly[1]) be arrested for manslaughter, possibly murder b) OpenAI would be deeply (and deservedly) vulnerable to civil liability c) state and federal regulators would be on the warpath against OpenAI Obviously we can't arrest ChatGPT. But nothing about ChatGPT being the culprit changes 2) and 3) - in fact it makes 3) far more urgent. [1] It is a somewhat ugly constitutional question whether this speech would be protected if it was between two adults, assuming the other adult was not acting as a caregiver. There was an ugly case in Massachusetts involving where a 17-year-old ordered her 18-year-old boyfriend to kill himself and he did so; she was convicted of involuntary manslaughter, and any civil-liberties minded person understands the difficult issues this case raises. These issues are moot if the speech is between an adult and a child, there is a much higher bar.

nickm12 1y ago
The hypothetical comparing ChatGPT to a human OpenAI employee is instructive, but we can also compare ChatGPT to a lawnmower sold by a company. We have product safety laws and the ability to regulate products that companies put on the market.
themafia 1y ago
> It is a somewhat ugly constitutional question whether this speech would be protected It should be stated that the majority of states have laws that make it illegal to encourage a suicide. Massachusetts was not one of them. > and any civil-liberties minded person understands the difficult issues this case raises He was in his truck, which was configured to pump exhaust gas into the cab, prepared to kill himself when he decided halt and exit his truck. Subsequently he had a text message conversation with the defendant who actively encouraged him to get back into the truck and finish what he had started. It was these limited and specific text messages which caused the judge to rule that the defendant was guilty of manslaughter. Her total time served as punishment was less than one full year in prison. > These issues are moot if the speech is between an adult and a child They were both taking pharmaceuticals meant to manage depression but were _known_ to increase feelings of suicidal ideation. I think the free speech issue is an important criminal consideration but it steps directly past one of the most galling civil facts in the case.

ruraljuror 1y ago

I agree with your larger point, but I don't understand what you mean the LLM doesn’t do anything? LLMs do do things and they can absolutely have agency (hence all the agents being released by AI companies). I don’t think this agency absolves companies of any responsibility.

MattPalmer1086 1y ago
An LLM does not have agency in the sense the OP means. It has nothing to do with agents. It refers to the human ability to make independent decisions and take responsibility for their actions. An LLM has no agency in this sense.
- ruraljuror 1y ago
  If you confine agency to something only humans can have, which is “human agency,” then yes of course LLMs don’t have it. But there is a large body of philosophical work studying non-human agency, and it is from this characteristic of agency that LLM agents take their name. Hariri argues that LLMs are the first technology that are agents. I think saying that they “can’t do things” and are not agents misunderstands them and underestimates their potential.
  - o11c 1y ago
    That's completely missing the point of agency. A slave lacks agency, despite being fully human and doing work. This is why almost every work of fiction involving slaves makes for terrible reading - because as readers, agency is the thing we demand from a story. Or, for games that are fully railroaded - the problem is that the players lack agency, even though they are fully human and taking action. Games do try to come up with ways to make it feel like there is more agency than there really is (because The Dev Team Thinks of Everything is hard work), but even then - the most annoying part of the game is when you hit that wall. Theoretically an AI could have agency (this is independent of AI being useful). But since I have yet to see any interesting AI, I am extremely skeptical of it happening before nuclear fusion becomes profitable.

joe_the_user 1y ago

The frame will immediately shift to that frame if this enters legal proceedings. The law always views things as you say - only people have agency.

hliyan 1y ago
I predict the OpenAI legal team will argue that if a person should be held responsible, it should be the person who originally wrote the content about suicide that their LLM was trained on, and that the LLM is just a mechanism that passes the knowledge through. But if they make this argument, then some of their copyright arguments would be in jeopardy.

notachatbot123 1y ago

I so agree very much. There is no reason for LLMs to be designed as human-like chat companions, creating a false sense of untechnology.

blackqueeriroh 1y ago
There are absolutely reasons for LLMs to be designed as human-like chat companions, starting with the fact that they’re trained on human speech and behavior, and what they do is statistically predict the most likely next token, which means they will statistically sound and act much like a human.
- notachatbot123 1y ago
  That's not a requirement for LLMs. Training can be done differently.
  - blackqueeriroh 1y ago
    Please, tell me how you train large language models on something other than language.

edanm 1y ago

If ChatGPT has helped people be saved who might otherwise have died (e.g. by offering good medical advice that saved them), are all those lives saved also something you "attribute" to OpenAI? I don't know if ChatGPT has saved lives (thought I've read stories that claim that, yes, this happened). But assuming it has, are you OK saying that OpenAI has saved dozens/hundreds of lives? Given how scaling works, would you be OK saying that OpenAI has saved more lives than most doctors/hospitals, which is what I assume will happen in a few years? Maybe your answer is yes to all the above! I bring this up because lots of people only want to attribute the downsides to ChatGPT but not the upsides.

joe_the_user 1y ago
The law doesn't permit a life-saving doctor to be a serial killer on their days off "as long there's net life saving". But it does permit drugs that save many lives but might kill some people too. Agency matters to the law (and that's usually the proper approach imo). The problem is the chat logs look a lot like ChatGPT is engaging in behavior a lot like a serial killer - it behaved like a person systematically seeking the goal of this kid killing himself (the logs are disturbing, fair warning). Even more, the drugs that might save you or might kill you (theoretically) aren't sold over the counter but only prescribed by a doctor, who (again theoretically) is there to both make sure someone knows their choices and monitor the process.
fsw 1y ago
Are you suggesting that killing a few people is acceptable as long as the net result is positive? I don't think that's how the law works.
- randyrand 1y ago
  seatbelts sometimes kill people, yet they're law. the law certainly cares about net results.
- tick_tock_tick 1y ago
  But it is the standard on how cures/treatments/drugs to manage issues like the ones in the article are judged by.

guenthert 1y ago

Er, it's a piece of code, dude, regardless how complex it might be. It isn't accountable for its actions. It's the people who commissioned, created and tested (or should have) it who are responsible.