5 ms·

Copilot turns a court reporter into a child molester

72 points by nolist_policy 2y ago

vargr616 2y ago
[flagged]
- amval 2y ago
  What do you mean, exactly?
- acdha 2y ago
  Do you mean Microsoft? Or are you saying there’s some kind of understanding which makes it okay for entering someone’s name into a Bing search box and see false accusations of serious crimes?
- bunnyfoofoo 2y ago
  It shouldn’t be allowed to respond with false information about a person being a child molester, and be unable to be corrected. If that’s not possible then it shouldn’t be legal.
  - zarzavat 2y ago
    It’s a bit like proposing to ban cars if someone drives one into a pedestrian isn’t it? LLMs are computer programs, tools. They’re not sentient, they don’t know what’s true or false. The error is entirely the human who chooses to put more than trivial weight into its output. Reminds me of the “talking to God” program in TempleOS. We are all Terry now.
    - vundercind 2y ago
      The tech should be legal, of course. If you want to run a program that sometimes tells dangerous lies to you, sure, whatever. Running a service that prints dangerous lies about people should come with so much liability it’s economically infeasible, though. Then again, I have similar feelings about ad networks and social media “feeds”.
    - pjc50 2y ago
      OK, fine, we can solve this with the informational equivalent of the Proposition 65 notice: "this content is the result of an LLM. It may contain falsehoods or libels. You are prohibited from relying on this information, and if you republish it you assume liability for any falsehoods". (remember the Java license clause about not using it for nuclear reactors?) Maybe we need a Data Protection Act adjustment: before using an LLM, the individual entering the prompt needs to be registered as a data controller, and needs to secure the consent of all individuals whose names appear in the LLM data?
    - pavel_lishin 2y ago
      > It’s a bit like proposing to ban cars if someone drives one into a pedestrian isn’t it? It's closer to banning false product labeling, which we do.
    - MissTake 2y ago
      No, that’s at best a bad reason fallacy, at worst a slippery slope.
    - acdha 2y ago
      That comparison doesn’t fit the situation: Microsoft is liable for their LLMs just like they’re liable for their corporate vehicles. If their company cars crash into pedestrians they don’t get to say “oh, that just happens sometimes” and shrug it off.
OJFord 2y ago
Paediatricians are somewhat frequently^ the victim of idiot attackers who think they are (based on the prefix of that job) paedophiles. LLMs giving confidently incorrect information like this is so much worse, takes much less of an idiot to take it at face value, and if so minded to attack the innocent journalist (or anyone similarly associated with the real criminals, as the article points out). (^I mean, relative to the incidence of paedophilia in the field, or certainly to attacks on other professions based on misguided assumptions; far too frequently, several occurrences in the last 24 years it seems (I was initially just wanting to check details on the one case I dimly recalled), but not like it's happening every week.)
- floam 2y ago
  I’ve never heard of a pediatrician being read as a pedophile. Is this somehow specific to the UK spelling or UK culture? There are cases where I’ve noticed British English speakers use truncated forms where Americans don’t. Like “uni” for university, or “veg” - is it the case that paediatricians/pediatricians are referred to as paedos/pedos and that causes ambiguity? That would be unfortunate.
  - beardyw 2y ago
    > is it the case that paediatricians/pediatricians are referred to as paedos/pedos No. It's just stupidity.
  - OJFord 2y ago
    No it's just barely literate idiot thugs. They're not pronounced or spelt any more similarly than in AmE or anything. It's just 'oh paediatrician that's what paedo is short for innit'. Some examples: https://pressgazette.co.uk/news/a-tale-told-too-much-the-paediatrician-vigilantes/ https://pressgazette.co.uk/news/a-tale-told-too-much-the-pae... http://news.bbc.co.uk/2/hi/uk_news/magazine/4719364.stm http://news.bbc.co.uk/2/hi/uk_news/magazine/4719364.stm https://www.independent.co.uk/news/uk/this-britain/vigilante-mob-attacks-home-of-paediatrician-710864.html https://www.independent.co.uk/news/uk/this-britain/vigilante... https://www.telegraph.co.uk/news/uknews/1353904/Paediatrician-attack-People-dont-want-no-paedophiles-here.html https://www.telegraph.co.uk/news/uknews/1353904/Paediatricia... (there might be some overlap there, I haven't actually read more than necessary to check they're not cases of the paediatrician actually being a paedophile too, there is some of that unfortunately)
pjc50 2y ago
Search engines have long used the defence that they're not publishers, they merely guide you to publications by other people. Replacing search with LLM changes that. Who's the publisher when Bing publishes the output of an LLM? I would say that it's Microsoft. And therefore Microsoft are liable when their machine libels people.
- acdha 2y ago
  I think this is going to be a billion dollar legal question, and that you’re right. Companies really want to cut jobs but if precedents like that Air Canada case hold that statements by chatbots are legally binding and they lose the platform defense for misinformation that’s going to really sharply limit adoption for anything public facing.
  - pjc50 2y ago
    > Air Canada case hold that statements by chatbots are legally binding I think that's the only outcome that makes any legal sense. Corporations may be able to buy favourable verdicts, and weak US consumer protection will help them, but EU jurisdictions are not going to let you run a machine that tells lies about people, products or services.
    - acdha 2y ago
      Agreed. I just hope that they aren’t too successful at evading responsibility here in the U.S. since moving is not easy.
- basch 2y ago
  if you have a calculator or a function, and you supply the input, who owns the output?
  - acdha 2y ago
    Exactly: once you choose to use it for your business, you have ownership of the results. The challenge here will be if companies try to convince judges or lawmakers that there’s something magical about LLMs which mean they shouldn’t be liable for their business decisions.
throwaway984393 2y ago
[dead]
nottorp 2y ago
[flagged]
deleted 2y ago
[deleted]
polotics 2y ago
This is interesting: "people in the EU also have the right to access the information stored about them under the GDPR" I think I am going to have to ask OpenAI about all the weights that are related to me. You should do the same.

72 points by nolist_policy 2y ago

vargr616 2y ago

[flagged]

amval 2y ago
What do you mean, exactly?
acdha 2y ago
Do you mean Microsoft? Or are you saying there’s some kind of understanding which makes it okay for entering someone’s name into a Bing search box and see false accusations of serious crimes?
bunnyfoofoo 2y ago
It shouldn’t be allowed to respond with false information about a person being a child molester, and be unable to be corrected. If that’s not possible then it shouldn’t be legal.
- zarzavat 2y ago
  It’s a bit like proposing to ban cars if someone drives one into a pedestrian isn’t it? LLMs are computer programs, tools. They’re not sentient, they don’t know what’s true or false. The error is entirely the human who chooses to put more than trivial weight into its output. Reminds me of the “talking to God” program in TempleOS. We are all Terry now.
  - vundercind 2y ago
    The tech should be legal, of course. If you want to run a program that sometimes tells dangerous lies to you, sure, whatever. Running a service that prints dangerous lies about people should come with so much liability it’s economically infeasible, though. Then again, I have similar feelings about ad networks and social media “feeds”.
  - pjc50 2y ago
    OK, fine, we can solve this with the informational equivalent of the Proposition 65 notice: "this content is the result of an LLM. It may contain falsehoods or libels. You are prohibited from relying on this information, and if you republish it you assume liability for any falsehoods". (remember the Java license clause about not using it for nuclear reactors?) Maybe we need a Data Protection Act adjustment: before using an LLM, the individual entering the prompt needs to be registered as a data controller, and needs to secure the consent of all individuals whose names appear in the LLM data?
  - pavel_lishin 2y ago
    > It’s a bit like proposing to ban cars if someone drives one into a pedestrian isn’t it? It's closer to banning false product labeling, which we do.
  - MissTake 2y ago
    No, that’s at best a bad reason fallacy, at worst a slippery slope.
  - acdha 2y ago
    That comparison doesn’t fit the situation: Microsoft is liable for their LLMs just like they’re liable for their corporate vehicles. If their company cars crash into pedestrians they don’t get to say “oh, that just happens sometimes” and shrug it off.

OJFord 2y ago

Paediatricians are somewhat frequently^ the victim of idiot attackers who think they are (based on the prefix of that job) paedophiles. LLMs giving confidently incorrect information like this is so much worse, takes much less of an idiot to take it at face value, and if so minded to attack the innocent journalist (or anyone similarly associated with the real criminals, as the article points out). (^I mean, relative to the incidence of paedophilia in the field, or certainly to attacks on other professions based on misguided assumptions; far too frequently, several occurrences in the last 24 years it seems (I was initially just wanting to check details on the one case I dimly recalled), but not like it's happening every week.)

floam 2y ago
I’ve never heard of a pediatrician being read as a pedophile. Is this somehow specific to the UK spelling or UK culture? There are cases where I’ve noticed British English speakers use truncated forms where Americans don’t. Like “uni” for university, or “veg” - is it the case that paediatricians/pediatricians are referred to as paedos/pedos and that causes ambiguity? That would be unfortunate.
- beardyw 2y ago
  > is it the case that paediatricians/pediatricians are referred to as paedos/pedos No. It's just stupidity.
- OJFord 2y ago
  No it's just barely literate idiot thugs. They're not pronounced or spelt any more similarly than in AmE or anything. It's just 'oh paediatrician that's what paedo is short for innit'. Some examples: https://pressgazette.co.uk/news/a-tale-told-too-much-the-paediatrician-vigilantes/ https://pressgazette.co.uk/news/a-tale-told-too-much-the-pae... http://news.bbc.co.uk/2/hi/uk_news/magazine/4719364.stm http://news.bbc.co.uk/2/hi/uk_news/magazine/4719364.stm https://www.independent.co.uk/news/uk/this-britain/vigilante-mob-attacks-home-of-paediatrician-710864.html https://www.independent.co.uk/news/uk/this-britain/vigilante... https://www.telegraph.co.uk/news/uknews/1353904/Paediatrician-attack-People-dont-want-no-paedophiles-here.html https://www.telegraph.co.uk/news/uknews/1353904/Paediatricia... (there might be some overlap there, I haven't actually read more than necessary to check they're not cases of the paediatrician actually being a paedophile too, there is some of that unfortunately)

pjc50 2y ago

Search engines have long used the defence that they're not publishers, they merely guide you to publications by other people. Replacing search with LLM changes that. Who's the publisher when Bing publishes the output of an LLM? I would say that it's Microsoft. And therefore Microsoft are liable when their machine libels people.

acdha 2y ago
I think this is going to be a billion dollar legal question, and that you’re right. Companies really want to cut jobs but if precedents like that Air Canada case hold that statements by chatbots are legally binding and they lose the platform defense for misinformation that’s going to really sharply limit adoption for anything public facing.
- pjc50 2y ago
  > Air Canada case hold that statements by chatbots are legally binding I think that's the only outcome that makes any legal sense. Corporations may be able to buy favourable verdicts, and weak US consumer protection will help them, but EU jurisdictions are not going to let you run a machine that tells lies about people, products or services.
  - acdha 2y ago
    Agreed. I just hope that they aren’t too successful at evading responsibility here in the U.S. since moving is not easy.
basch 2y ago
if you have a calculator or a function, and you supply the input, who owns the output?
- acdha 2y ago
  Exactly: once you choose to use it for your business, you have ownership of the results. The challenge here will be if companies try to convince judges or lawmakers that there’s something magical about LLMs which mean they shouldn’t be liable for their business decisions.

throwaway984393 2y ago

[dead]

nottorp 2y ago

[flagged]

deleted 2y ago

[deleted]

polotics 2y ago

This is interesting: "people in the EU also have the right to access the information stored about them under the GDPR" I think I am going to have to ask OpenAI about all the weights that are related to me. You should do the same.