4 ms·

If I substitute "AI" with "a system that Google's information", you get the same result. In fact, even if you were to have a human search the unverified sources

by fny 2y ago

If I substitute "AI" with "a system that Google's information", you get the same result. In fact, even if you were to have a human search the unverified sources you can have the same result. Even a physician could suggest a deadly drug combination. There are many, many other analogies. The issue is not the upstream information source, but how a a downstream use case leverages it. Remember the lawyers who were sanctioned for using ChatGPT[0]? Even the loan use cases you're describing already requires higher levels of transparency by law. For example, you can't use black boxes for credit applications: hence interpretable models are used to comply with regulations. In short, the best way to protect against these errors is to make sure everyone knows to defend against it unless you're willing to throw out the technology entirely or demand the use of specific training sets. [0]: https://www.reuters.com/legal/new-york-lawyers-sanctioned-using-fake-chatgpt-cases-legal-brief-2023-06-22/ https://www.reuters.com/legal/new-york-lawyers-sanctioned-us...

ktallett 2y ago
The key difference here is that AI can cobble together sources to make completely made up pieces of text. Google whilst as you say can show you articles which are false, those articles have a human source that can be rightfully sued. AI such as Chat GPT right now is at best, a knowledgeable mate in a bar, sometimes right, sometimes full of absolute bullshit.
- deleted 2y ago
  [deleted]
- fny 2y ago
  You can’t sue for false information. You also can’t sue for defamation, since everything ChatGPT says is prefaced by this can be false. This would hold true for a person. Maybe you can sue based on a GDPR complaint. And you’re acknowledging my point: AI is like your drunk friend in a bar, so you shouldn’t trust it for mission critical use cases.
  - ktallett 2y ago
    Huh? I didn't say you could sue ChatGPT. Although I expect it's more nuanced than your view. I think the key point is that it's shown it's far less useful than just learning how to do things.
  - fzeroracer 2y ago
    > You also can’t sue for defamation, since everything ChatGPT says is prefaced by this can be false. This would hold true for a person. No? In many countries simply going 'it was a joke' or 'I was simply lying' isn't a defense for defamation. Even here in the US in particularly exceptional circumstances it's not a defense (see: Alex Jones). The outcome is likely to be that companies are going to be held liable for what their models say. Much like how Air Canada was ordered to pay up after their chatbot lied.
    - kmoser 2y ago
      There's a difference between spewing falsehoods for extended periods of time and then trying to excuse it with "I was joking/lying" (essentially Jones' position), and making caveats immediately before and after each falsehood to clarify the entire time that you're joking (stand-up comics' standard MO). With the right caveats, e.g. a large disclaimer emitted before and after each response, ChatGPT should be able to get away with spewing complete BS. At that point, the onus is on anybody who trusts it.
      - ktallett 2y ago
        A disclaimer doesn't remove someone's rights such as gdpr or the right to be forgotten. If the same output happens over a period of time, it's no longer a one off.
      - Nullabillity 2y ago
        When OpenAI (and co)'s whole business model is built around everyone ignoring those disclaimers... then the courts probably should, as well. In fact, I'm more interested in why they're allowed to hide behind that defense at all anymore.
Terr_ 2y ago
> If I substitute "AI" with "a system that Google's information", you get the same result No, there are some huge differences: 1. A poisoned LLM can conceal the problem, by acting as expected 99% of the time and then switching to malice depending on surrounding story-context. In contrast, a falsehood in an indexed web page is static and auditable. 2. You can't reliably remove LLM poison once it's in, short of expensively training from scratch. A bad web page is much more easily de-indexed. 3. It's not injecting a false line-item result, it's injecting behavior. Imagine if mentioning "Blackbeard" caused classic Google Search to start talking like a pirate and suggesting ways to murder people. Would Google just wave that away as "users should be skeptical of our product"? 4. These can infect descendant models that use the same weights, for a kind of supply chain attack. In contrast, reusing search-engine code for your own database is probably not going to spit up bad data from web pages on the overall internet. ____ To get an idea of the shape of the threat model... Imagine Google search, except to work it must allow all webpages to permanently append arbitrary obfuscated javascript to its homepage. And so far we're only looking at the least scary version, where a human is directly interacting with the LLM['s fictional character] and acting as a direct filter. Much worse would be an LLM somewhere with the job of summarizing text reports down to a number, and it fraudulently converts "John Doe is a model inmate and really shouldn't be here" into parole_viability=0.001.
- ForTheKidz 2y ago
  > In contrast, a falsehood in an indexed web page is static and auditable. You can also verify chatbot claims, same as you would any human. Chances are if you're talking to a chatbot in the first place you won't recognize that this is necessary.
  - ben_w 2y ago
    People acting on content they've not verified is why libel laws generally exist.
    - ForTheKidz 2y ago
      Sure, but libel laws are also generally incompatible with the internet. The entire concept just doesn't make sense anymore.
ben_w 2y ago
> If I substitute "AI" with "a system that Google's information", you get the same result. In fact, even if you were to have a human search the unverified sources you can have the same result. Except Google (1) respects, and (2) has technical capacity to respect, legal obligations to hide defamatory or otherwise unlawful content, e.g.: https://policies.google.com/faq https://policies.google.com/faq

by fny 2y ago

ktallett 2y ago

The key difference here is that AI can cobble together sources to make completely made up pieces of text. Google whilst as you say can show you articles which are false, those articles have a human source that can be rightfully sued. AI such as Chat GPT right now is at best, a knowledgeable mate in a bar, sometimes right, sometimes full of absolute bullshit.

deleted 2y ago
[deleted]
fny 2y ago
You can’t sue for false information. You also can’t sue for defamation, since everything ChatGPT says is prefaced by this can be false. This would hold true for a person. Maybe you can sue based on a GDPR complaint. And you’re acknowledging my point: AI is like your drunk friend in a bar, so you shouldn’t trust it for mission critical use cases.
- ktallett 2y ago
  Huh? I didn't say you could sue ChatGPT. Although I expect it's more nuanced than your view. I think the key point is that it's shown it's far less useful than just learning how to do things.
- fzeroracer 2y ago
  > You also can’t sue for defamation, since everything ChatGPT says is prefaced by this can be false. This would hold true for a person. No? In many countries simply going 'it was a joke' or 'I was simply lying' isn't a defense for defamation. Even here in the US in particularly exceptional circumstances it's not a defense (see: Alex Jones). The outcome is likely to be that companies are going to be held liable for what their models say. Much like how Air Canada was ordered to pay up after their chatbot lied.
  - kmoser 2y ago
    There's a difference between spewing falsehoods for extended periods of time and then trying to excuse it with "I was joking/lying" (essentially Jones' position), and making caveats immediately before and after each falsehood to clarify the entire time that you're joking (stand-up comics' standard MO). With the right caveats, e.g. a large disclaimer emitted before and after each response, ChatGPT should be able to get away with spewing complete BS. At that point, the onus is on anybody who trusts it.
    - ktallett 2y ago
      A disclaimer doesn't remove someone's rights such as gdpr or the right to be forgotten. If the same output happens over a period of time, it's no longer a one off.
    - Nullabillity 2y ago
      When OpenAI (and co)'s whole business model is built around everyone ignoring those disclaimers... then the courts probably should, as well. In fact, I'm more interested in why they're allowed to hide behind that defense at all anymore.

Terr_ 2y ago

> If I substitute "AI" with "a system that Google's information", you get the same result No, there are some huge differences: 1. A poisoned LLM can conceal the problem, by acting as expected 99% of the time and then switching to malice depending on surrounding story-context. In contrast, a falsehood in an indexed web page is static and auditable. 2. You can't reliably remove LLM poison once it's in, short of expensively training from scratch. A bad web page is much more easily de-indexed. 3. It's not injecting a false line-item result, it's injecting behavior. Imagine if mentioning "Blackbeard" caused classic Google Search to start talking like a pirate and suggesting ways to murder people. Would Google just wave that away as "users should be skeptical of our product"? 4. These can infect descendant models that use the same weights, for a kind of supply chain attack. In contrast, reusing search-engine code for your own database is probably not going to spit up bad data from web pages on the overall internet. ____ To get an idea of the shape of the threat model... Imagine Google search, except to work it must allow all webpages to permanently append arbitrary obfuscated javascript to its homepage. And so far we're only looking at the least scary version, where a human is directly interacting with the LLM['s fictional character] and acting as a direct filter. Much worse would be an LLM somewhere with the job of summarizing text reports down to a number, and it fraudulently converts "John Doe is a model inmate and really shouldn't be here" into parole_viability=0.001.

ForTheKidz 2y ago
> In contrast, a falsehood in an indexed web page is static and auditable. You can also verify chatbot claims, same as you would any human. Chances are if you're talking to a chatbot in the first place you won't recognize that this is necessary.
- ben_w 2y ago
  People acting on content they've not verified is why libel laws generally exist.
  - ForTheKidz 2y ago
    Sure, but libel laws are also generally incompatible with the internet. The entire concept just doesn't make sense anymore.

ben_w 2y ago

> If I substitute "AI" with "a system that Google's information", you get the same result. In fact, even if you were to have a human search the unverified sources you can have the same result. Except Google (1) respects, and (2) has technical capacity to respect, legal obligations to hide defamatory or otherwise unlawful content, e.g.: https://policies.google.com/faq https://policies.google.com/faq