3 ms·

This case illustrates a fundamental misunderstanding about how generative AI works. There is no data supporting the false story. It’s literally made up the same

This case illustrates a fundamental misunderstanding about how generative AI works. There is no data supporting the false story. It’s literally made up the same way you or I might make up any story we like without reference to real facts. OpenAI can no more “delete the data” than they can prevent any human from making anything up. What they can do is try to prevent their AI from making up harmful stories in the future. That’s it. There’s nothing to delete.

taylodl 2y ago
Tort courts are going to have a heyday with this!
burkaman 2y ago
Yes but this is also the common understanding of how generative AI works. Most people who use ChatGPT think that it knows things and can answer questions accurately. It is valuable for AI developers to read stories like this so they can see how normal people view their technology. I agree that OpenAI can't "delete the data" without deleting the whole model, although they could put in a manual filter on this guy's name. I'm sure he wouldn't mind if ChatGPT just refused to produce any output about him at all.
- Suppafly 2y ago
  They went a step further and made it so if you search for person, it does a google search or similar and provides that data instead (at least according to the article).
  - burkaman 2y ago
    It does not consistently do this. I just tried "who is Arve Hjalmar Holmen" again and it said: > Arve Hjalmar Holmen is a Norwegian man who gained attention due to his involvement in a notorious criminal case. He was convicted in 2019 for his role in a large-scale fraud scheme, which involved selling non-existent apartments in Norway. Holmen, along with accomplices, deceived victims into paying for apartments that didn’t exist, leading to significant financial losses for those involved. > If you're referring to a different person with the same name, or need more details on the case, feel free to clarify! If I keep asking it alternates between "I don't know anything about this guy" and making up a different crime every time (like "He was involved in the high-profile Norsk Tippings fraud case."). It's honestly bizarre, I can't get it to do this with any other non-famous name. This is all within the same chat session, previous answers are in the context window and it's still making up brand new crimes.
- blibble 2y ago
  the manual filter is great, you can then say his name in Teams and then it disables copilot everywhere (meetings, conversations, group chats)
arduanika 2y ago
But your honor, my machine I made is incapable of not committing this crime. You misunderstand how it works, you dolt. Randomly committing crimes sometime is just the future of technology.
juancn 2y ago
The misleading info, yeah, I'll grant you that comes from somewhere else, but what about PII? GDPR requires you to delete the data if I ask, both the training data and the derived data (i.e. the model), so if I do a GDPR request to OpenAI needs to delete the data and retrain to be compliant. It's not enough to "block the output" to follow GDPR, they really need to wipe the weights if I understand the law correctly.

taylodl 2y ago

Tort courts are going to have a heyday with this!

burkaman 2y ago

Yes but this is also the common understanding of how generative AI works. Most people who use ChatGPT think that it knows things and can answer questions accurately. It is valuable for AI developers to read stories like this so they can see how normal people view their technology. I agree that OpenAI can't "delete the data" without deleting the whole model, although they could put in a manual filter on this guy's name. I'm sure he wouldn't mind if ChatGPT just refused to produce any output about him at all.

Suppafly 2y ago
They went a step further and made it so if you search for person, it does a google search or similar and provides that data instead (at least according to the article).
- burkaman 2y ago
  It does not consistently do this. I just tried "who is Arve Hjalmar Holmen" again and it said: > Arve Hjalmar Holmen is a Norwegian man who gained attention due to his involvement in a notorious criminal case. He was convicted in 2019 for his role in a large-scale fraud scheme, which involved selling non-existent apartments in Norway. Holmen, along with accomplices, deceived victims into paying for apartments that didn’t exist, leading to significant financial losses for those involved. > If you're referring to a different person with the same name, or need more details on the case, feel free to clarify! If I keep asking it alternates between "I don't know anything about this guy" and making up a different crime every time (like "He was involved in the high-profile Norsk Tippings fraud case."). It's honestly bizarre, I can't get it to do this with any other non-famous name. This is all within the same chat session, previous answers are in the context window and it's still making up brand new crimes.
blibble 2y ago
the manual filter is great, you can then say his name in Teams and then it disables copilot everywhere (meetings, conversations, group chats)

arduanika 2y ago

But your honor, my machine I made is incapable of not committing this crime. You misunderstand how it works, you dolt. Randomly committing crimes sometime is just the future of technology.

juancn 2y ago

The misleading info, yeah, I'll grant you that comes from somewhere else, but what about PII? GDPR requires you to delete the data if I ask, both the training data and the derived data (i.e. the model), so if I do a GDPR request to OpenAI needs to delete the data and retrain to be compliant. It's not enough to "block the output" to follow GDPR, they really need to wipe the weights if I understand the law correctly.