6 ms·
ChatGPT provides false information about people, and OpenAI can't correct it
- esher 2y agoPopcorn time: NOYB (Max Schrems) filed a complaint against OpenAI with the Austrian DPA: ChatGPT is not GDPR compliant.
- oldpersonintx 2y agoSounds like a win for the US...get competing economies to block the technology for trivial reasons, then by time the bugs are worked out they will be so far behind that their only choice will be to choose US-based solutions
- openplatypus 2y ago> trivial reasons Come again?
- deleted 2y ago[deleted]
- DonHopkins 2y agoYou're suffering from ChatGPT Derangement Syndrome.
- rsynnott 2y agoYes; the US will lead the world in plausible-looking bullshit generation.
- mrtksn 2y agoIMHO this is not a fixable problem, it's not even a new problem because false information existed since ever. With the Web it become globally available, with social media it become globally spreadable and with "AI" it has become of endless supply in form and variety. Humans need to develop humane defense mechanisms for the new reality, tech cannot be stopped and cannot be hermetically insulated against mistakes or bad actors.
- brianjacobss 2y agoYou're totally right.We can't just hit the brakes on technology. It's here to stay, with its flaws and perks. As users, we need to be savvy about what we see online, questioning stuff instead of just swallowing it whole. Education is key here. The more we can help people navigate digital sources of information, the better equipped they'll be to identify misinformation.
- andersa 2y agoWhy do all of your comments look like ChatGPT wrote them?
- Mordisquitos 2y agoWhy is it "not a fixable problem"? So many problems introduced by new technologies have been claimed to be "non fixable" by the businesses promoting them until, lo and behold, the law has forced them to do so.
- mrtksn 2y agoWas lying online fixed? The attempts to fix it come at great costs, which is usually suppression of speech which leads to all kind of problems associated with lack of information or feeding faulty information. Attempts fixing hate speech on social media resulted in similar situation, now hate becoming mainstream. American social media banned keywords for racism or hate speech, only to push them to dog whistle racism and hate speech. The problem about fixing lie is that its impossible to objectively decide what is a lie and this is true for all kind of stuff about people.
- Mordisquitos 2y agoYes, it was.
- mrtksn 2y agoSee, one lie just sneaked in.
- imdsm 2y agoThat's because it's being used in the wrong way. It isn't a factual database.
- kromem 2y agoIt sometimes feels like I've taken crazy pills watching what was effectively a tech demo that went viral become the usecase now dictating billions of dollars of development and optimization. It's a crappy usecase. And much better ones are typically being overlooked outside a few smart enterprise integrations. To put it mildly - if someone wants to use LLMs to build a factual chatbot, they should probably just start mining crypto instead, as they'll waste less money on jumping on a trend. But if they think a bit about how LLMs can be used in nearly any other situation, they'll be miles ahead of the majority chasing this gold rush.
- trustno2 2y agoAI pushers are presenting them as factual databases. REpeatedly.
- Squeeze2664 2y agoThis seems so backwards to me. As a user of LLMs, it's clear to me that tokens generated by ChatGPT and similar are not to be interpreted as a legally-valid statement by the company making the LLMs, unless explicitly stated by said company. I certainly don't engage with them that way. I believe this to be fairly obvious to anyone who has used such tools, so I just see this as an opportunity to sue for a quick buck.
- stavros 2y agoNOYB doesn't make money out of suing. I otherwise agree with the rest of your comment.
- Squeeze2664 2y agoI just looked into what noyb is, thank you for pointing this out. Perhaps this is something that had to be brought to court at one point or another, so we can set a precedent one way or the other, then. At the moment I think I'm hoping nothing comes out of this.
- stavros 2y agoYeah, me too, and I have a recurring donation to NOYB, as they generally do fantastic work. I think this one is a rare miss.
- BlackFly 2y agoAre the statements by the neighbourhood drunk in the corner pub commonly interpreted as legally valid statements by the individual making them? Nobody takes the drunk seriously, yet the neighbourhood drunk shouldn't be treated more seriously with respect to slander than a large corporation with respect to libel.
- latexr 2y ago> I believe this to be fairly obvious to anyone who has used such tools It is not. Not even people in tech understand this, let alone non-technical people. These tools are being marketed as a way to get factual information. Don’t let your knowledge of the technology blind you to the fact that people outside your circle don’t know what you do.
- chx 2y agoEvery single time... https://hachyderm.io/@inthehands/112006855076082650 https://hachyderm.io/@inthehands/112006855076082650 > You might be surprised to learn that I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” can be a genuinely helpful question to have answered, in code and in natural language — for brainstorming, for seeing common conventions in an unfamiliar context, for having something crappy to react to. > Alas, that does not remotely resemble how people are pitching this technology.
- trustno2 2y agoNobody put what I think about AI better than this. Yes, there are good cases where "realistically sounding bullshit" is useful! I remember in the early days, before everyone grew kind of tired (well, at least me) of GPT, people used "hehe write me PC manual in style of slam poetry" or whatever, and that is fun. However, people use it to get facts! Just a few days ago, I have read somewhere that hospitals are planning to use LLMs to replace nurses. That is horrifying. Absolutely horrifying.
- TheRoque 2y agoPeople use it to get facts, and trust it more than Google. I have some tech illiterate boss that asked me to do stuff some way because "ChatGPT said so", instead of trusting me, an experience professional. It wasn't like this with a Google search, so why now ? Natural language has a big impact on how the product is perceived
- krapp 2y agoWe've seen numerous stories at this point about lawyers trusting AI to generate case documents that turned out to have false cites - AI generated scientific papers are being published. Doctors are using AI. Law enforcement is using AI. Everyone is using it and a lot of people are using it with the assumption that it's intelligent and factual. That it works like the computer from Star Trek. People on this very forum who should know better have said they trust AI more than they trust themselves and more than other people. AI probably has a niche where it's useful, but because it smells like a magic money machine that will allow managers to replace employees and create value from essentially nothing, modern capitalism dictates we must optimize our entire economy around it, no holds barred, damn the torpedos and full speed ahead because "money." I just hope the fever breaks before people start getting killed.
- LASR 2y agoPeople have a fundamental misunderstanding of what LLMs are. I’ve had to explain this so many times even to engineers. People keep using it as Google. It is not a mechanism to retrieve facts. It’s rather a reasoning mechanism, that when used like a search engine generates text that looks like output from a retrieval of facts. The article talks about OpenAI being unwilling to correct errors. But they just can’t. There just aren’t facts like birthdays for specific people discernible from the weights. Maybe for some. So what would they correct? The best that can be done is like it is said in the article, apply filtering to refuse to answer.
- deleted 2y ago[deleted]
- Dalewyn 2y ago>People have a fundamental misunderstanding of what LLMs are. It's labeled "AI", aka "artificial intelligence". So clearly, it's intelligent enough to tell fact from fiction, right? It's intelligent enough to read and repeat facts, right? It's artificial intelligence, isn't it? I'm playing Devil's Advocate here, obviously. I'm aware how this actually works like you, but the way it's billed you can't expect the commons to understand any of this in any other way. There is a very specific concept of what AI is among the masses, regardless if "AI" is it or not. If you call something a duck, you can't blame someone for saying it's a duck.
- alpaca128 2y agoEven real intelligence struggles to tell facts from fiction, otherwise the world would look very different. That's yet another problem with AI, people often aren't sure what they can expect but many err on the side of "magical oracle".
- raincole 2y agoIs Gemini that unpopular? It seems that nobody in this thread is aware of its features. The best that can be done is just what Gemini does: a button to (smartly) compare the generated text against google search results.
- 2y ago
- phantomathkg 2y ago<sarcasm>noyb.eu provides false information about ChatGPT, will noyb.eu correct it?</sarcasm>
- excusemyfrench 2y agoAm I wrong or LLMs are text not fact generators ?
- blitzar 2y ago"We" ask them to generate text that includes the facts.
- subroutine 2y agoYou can make LLMs say pretty much whatever you want with the right prompts. This is a complex issue, and if EU citizens want access to LLMs the GDPR is going to need a different set of rules for LLMs than for websites and search engines.
- bux93 2y agoMeh. GDPR sets limits on 'processing' personally identifiable information. In the context of an LLM, its outputs may contain PII if its inputs do. Those inputs are training input and prompts. So long(!) as the training input doesn't have PII, the output will only have it if the prompts do. Same as if you save a file on onedrive, if you save PII there, you're the data controller, and Microsoft is a processor on your behalf.
- Vuizur 2y agoIt is impossible to remove personal data ("any information which are related to an identified or identifiable natural person") from the LLM training data. As far as I understand it ChatGPT and all other similar systems are blatantly violating GDPR, they would have to for example publish their related training data to conform. I guess the EU authorities don't do anything for now because they don't want to admit that their funny law basically bans all state-of-the-art AI. (Ok, Openai also broke the law in almost all countries by downloading shadow libraries, but here they at least have more plausible deniability.)
- Mordisquitos 2y agoIf LLM providers want access to the EU market they will need to find a way to comply with GDPR, and if OpenAI cannot find a way to do it then a different LLM provider will.
- subroutine 2y ago> the GDPR requires information about individuals is accurate Given that you can make LLMs say pretty much whatever you want using the right prompts, this seems impossible. LLMs are not a search engine, and based on conversational context might say Emmanuel Macron is the president of France or a baby giraffe.
- palad1n 2y agoI wonder if my insight on these is wrong, but from what I understand, the GPTs and such are basically all hallucinations, and it's just that most of these are also correlated to reality, what actually exists.
- r_c_a_d_t_s 2y agoIt's whatever is in the training data, plus some hacks. For example, if I ask ChatGPT when my birthday is it says "I don't have personal information on individuals"; but if I ask when King Charles III birthday is it tells me a date... which matches what's in Wikipedia... so it might be right. If there are lots of instances in the training data where the date is wrong, then it will just repeat that wrong date back to you as fact.
- Kwpolska 2y agoIt might be able to produce true data about very famous individuals, and it might refuse to provide information about unknown individuals. But if you ask about a YouTuber, or a smaller star/celebrity, it is more likely to produce a false statement. I asked about the birthday of Tom Scott (the YouTuber) three times and got three different dates (and none supported by a Google search).
- r_c_a_d_t_s 2y agoYes, I agree. That's what I was trying to say. It also "doubles down" when called out: I asked about the mathematician Richard Taylor and it responded with a paragraph that called him Sir Richard Lawrence Taylor, so I asked when he was knighted (he wasn't) and it said "The 2014 birthday honours list" which is wrong... probably because there was a Dr Richard Thomas Taylor awarded an MBE (not a knighthood) in that list.
- Kwpolska 2y agoChatGPT will confidently fabricate data out of thin air, it is less likely to admit it does not know something or is not sure. And that’s the primary problem with it: the statement is believable, but incorrect.
- ShamelessC 2y agoDidn’t the media cover this already over and over since ChatGPT’s inception? Not that it isn’t important but to imply it is a new revelation seems sensationalist. > While inaccurate information may be tolerable when a student uses ChatGPT to help him with their homework, it is unacceptable when it comes to information about individuals. I don’t even understand this part. Why would inaccurate information be at all acceptable for homework? You need accurate information there just as much as with people. In fact there’s probably a good deal of overlap. This article is bad.
- latexr 2y ago> Didn’t the media cover this already over and over since ChatGPT’s inception? This is not “the media”. noyb isn’t reporting on what other people did, they are informing us of what they just did. > Not that it isn’t important but to imply it is a new revelation seems sensationalist. The article isn’t about the failings of ChatGPT, it’s about a specific legal complaint being made. > Why would inaccurate information be at all acceptable for homework? It’s acceptable in a legal sense, you get a bad grade and that’s that. But falsehoods about a particular individual on a popular resource could be quite damaging and are against EU law. > This article is bad. Rather, you completely misunderstood it.
- ShamelessC 2y agoI mean, yeah I was having a tough time understanding it. I felt like my comment conveyed my confusion pretty well. I still think the wording is strange and the most clarifying thing you’ve provided is that this guy is a blogger and his target audience is far less broad than HN encapsulates. Sorry if you took offense to my saying the article was bad. I regret saying that now, it was unnecessary.
- latexr 2y ago> this guy is a blogger and his target audience is far less broad than HN encapsulates. It’s not one blogger, noyb (stands for “None Of Your Business”) is a non-profit focused on protecting privacy rights in the EU. https://en.wikipedia.org/wiki/NOYB https://en.wikipedia.org/wiki/NOYB > I regret saying that now, it was unnecessary. Thank you for saying that. Especially on the internet where we all have the compulsion to double down, I believe those types of admissions take guts and should be celebrated and normalised.
- sharpshadow 2y agoWe here in Germany, as so other countries, have a issue with Wikipedia and their actions against a certain clientele. If you get branded a conspiracy theorist your Wikipedia entry will denounce you as such and the admins and moderators will lock the article and you can do nothing about it. Obviously ChatGPT will take information about your persona from Wikipedia if available and will update this information accordingly to the changes in Wikipedia. And you can’t do anything about it.
- oguz3 2y agoChatGPT is not a magical database that provides correct answers to all questions. It is an LLM that draws conclusions and propositions from the context you offer. Instead of just asking a question and waiting for an answer, you can provide detailed context and engage in a discussion about it.
- eclectic29 2y agoGlad that people are realizing the ChatGPT/AI hype that is driving all companies nuts by creating unnecessary peer pressure to integrate some kind of GenAI in their products even if it doesn’t make sense.
- krapp 2y agoThey've always been around, they've just been dismissed as "Luddites" who fear an inevitable future. But of course, the Luddites are always proven correct in hindsight.
- ahstilde 2y agowhen has your claim "the Luddites are always proven correct in hindsight." been true?
- krapp 2y agoAlways. And every now and then HN realizes it, then promptly forgets. Contrary to popular propaganda, the original Luddites weren't opposed to technology, they were opposed to the effect of technological progress on the working class. They knew that automation and mass production would be used to devalue labor, flood the market with inferior products, and that all of the benefits and profit from the industrial revolution would go to the corporations, at the expense of their quality of life. And they were correct. And modern day "Luddites" were correct about the centralization and commoditization of the web, social media, crypto and NFTs, Elon Musk and autonomous vehicles, and will be proven right about AI. Tech has nothing left to offer but grift upon grift. https://www.newyorker.com/books/page-turner/rethinking-the-luddites-in-the-age-of-ai https://www.newyorker.com/books/page-turner/rethinking-the-l... https://news.ycombinator.com/item?id=37664682 https://news.ycombinator.com/item?id=37664682 https://librarianshipwreck.wordpress.com/2018/01/18/why-the-luddites-matter/ https://librarianshipwreck.wordpress.com/2018/01/18/why-the-... https://news.ycombinator.com/item?id=17053994 https://news.ycombinator.com/item?id=17053994
- Lerc 2y agoDoes the GDPR require information that is not asserted to be factual to actually be factual? If I have a random number generator producing arbitrary strings, am I required to ensure that the strings do not contain untrue statements about individuals?
- croes 2y agoWhen people treat the information as factual and the company doesn't do enough to clarify that, then yes.
- Lerc 2y agoSo you would agree that rather than this being a case of a definite violation of GDPR rules this would entirely hinge on what would constitute enough? The fact that LLMs hallucinate is certainly no secret, even the linked article says OpenAI openly admits that they can't avoid it right now. What would constitute enough? Would it be enough to place a statement placed onscreen at the start of every conversation to say that information may not be accurate and that if the information was significant then it should be independently verified?
- deleted 2y ago[deleted]
- mediumsmart 2y agoI like chatgpt a little more after reading this.