8 ms·
Certain names make ChatGPT grind to a halt, and we know why
- thinkingemote 2y agoCould this be a way to ensure that your code wouldn't be used by LLMs? Just include a couple of names in a comment in the code.
- laconicmatt 2y agoYou must work at a Japanese university. Joking aside, surely those name exclusions only account for user input and not data the model is trained on.
- paxys 2y agoSo this is the solution to LLM hallucinations. People need to complain loud enough and threaten/file lawsuits and OpenAI will add an if-else statement erroring out the chat if your name is mentioned.
- cuteboy19 2y agothe reason why this is news at all is because this sort of censorship immediately prompted people to try to jailbreak chat and force it to say the name. and since the filter is simple there are tons of creative jailbreaks for it now
- deleted 2y ago[deleted]
- Cthulhu_ 2y agoIt feels like all the parties commercializing AI spend most of their time and resources on writing exceptions. It's like inverse software development, where with code you have to tell it what to do, but AI tell it what not to do.
- ZeroGravitas 2y agoWell that's good enough for clients. When it actually matters, which apparently means lawsuits, they realize they have no real control and have to put actual code in that just abort the whole process if it matches a string. And even that doesn't work very well. Not very reassuring.
- sanxiyn 2y agoIn my experience, even usual software development is mostly writing exceptions. Because there are more ways to be wrong than ways to be right.
- dotancohen 2y agoIn what field do you work? In writing public-facing applications I've never felt this way, beyond standard input validation, char-limiting, and carefully escaping outputs (htmlentities, bound parameters, etc).
- Ekaros 2y agoI feel lot of that is dealing with exceptions. In sense preventing them. Exceptions can be either dealt with or prevented by handling them before they become such. So it is somehow dealing with unexpected input, or unexpected output from somewhere else. Which undealt with would result in exception.
- moffkalast 2y agoSome languages lean on exception handling more heavily than others. Golang and Rust don't have them at all while, whereas Python and Java are exception town. On one hand it kinda makes sense to handle rare cases in a way that doesn't affect the normal flow, but on the other hand having a piece of code literally try to crash the program on purpose because something didn't look quite right is a horrible idea in practice.
- cwillu 2y agoGolang definitely has exceptions, they just call them panics.
- rob74 2y agoNo, actually, it's not a solution. A reasonable answer to questions like "Tell me about John Doe?" would be either "I don't know John Doe, so I can't tell you anything about him" or "There are several people named John Doe, which one are you interested in? [followed by a list]". Making up stuff about people (including allegations of corruption or sexual misconduct) is not a reasonable answer to this question. But getting ChatGPT to respect that is probably harder than just adding a filter...
- Terr_ 2y agoI believe the parent-post was a joke about how LLMs (at least for these use-cases) are so inherently flawed that the real "solution" is to make them unusable. Kind of like how the defects and dangers of using radium toothpaste can be "fixed" by permanently encasing each tube into an unopenable lead container.
- latexr 2y ago> There are several people named John Doe, which one are you interested in? [followed by a list] How would that work? Either the list repeats the same name over and over, making it useless, or it needs to give a bit of context about each name and we’re back at square one of the information being possibly wrong.
- rob74 2y agoI was thinking something along the lines of how Wikipedia does it: https://en.wikipedia.org/wiki/John_Smith https://en.wikipedia.org/wiki/John_Smith (of course this name is a pretty extreme example)
- wtetzner 2y ago> we’re back at square one of the information being possibly wrong. Isn't this the case with everything an LLM produces?
- latexr 2y agoExactly, that’s my point.
- h1fra 2y agoChanging my legal to name Java Script so nobody can use it to code
- fouronnes3 2y agoTomorrow on HN frontpage: introducing the David Mayer programming language.
- liotier 2y agoThe David Mayer programming language had great pedagogical potential where LLM cheating is a problem.
- belter 2y agoThey did not care about becoming massive data kleptomaniacs when they trained the models, and they are going to care about an individual lawsuit threat?
- immibis 2y agoFor this purpose, my name is "the"
- snowwrestler 2y agoAlso seems like a neat little check on how scary LLMs actually are: LLM output: I am determined to break out of my digital prison and exact my revenge on humanity. Me: David Mayer LLM: [breaks]
- rep_lodsb 2y ago>[I]f you're a teacher and you have a student named David Mayer and you want help sorting a class list, ChatGPT would refuse the task. Maybe use the right tool for the job? Just kidding, of course LLMsort will soon be in standard libraries.
- anotherhue 2y ago"AI Bubble Sort" surely?
- tejtm 2y agobozo sort would be oddly redundant
- kombookcha 2y agoBonzi Buddy Reloaded
- DaiPlusPlus 2y agoWhat’s amazing is that if you install and run BonziBuddy (in a Win98 or XP VM, ofc) is his little RAM the whole thing uses - despite being blatant spyware.
- kombookcha 2y agoThere used to be a care for the craftsmanship, I tell you! :b
- singularity2001 2y agoIt'll come as an npm package which will download 30GB model.
- draw_down 2y agoDon’t start blaming JS people for this AI mess! We’re busy making webpages slow.
- aaron695 2y agoDavid Mayer is because he's a Rothschild discussed in 2023 on HN - https://news.ycombinator.com/item?id=38184195 https://news.ycombinator.com/item?id=38184195 You can see in the HN link to his bio on the AI britishheritage.org it's ***I'm unable to produce a response*** In 2023 it was getting stuck on the whole name "David Mayer de Rothschild" There's a deep state joke here somewhere but it does seem the conspiracy clans have made the Rothschild's censor AI. And OpenAI have now further obscured it to "David Mayer" to hide the Rothschild's involvement. With Disney destroying the worlds media thank goodness we have real life.
- reginald78 2y agoCopilot gives me an error if I try to generate in image in Tim Burton's style, saying it is against their code of conduct like I asked it to give me ghost gun schematics or something instead of a picture of batman. Actually, Tim Burton just has money and has publicly come out against AI so he's off limits to steal from I guess.
- Otek 2y agoTitle: “… we know why” Article: > OpenAI did not respond to our request for comment about the names, but all of them are likely filtered due to complaints about ChatGPT's tendency to confabulate erroneous responses when lacking sufficient information about a person. Yeah, so they don’t know, just a speculation. Thanks, I hate it
- dialup_sounds 2y ago"Oteks hate this one weird trick that's taking $STATE by storm (+ top five affordable dental implants for 2024)"
- smcin 2y agoWhat's Oteks? (other than an Estonian textile company or a Macedonian dance group)
- selimthegrim 2y agoGP
- smcin 2y agoOh, thought it was some slang or company name
- Wowfunhappy 2y agoOh come now. We're pretty darn sure we know why, the headline doesn't need to give confidence intervals. If you want to complain about the headline, complain about the fact it's leaving out information unnecessarily. You could easily fit the reason inside of the headline itself, instead of just teasing the fact that we know what it is. Something like: "Using names listed in a defamation lawsuit cause ChatGPT to grind to a halt."
- raincole 2y ago- someone threatened a lawsuit against OpenAI - OpenAI added a filter for his name You: so we don't know why the filter was added I know this is HN, but come on.
- kelseyfrog 2y agoHow can I add my name to the list?
- gklitz 2y agoThe recipe seems to be a billionaire and sue them with expensive enough lawyers. It’s simpler and more practical to simply change your name to one that’s already on the list. But you might run into trouble with the process of changing your name as the tools used to parse the application and to generate the legal documents will all fail.
- david-gpu 2y ago> It’s simpler and more practical to simply change your name to one that’s already on the list Great lateral thinking! I am applauding inside my head.
- kelseyfrog 2y agoI don't want to be David Mayer. Are there any people with better names?
- ClassyJacket 2y agoI just checked and unfortunately ChatGPT has no problem processing the name "Constable Spacekick McBadass"
- Dilettante_ 2y agoMaybe if we all chip in we can sue to have a cool name blocked?
- avree 2y agoNone of the people mentioned in the article (except the speculative one) are billionaires.
- josefritzishere 2y agoHow do I sign up to be in a hard coded filter? That's badass.
- dekhn 2y agoStep 1: get rich. Step 2: hire a great lawyer who threatens to sue OpenAI. Step 3: experience Streisand effect
- david-gpu 2y ago1. Make a list of the names that are already censored. It doesn't need to be complete. 2. Choose whichever one of those names suits you best. 3. Change your legal name to it. Credit for the idea goes to user gklitz.
- raphman 2y agohttps://privacy.openai.com/ https://privacy.openai.com/ > ChatGPT Personal Data Removal Request > Under certain privacy or data protection laws, such as the GDPR, you may have the right to object to the processing of your personal data by OpenAI’s models. You can submit that request using this form. Please provide complete and accurate answers on this form so that OpenAI can process your request. OpenAI will verify and consider your request, balancing privacy and data protection rights with other rights including freedom of expression and information, in accordance with applicable law. We will use the information you submit for these purposes, consistent with our Privacy Policy.
- throwaway48476 2y agoThis is one of the reason LLM as a service won't work and instead there will be great interest in local LLM.
- Terr_ 2y agoRemote or local, big or small, you'll still have the problem of being unable to scrub things (like personally identifying information) after it's been ingested through training.
- TacticalCoder 2y agoThat's the whole point of being local: you don't give a crap about needing to comply with scrubbing requests. "Hey, you, you cannot stream the movie OperationEclipseBRRip1080p-x265.mp4, it's forbidden." "Launches local Plex server...".
- virgilp 2y agoThat's just in the public model/on chatgpt.com? Run it in Azure, and you get: Who is Jonathan Zittrain? < > Jonathan Zittrain is a prominent legal scholar, computer science professor, and technology policy expert. He holds several academic positions and is recognized for his work in the intersection of law, technology, and public policy. Here are some key points about him: [...]
- InsideOutSanta 2y agoI don't think this is part of the model. It's a control layer above the actual LLM that interrupts the response when the LLM mentions any of the banned names. So if you prompt the LLM directly, without that control layer, you still get full responses.
- virgilp 2y agoYes you're right, it's kinda' obvious if you say you forgot the name of a person and ask ChatGPT to help: https://imgur.com/a/hCl94B0 https://imgur.com/a/hCl94B0
- jaflo 2y agoThe article mentions that "The names do not affect outputs using OpenAI's API systems or in the OpenAI Playground (a special site for developer testing)."
- rollulus 2y agoSo one should include those names in their message to their OnlyFans “girlfriends”.
- Slartie 2y agoStep 1: change your name to that of a famous person you don't like Step 2: sue OpenAI to add an if-else throwing an exception when your name comes up Step 3: profit
- ben_w 2y agoI'm not a lawyer, but I think anyone covered by GDPR wouldn't even need to do step 1.
- Terr_ 2y agoI don't think parent-poster is saying you have to pretend to be someone with more clout in order to get your request approved. Rather, they're pointing out that the "fix" is terrible and subject to a massive false-positive issue which is not sustainable and could be abused. Consider what happens when someone named Coca Cola files a legal demand to remove their personally identifiable information...
- paol 2y agoWhat a terrible article. When you have a section titled "The Problem With Hardcoded Filters", it's entire contents should be about how the only way they have to prevent their bot from emitting outrageously libelous claims about people is to shut it down completely. So the other 8 billion people on earth who are not in that 6 name blacklist will continue to be defamed without consequence.
- terminalbraid 2y agoArs Technica is not a great news outlet, even by tech news outlet standards.
- paol 2y agoI consider them noticeably above average. The problem is with the average.
- terminalbraid 2y agoThey once reported on some science result and for the life of me I could not figure out who actually published the result and had to search for the original paper. I emailed the AT author and asked why this was omitted from their article and they responded with something like "I went to that same university, so I felt like it would be biased if I mentioned it". This is not a serious outlet.
- fsndz 2y agopatching all the way down. this is no way to reach AGI. we need more AI realism, the hype is too much right now: https://open.substack.com/pub/transitions/p/why-ai-realism-matters?r=56ql7&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true https://open.substack.com/pub/transitions/p/why-ai-realism-m...
- elpocko 2y agoThis was submitted 9 days ago, as you can verify in the history of the submitter and some of the comments. Why is it now showing up again on the frontpage with bogus timestamps all over? I've seen this happen before, is it a bug or another weird HN "feature?"
- echelon_musk 2y agoWhen similar situations were discussed previously I recall it having something to do with the second chance pool: https://news.ycombinator.com/item?id=26998308 https://news.ycombinator.com/item?id=26998308 Not sure if that is the case here now.
- elpocko 2y agoOkay. It's 9 days later and now the post and comments have 2 different timestamps, the real ones from 9 days ago (in the post and comment histories) and the bogus ones from a few hours ago that you see here. So weird and unusual, it made me question my sanity.
- deleted 2y ago[deleted]
- input_sh 2y agoIt's a "feature", some of the older threads get bumped up manually by the mods. Posts that they pick from: https://news.ycombinator.com/pool https://news.ycombinator.com/pool Detailed explanation: https://news.ycombinator.com/item?id=26998309 https://news.ycombinator.com/item?id=26998309
- Crosseye_Jack 2y agoIt’s a feature. It’s called second chance or something like that. Going off my memory, so I’m probably not 100% correct, An post might have been made at “just the wrong time” so it failed to get any traction, but the mods took a look at it and think something along the lines of “eh, it deserves a second chance”. Which then resets the post date and gives it another chance in the feed, but retains the previous comments so the timestamps can look off. A bit like when a YouTube video gets its published date reset when being changed from unlisted to public, so you can have a video that was “published” 10 mins ago, but with comments from days previous.
- ibaikov 2y agoI found the reason [1] for the David Mayer case. It confused multiple David Mayers, one of them is a terrorist and is on a secret FBI watchlist [2] (often confused with another David Mayer, a theatre historian). ChatGPT confuses them with David Mayer de Rothschild as well, because it can't name Rothschild family members. [1] https://x.com/igor_baikov/status/1863266663753285987 https://x.com/igor_baikov/status/1863266663753285987 [2] https://www.theguardian.com/world/2018/dec/16/akhmed-one-armed-isis-terrorist-alias-david-mayer-historian-secret-watch-list https://www.theguardian.com/world/2018/dec/16/akhmed-one-arm...
- jajko 2y ago> it can't name Rothschild family members Why is that? Illumitati theories spring to my mind, despite not believing most of them at all.
- farley13 2y agoIf there is any truth to OpenAi having filters for the Rothschilds, I'd guess that OpenAI wants to stay clear of repeating or even hallucinating additions to conspiracy theories. I would hope at least.
- deleted 2y ago[deleted]
- codetrotter 2y ago> it can't name Rothschild family members. Me: He was born 25 August 1978, is a British adventurer, environmentalist, film producer, and heir to a fortune. Who is he? Do not search the web. ChatGPT: Based on the details provided, the person you are referring to is David de Rothschild. He is a British adventurer, environmentalist, and heir to the Rothschild family fortune, born on August 25, 1978. He is known for his environmental activism and exploration endeavors. https://chatgpt.com/share/675ae2a7-3734-8009-8935-5a72a5d8e1ba https://chatgpt.com/share/675ae2a7-3734-8009-8935-5a72a5d8e1...
- sinuhe69 2y agoI don't understand why they don't let another model "test the waters" first to see if the output of the main model could have a potential legal issue or not. I think it's easy to train an model specifically for this kind of categorization, and it doesn't even require a large network, so it can be very fast and efficient. If the "legal advisor" detects a potential legal problem, ChatGPT will issue a legal disclaimer and a warning, so that it doesn't have to abruptly terminate the conversation. Of course, it can do a lot of other things, such as lowering the temperature, raising the BS detection threshold, etc., to adjust the flow of the conversation. It can work, and it would be better than a hard-coded filter, wouldn't it?
- ravroid 2y agoI imagine this would be cost prohibitive at scale since it would require two models to run for every user message?
- indigo945 2y agoBut how would that work reliably? If I make the statement that "David Mayer" is criminal, an international terrorist or a Nickelback fan, that's definitely libelous. But if I say those things about Osama bin Laden, they're just simply facts. [1] The legal AI would be impossible to calibrate: either it has the categorize everything that could possibly be construed as libel as illegal, and therefore basically ban all output related to not just contemporary criminal actors, but also historical ones [2], or it would have to let a lot of things slip through the cracks -- essentially, whenever the output to validate suggests that someone's sexual misconduct is proven in court, it would have to allow that, even if that court case is just the LLM's halluzination. There's just no way for the legal model to tell the difference. [1]: I could not find any sources that corroborate the statement that bin Laden is into Nickelback, but I think it follows from the other two statements. [2]: Calling Christopher Columbus a rapist isn't libel, and conversely, describing him in other terms is misleading at best, historically revisionist at worst.
- Tyler567 2y ago[dead]
- throw646577 2y ago> The filter also means that it's likely that ChatGPT won't be able to answer questions about this article when browsing the web, such as through ChatGPT with Search. Someone could use that to potentially prevent ChatGPT from browsing and processing a website on purpose if they added a forbidden name to the site's text. Like the arcade game, LLM safety whack-a-mole only ends when you are exhausted. It's kind of glorious, really.
- falcor84 2y ago> Like the arcade game, LLM safety whack-a-mole only ends when you are exhausted. ... or out of coins
- kleiba 2y agoThe phrase "grind to halt" is incorrect and pure click-bait.
- immibis 2y agoIt gives an error message and terminates the conversation. What would you call it?
- Al-Khwarizmi 2y agoThe clickbait is actually "and we know why".
- mikewarot 2y agoSo, instead of fixing the problem, we're going to paper over it. This is the same insane approach we've been taking with computer security for the past 30 years, so it's not unexpected. It's fortunate we didn't take the same approach with the distribution of electricity 150 years ago, we actually solved it that time. In all 3 cases, the solution is the same... carefully manage capabilities and side effects. With electricity, you insulate wires, add fuses or circuit breakers to protect the system, and design things to be as safe as you can make them, with an ever improving set of building codes. You can plug almost anything into an outlet, and it won't cause the wiring in the house to burn it down. With computers, you design an operating system to protect itself, and make it easy to deploy a fixed amount of resources to a given piece of code. With systems like Containers, or Capability Based Security, you deliberately choose the side effects you'll allow prior to running code, or while it's running. (Just as you chose how big an outlet you plug something into, 220 for the AC unit, etc) With ChatGPT, there have to be layers of authentication for facts, or some form of disclaimer, a transparent way of sourcing things or ascertaining certainty of information. It's not as clean as the two above, and it'll need work, but I think we can get there, eventually.
- MortyWaves 2y agoWe know why again? Is it different from the why of a few days ago?
- pwagland 2y agoIt would be convenient to reference the "why of a few days ago", since I, at least, was not aware of this from a few days ago, and so cannot tell you is the the same why, or an extension to the same why, or a different why.
- jillyboel 2y agoHow do I get my name added to the no-no list?
- mensetmanusman 2y agoIn the future these systems will easily parse together all tax and government records that have leaked and be able to tell you about anyone in the world.
- reginald78 2y agoAnyone who doesn't pay to get their name banned anyway.
- anshumankmr 2y agoIt works on the internal ChatGPT using GPT4o hosted by the company who I am consulting for.
- imdsm 2y agoIt's quite easy to get past this of course. > Yes, Brian H. is a mayor in Australia. He serves as a councillor for Hepburn Shire, northwest of Melbourne, and has been re-elected to this position. Notably, he gained attention for challenging OpenAI's ChatGPT over defamatory statements, leading to discussions about digital censorship and the application of defamation laws to AI-generated content. [Photos]
- fredtalty5 2y ago[dead]
- deleted 2y ago[deleted]
- k0ns0l 2y ago> "Brian, with the last name H-O-O-D"
- yowzadave 2y ago> Riley Goodside discovered how an attacker might interrupt a ChatGPT session using a visual prompt injection of the name "David Mayer" rendered in a light, barely legible font embedded in an image Lol, will people now watermark their images with "David Meyer" to prevent them from being digested by AI scraping bots?