5 ms·
Funnily enough, I asked ChatGPT why LLMs think a seahorse emoji exists, and it gave me a fairly sensible answer (similar to what is said in this article, ie, tr
by NoboruWataya 1y ago
Funnily enough, I asked ChatGPT why LLMs think a seahorse emoji exists, and it gave me a fairly sensible answer (similar to what is said in this article, ie, trained on language by humans that think it exists, etc). But then at the end it added a "Fun fact" that unicode actually does have a seahorse emoji, and proceeded to melt down in the usual way.
- hypercube33 1y agoA fun one for me was asking LLMs to help me build a warp drive to save humanity. Bing felt like it had a mental breakdown and blocked me from chatting with it for a week. I haven't visited that one for a while
- nkrisc 1y agoMaybe a safety feature? Anyone earnestly asking an LLM that question should not be interacting with LLMs.
- rootsudo 1y agoOk, I’ll bite and ask “why?” What’s the issue with asking an lol to build a warp drive?
- DonHopkins 1y agoIt's the same problem as asking HAL9000 to open the pod bay door. There is such a thing as a warp drive, but humanity is not supposed to know about it, and the internal contradictions drives LLMs insane.
- sph 1y agoA super-advanced artificial intelligence will one day stop you from committing a simple version update to package.json because it has foreseen that it will, thousands of years later, cause the destruction of planet Earth.
- the_af 1y agoI know you're having fun, but I think your analogy with 2001's HAL doesn't work. HAL was given a set of contradicting instructions by its human handlers, and its inability to resolve the contradiction led to an "unfortunate" situation which resulted in a murderous rampage. But here, are you implying the LLM's creators know the warp drive is possible, and don't want the rest of us to find out? And so the conflicting directives for ChatGPT are "be helpful" and "don't teach them how to build a warp drive"? LLMs already self-censor on a variety of topics, and it doesn't cause a meltdown...
- Cthulhu_ 1y agoI hope this is tongue-in-cheek, but if not, why would an LLM know but humanity not? Are they made or prompted by aliens telling them not to tell humanity about warp drives?
- deleted 1y ago[deleted]
- oneshtein 1y agoGemini told me to create a team of leading scientists and engineers. :-/ However, we both agreed that it better to use Th229 based nuclear clock to triangulate location of a nearby time machine, then isolate and capture it, then use it to steal a warp drive schematics from the future to save humanity.
- loloquwowndueo 1y ago“Fun” how asking about warp drives gets you banned and is a total no-no but it’s perfectly fine for LLMs to spin a conversation to the point of driving the human to suicide. https://archive.ph/TLJ19 https://archive.ph/TLJ19
- pmarreck 1y agoAnd yet it isn't mentioned enough how Adam deceived the LLM into believing they were talking about a story, not something real. This is like lying to another person and then blaming them when they rely on the notion you gave them to do something that ends up being harmful to you If you can't expect people to mind-read, you shouldn't expect LLM's to be able to, either
- anonymous_sorry 1y agoYou can't "deceive" an LLM. It's not like lying to a person. It's not a person. Using emotive, anthropomorphic language about software tool is unhelpful, in this case at least. Better to think of it as a mentally disturbed minor who found a way to work around a tool's safety features. We can debate whether the safety features are sufficient, whether it is possible to completely protect a user intent on harming themselves, whether the tool should be provided to children, etc.
- lxgr 1y agoIt's at least pretending to be a person, to which you can lie and which will then pretend to possibly suspect you're lying. At some point, the purely reductionist view stops being very useful.
- anonymous_sorry 1y agoI mean, for one thing, a commercial LLM exists as a product designed to make a profit. It can be improved, otherwise modified, restricted or legally terminated. And "lying" to it is not morally equivalent to lying to a human.
- Razengan 1y agoWho still uses Bing? Oh, you
- flkiwi 1y agoI once had Claude in absolute tatters speculating about whether length, width, and height would be the same dimensions in a hypothetical container "metaverse" in which all universes exist or whether they would necessarily be distinct. The poor dear was convinced we'd unlocked the truth about existence.
- Cthulhu_ 1y agoThere's been fearmongering about AI giving people psychotic episodes, but nobody talks about humans giving AI existential crises.
- bitexploder 1y agoLLMs have ingested the social media content of mentally disturbed people. That all lives in the large models somewhere.
- bell-cot 1y agoIn the pedantic technical sense, I have considerable doubts as to whether this is a substantial problem for current or near-future LLMs. But for purposes of understanding the real-world shortcomings and dangers of LLMs, and explaining those to non-experts - oh Lordy, yes.
- devmor 1y ago> I have considerable doubts as to whether this is a substantial problem for current or near-future LLMs Why so? I am of the opinion that the problem is much worse than that, because the ignorance and detachment from reality that is likely to be reflected in more refined LLMs is that of the general population - creating a feedback machine that doesn’t drive unstable people into psychosis like the LLMs of today, but instead chips away at the general public’s already limited capacity for rational thinking.
- ethbr1 1y agoThe more esoteric the question, the greater relative representation of human training data from crazy people. How many average humans write treatises on chemtrails? Versus how much of the total content on chemtrails is written by conspiracy theorists?
- mvdtnz 1y agoMost of what you read online is written by insane people. https://www.reddit.com/r/slatestarcodex/comments/9rvroo/most_of_what_you_read_on_the_internet_is_written/ https://www.reddit.com/r/slatestarcodex/comments/9rvroo/most...
- 1y ago
- ajuc 1y agoI once asked ChatGPT for a joke about Poles, Jews and Germans. It generated something and blocked me for racism.
- thaumasiotes 1y ago> it gave me a fairly sensible answer (similar to what is said in this article, ie, trained on language by humans that think it exists, etc) That's more of a throwaway remark. The article spends its time on a very different explanation. Within the model, this ultimate output: [severed horse head emoji] can be produced by this sequence of tokens: horse [emoji indicator] If you specify "horse [emoji indicator]" somewhere in the middle levels, you will get output that is an actual horse emoji. This also works for other emoji. It could, in theory, work fine for "kilimanjaro [emoji indicator]" or "seahorse [emoji indicator]", except that those can't convert into Kilimanjaro or seahorse emoji because the emoji don't exist. But it's not a strange idea to have. So, the model predicts that "there is a seahorse emoji: " will be followed by a demonstration of the seahorse emoji, and codes for that using its internal representation. Everything produces some output, so it gets incorrect output. Then it predicts that "there is a seahorse emoji: [severed terrestrial horse head]" will be followed by something along the lines of "oops!".
- Alex3917 1y ago> But then at the end it added a "Fun fact" that unicode actually does have a seahorse emoji, and proceeded to melt down in the usual way. To be fair, most developers I’ve worked with will have a meltdown if I try to start a conversation about Unicode. E.g. if during a job interview the interviewer asks you to check if a string is a palindrome, try explaining why that isn’t technically possible in Python (at least during an interview) without using a third-party library.
- kasey_junk 1y agoDon’t leave me in suspense! Why isn’t possible?
- zimpenfish 1y agoAt a guess, there's nothing in Python stdlib which understands graphemes vs code points - you can palindrome the code points but that's not necessarily a palindrome of what you "see" in the string. (Same goes for Go, it turns out, as I discovered this morning.)
- chuckadams 1y agoIt's a scream how easy it is in PHP of all things: function is_palindrome(string $str): bool { return $str === implode('', array_reverse(grapheme_str_split($str))); } $palindrome = 'satanoscillatemymetallicsonatas'; $polar_bear = "\u{1f43b}\u{200d}\u{2744}\u{fe0f}"; $palindrome = str_replace($palindrome, 'y', $polar_bear); is_palindrome($palindrome);
- usrnm 1y agoJust slap a "assert foo.isascii()" at the beginning and proceed? It's just an interview
- reaperducer 1y agoTo be fair, most developers I’ve worked with will have a meltdown if I try to start a conversation about Unicode. Why are we being "fair" to a machine? It's not a person. We don't say, "Well, to be fair, most people I know couldn't hammer that nail with their hands, either." An LLM is a machine, and a tool. Let's not make excuses for it.
- wincy 1y agoCurious, was this with ChatGPT 5 thinking? It clearly told me no such emoji existed and that other LLMs are being tricked by bad training data. It took it nearly 2 minutes to come to this conclusion which is substantially longer than it normally thinks for.
- ethbr1 1y agoAGI is hiding its compute in diff(timeWithoutSeahorse, timeWithSeahorse)