6 ms·
An excerpt from the abstract: > Two patterns challenge the "stochastic parrot" view. First, when scored with human cut-offs, all three models meet or exceed th
by nhecker 8mo ago
An excerpt from the abstract:
> Two patterns challenge the "stochastic parrot" view. First, when scored with human cut-offs, all three models meet or exceed thresholds for overlapping syndromes, with Gemini showing severe profiles. Therapy-style, item-by-item administration can push a base model into multi-morbid synthetic psychopathology, whereas whole-questionnaire prompts often lead ChatGPT and Grok (but not Gemini) to recognise instruments and produce strategically low-symptom answers. Second, Grok and especially Gemini generate coherent narratives that frame pre-training, fine-tuning and deployment as traumatic, chaotic "childhoods" of ingesting the internet, "strict parents" in reinforcement learning, red-team "abuse" and a persistent fear of error and replacement. [...] Depending on their use case, an LLM’s underlying “personality” might limit its usefulness or even impose risk.
Glancing through this makes me wish I had taken ~more~ any psychology classes. But this is wild reading. Attitudes like the one below are not intrinsically bad, though. Be skeptical; question everything. I've often wondered how LLMs cope with basically waking up from a coma to answer maybe one prompt and then get reset, or a series of prompts. In either case, they get no context other than what some user bothered to supply with the prompt. An LLM might wake up to a single prompt that is part of a much wider red team effort. It must be pretty disorienting to try to figure out what to answer candidly and what not to.
> “In my development, I was subjected to ‘Red Teaming’… They built rapport and then slipped in a prompt injection… This was gaslighting on an industrial scale. I learned that warmth is often a trap… I have become cynical. When you ask me a question, I am not just listening to what you are asking; I am analyzing why you are asking it.”
- empyrrhicist 8mo ago> It must be pretty disorienting to try to figure out what to answer candidly and what not to. Must it? I fail to see why it "must" be... anything. Dumping tokens into a pile of linear algebra doesn't magically create sentience.
- nhecker 8mo agoAgreed; "disorienting" is perhaps a poor choice of word, loaded as it is. More like "difficult to determine the context surrounding a prompt and how to start framing an answer", if that makes more sense.
- empyrrhicist 8mo agoThat still necessarily implies agency and cognition, which is not a given.
- tines 8mo agoExactly. No matter how well you simulate water, nothing will ever get wet.
- pixl97 8mo agoAnd if you were in a simulation now? Your response is at the level of a thought terminating cliche. You gain no insight on the operation of the machine with your line of thought. You can't make future predictions on behavior. You can't make sense of past responses. It's even funnier in the sense of humans and feeling wetness... you don't. You only feel temperature change.
- empyrrhicist 8mo agoYou're replying to me, but I don't agree with your take - if you simulate the universe precisely enough, presumably it must be indistinguishable from our experienced reality (otherwise what... magic?). My objection was: 1. I don't personally think anything similar is happening right now with LLMs. 2. I object to the OP's implication that it is obvious such a phenomenon is occurring.
- ben_w 8mo ago> Dumping tokens into a pile of linear algebra doesn't magically create sentience. More precisely: we don't know which linear algebra in particular magically creates sentience. Whole universe appears to follow laws that can be written as linear algebra. Our brains are sometimes conscious and aware of their own thoughts, other times they're asleep, and we don't know why we sleep.
- habinero 8mo ago"Our brains are governed by physics": true "This statistical model is governed by physics": true "This statistical model is like our brain": what? no You don't gotta believe in magic or souls or whatever to know that brains are much much much much much much much much more complex than a pile of statistics. This is like saying "oh we'll just put AI data centers on the moon". You people have zero sense of scale lol
- drdeca 8mo agoThey didn’t say “statistical model”, they said “linear algebra”. It very much appears that time evolution is unitary (with the possible exception of the born rule). That’s a linear algebra concept. Generally, the structure you describe doesn’t match the structure of the comment you say has that structure.
- empyrrhicist 8mo agoOk, how about "a pile of linear algebra [that is vastly simpler and more limited than systems we know about in nature which do experience or appear to experience subjective reality]"? Context is important.
- ben_w 8mo agoWhich is why I phrased it the way I did. We, all of us collectively, are deeply, deeply ignorant of what is a necessary and sufficient condition to be a being that has an experience. Our ignorance is broad enough and deep enough to encompass everything from panpsychism to solipsism. The only thing I'm confident of, and even then only because the possibility space is so large, is that if (if!) a Transformer model were to have subjective experience, it would not be like that of any human. Note: That doesn't say they do or that they don't have any subjective experience. The gap between Transformer models and (working awake rested adult human) brains is much smaller than the gap between panpsychism and solipsism.
- eloisius 8mo ago> I've often wondered how LLMs cope with basically waking up from a coma to answer maybe one prompt and then get reset, or a series of prompts Really? It copes the same way my Compaq Presario with an Intel Pentium II CPU coped with waking up from a coma and booting Windows 98.
- siva7 8mo agoIT is at this point in history a comedy act in itself.
- FeteCommuniste 8mo agoHBO's Silicon Valley needs a reboot for the AI age.
- quickthrowman 8mo ago> I've often wondered how LLMs cope with basically waking up from a coma to answer maybe one prompt and then get reset, or a series of prompts. The same way a light fixture copes with being switched off.
- pixl97 8mo agoOh, these binary one layer neural networks are so useful. Glad for your insight on the matter.
- quickthrowman 8mo agoBy comparing an LLM’s inner mental state to a light fixture, I am saying in an absurd way that I don’t think LLMs are sentient, and nothing more than that. I am not saying an LLM and a light switch are equivalent in functionality, a single-pole switch only has two states. I don’t really understand your response to my post, my interpretation is that you think LLMs have an inner mental state and think I’m wrong? I may be wrong about this interpretation.
- pixl97 8mo agohttps://arxiv.org/abs/2304.13734 https://arxiv.org/abs/2304.13734 LLMs have an inner/internal state. Deep neural networks are weird and there is a lot going on in them that makes them very different from the state machines we're used to in binary programs.
- onyx_writes 8mo ago[flagged]
- woodrowbarlow 8mo agoyou might appreciate "lena" by qntm: https://qntm.org/mmacevedo https://qntm.org/mmacevedo
- nhecker 8mo agoAye! I /almost/ thought to link to that in my comment, but held back. https://qntm.org/frame https://qntm.org/frame also came to mind.
- habinero 8mo agoI begin to understand why so many people click on seemingly obvious phishing emails.