5 ms·
I can't pretend to understand how LLMs work, but I can be sure that anthropomorphizing their functions is not helpful to an objective debate over their abilitie
by pcrh 5mo ago
I can't pretend to understand how LLMs work, but I can be sure that anthropomorphizing their functions is not helpful to an objective debate over their abilities.
Does a motor vehicle get "sleep" when it is serviced? When I reboot a computer, is that equivalent to a nap?
- eithed 5mo agoI assume compacting is the sleep here; so, yes
- ajs1998 5mo agoThis is the struggle of naming papers. You could stretch definitions and make your own sexy headline or you could be precise and fewer people will read it.
- colechristensen 5mo ago>we study a sleep-like consolidation mechanism in which a model periodically converts recent context into persistent fast weights before clearing its key-value cache There is a strong, non-trivial connection here between what your brain does in sleep and what they are studying. You wouldn't object to referring to robot eyes or robot legs.
- cowlby 5mo agoThe analogy is helpful, but yes we should be able to “intelligently design” something better than sleep analogues since we’re not constrained by evolution like in humans.
- lxgr 5mo agoWe are however constrained by the complexity of any purported solution. That's the bitter lesson, in a nutshell. At the very least, we know that sleep and dreaming do exist in biological brains. (Doesn't mean any of it is applicable to artificial neural nets, doesn't mean it'll work for our specific architectures etc. etc., but at least the idea requires fewer assumptions than a completely untested novel theory.)
- SR2Z 5mo agoEvolution constrains the evolution of human beings, but it's also excellent at discovering elegant designs that work very reliably at a low cost. Maybe someday we'll understand the way our minds work well enough to design from first principles but until then we've only got one template for how a thinking machine should look.
- tom_ 5mo agoSee also, perhaps: https://news.ycombinator.com/item?id=48273597 https://news.ycombinator.com/item?id=48273597
- lxgr 5mo agoIf it works, it's called bionics, not anthropomorphization ;)
- famouswaffles 5mo agoAnthropomorphization is not inherently wrong, and in some instances, it actually lets you reason better about about complex behavior than whatever convoluted (and often wrong, especially in the case of giant neural networks) mechanistic description one might conjure. Here the analogy isn't without reason.
- DonHopkins 5mo agoIs it "Anthropicmorphization" when Claud treats human beings like LLMs?
- sillysaurusx 5mo agoInteresting question. Is there an actual term for that? It’s like inverse anthropomorphization, but not quite.
- incognito124 5mo agoMechanomorphisation
- bouncing_bolete 5mo agoDehumanization, assuming this wasn't sarcasm
- forshaper 5mo agoWason Selection task performance improvements based on social framing suggest that it's easier for us to think about problems when some anthropomorphization is going on. https://www.cep.ucsb.edu/wp-content/uploads/2023/05/Cogadapt.pdf https://www.cep.ucsb.edu/wp-content/uploads/2023/05/Cogadapt...
- gabriela_c 5mo agoFeels like we're having a computer world Jane Goodall moment.
- pfdietz 5mo ago
- djeastm 5mo agoThey provide an explanation for using the term "sleep": > In animals, the transfer from short-term memory to long-term memory is thought to be supported by hippocampal replay [33], especially during sleep [41]; in this phase, short-term hippocampal memories are reactivated and consolidated into cortical synaptic weights. Sleep makes animals unable to respond to external stimuli, suggesting that it must provide enough cognitive benefit to justify this cost [41]. Inspired by these biological processes, we propose a method for transferring context-window memory into persistent weights. When the model’s context window becomes full during inference, the model enters a “sleep” in which it performs multiple forward passes over the accumulated context and recursively updates its fast weights via a learned local rule. As in animal sleep, the model receives no external input tokens during this phase. After consolidation, the context window is cleared, and the model resumes operation with updated fast weights. During training, the model is optimized end-to-end by backpropagating through the entire process to maximize task performance after sleep.
- pcrh 5mo agoThe function of sleep in animals is largely obscure. One thing we do know for certain is that it is necessary, it is needed in "dumb" animals as well as in you and I. If an animal can't sleep it will eventually die. I don't think that applies to the activity described in the OP. Does their LLM "die" if it can't perform the function described?
- bayarearefugee 5mo ago> Does their LLM "die" if it can't perform the function described? If you don't periodically clean the context, an LLM effectively goes insane in terms of outputs. If the LLM were fully controlling a physical system (like a robot body) that contained it the resulting insanity of an ever-growing, never cleaned context would likely result in some sort of death-like event.
- pcrh 5mo agoThat's probably the closest analogy posted here. It's still weak, though. An LLM without constant human input is likely more similar to a bicycle that starts to lose its gyroscopic balance as it moves more slowly, a human can however keep a stationary bicycle upright (while riding it).
- raincole 5mo agoThis is why I object to sleep() from unistd.h. What an anthropomorphizing notion. Didn't early unix programmers understand that a computer isn't a living creature and therefore isn't capable of sleep? They must have been really stupid!
- not_a_bot_4sho 5mo agoSome of them were straight up psychopaths too, as evidenced by `kill()` !
- prerok 5mo agoIndeed and using SIGKILL is really cruel. At least with SIGTERM the process can say its goodbyes. /j
- simonw 5mo ago> When I reboot a computer, is that equivalent to a nap? I mean, you do put your computer into "sleep" mode and then "wake" it. Analogies are useful. I think we need to learn how to continue to benefit from them despite the risk of anthropomorphication.
- deleted 5mo ago[deleted]
- verisimi 5mo ago... and anyway, maybe it was hungry? Or getting the sniffles?
- aaroninsf 5mo agoVery much agree that while it is is useful in description of motivation and inspiration, it is very non-helpful—or worse—to use this language, this way. One might as well say "need neural plasticity" which is as much an analogy and equally misleading and counterproductive in shaping the right model of the system. One might even call this pernicious, what it encourages is already a social problem; and it doesn't aid understanding, it confounds it.
- wat10000 5mo agoJust from the title, I’m assuming it refers to a period of downtime used to perform some sort of maintenance on the knowledge held by the system. Clicking through, that’s exactly what it is. Seems like “sleep” is an excellent term to use here.
- burnte 5mo ago> Does a motor vehicle get "sleep" when it is serviced? That's more like a doctor visit and a workout. The sleep will be the part of the duty cycle when it's not operating. > When I reboot a computer, is that equivalent to a nap? Yes, it wakes up completely refreshed and in good working order, usually, and if there's still a problem you know you need a technician.
- madibo3156 5mo agoI find this annoying too. "Sleep" is okay, but the quippy headlines ("need sleep"—short, snappy and vague) infiltrating journals bother me. I've seen it well before LLMs, but as an example, there is a long list of title snowclones of the famous attention paper: https://github.com/vinayprabhu/X-is-all-you-need https://github.com/vinayprabhu/X-is-all-you-need.
- cush 5mo agoI think it's interesting that folks are suddenly taking issue with "anthropomorphizing" language used in AI as if we haven't been doing this since the earliest days of computing (see "memory", "child", "parent", etc). It helps folks understand things at the correct level without needing domain knowledge
- deleted 5mo ago[deleted]
- CuriouslyC 5mo agoSaying something needs sleep isn't anthropomorphizing, since pretty much all complex living organisms need sleep. Also, even when something is "specific" to humans, it might not be anthropomorphizing to observe it in something else, it could just be an emergent pattern of high intelligence.
- gchamonlive 5mo agoJust like LLM sleep has nothing to do with animal sleep, the neuron in a neural network has nothing to do with an actual neuron, and nobody should pretend they do. I agree we need to be mindful of our metaphores, but they do help both with inspiration for developing techniques as well as for naming things. The onus of keeping bias in check when using metaphores is on the reader, authors can't really do that for you. However once bias is in check you can have a very productive debate in terms of these namings given that everyone is aware of their ontology.
- skeledrew 5mo agoHow do you concisely describe a low power state of an entity that processes, whereby when in that state it has little to no reaction to input and it may or may not be performing tasks in that state, for a mixed education audience? Also keep in mind that most if not all devices with a chip have had a function called "sleep" for many years, without this argument.
- motoxpro 5mo agoOne of the most common functions in programming is sleep(ms). There is wake, heartbeat, handshake, orphan, listen, starve, parent/child, etc. This is not anything new, its just a word that fits the function.
- reaperducer 5mo agoDoes a motor vehicle get "sleep" when it is serviced? One of the mayors of New York in the 80's (Koch?) famously doubled the city's bus fleet for zero cost by running them 24 hours, instead of letting them rest at the end of their shifts, as was the previous policy.
- Dusseldorf 5mo agoWas it anthropomorphizing computers when they named "memory"? Seems to me like it's more analogizing for the sake of easy understanding. Sure, it's not literally the same exact mechanism, but it's certainly modeled after the biological concept.
- throw310822 5mo agoFirst, this is not a "debate over the abilities" of LLMs. It's a proposed method to improve their performance, and the authors are free to call it however they think it makes sense. Second, explicitly avoiding things that sound like anthropomorphisation is equally not helpful- why avoid a metaphor that works? Third, it's really a pity that this pointless nitpicking is dominating the thread.
- pcrh 5mo ago>this pointless nitpicking is dominating the thread. What is dominating the thread are claims that the LLM operation in question is analogous to the function of sleep in humans. It obviously is not. The anthropomorphization of LLMs has reached ridiculous proportions. Applying the same standards as used in this field to others would result in claims that laundry machines "hallucinated" that they had had sufficient water when they failed due to the faucet being turned off.
- ComplexSystems 5mo agoThat's because the purpose of this article is not to have an objective debate over their abilities at all. Most interesting research in this field isn't. Instead, it's to present a new technique to improve LLM performance, which is much more interesting than (once again) rehashing the philosophy of LLM personhood.
- jonnyasmar 5mo agoWhen the goal of that function is to think (a notoriously human behavior), it's perfectly understandable to anthropomorphize it.
- genxy 5mo agoPlease re-read up to the end of page 2 and then re-ask this question.
- zeckalpha 5mo agoYet this is how "thinking" models got started.
- FarmerPotato 5mo ago>Does a motor vehicle get "sleep" when it is serviced? When I reboot a computer, is that equivalent to a nap? Do androids dream of electric sheep?