5 ms·
There are people whose brains don’t form new memories anymore after an accident or surgery, and they eternally live in the time before it happened, and have no
by layer8 5mo ago
There are people whose brains don’t form new memories anymore after an accident or surgery, and they eternally live in the time before it happened, and have no memory of what happened a minute ago. Still they are conscious.
- eberkund 5mo agoI think it's a little more complicated than that. In a 50 First Dates type of scenario, their ability to form certain types of memories is damaged, not non-existent. And I would argue that with enough brain damage someone like an extreme lobotomy victim may stop being considered conscious.
- layer8 5mo agoI’m not familiar with 50 First Dates, I was thinking of cases like Clive Wearing [0]. I would agree that consciousness requires some sort of ultra-short-term working memory, but I also think that mechanisms similar to CoT loops can conceivably fulfill that role. Today’s generative AIs consist of more than just the static network-of-weights model. [0] https://en.wikipedia.org/wiki/Clive_Wearing#Amnesia https://en.wikipedia.org/wiki/Clive_Wearing#Amnesia
- krupan 5mo ago"Wearing can learn new procedures and even a few facts, not from episodic memory or encoding, but by acquiring new procedural memories through repetition. For example, having watched a certain video recording multiple times on successive days, he never had any memory of ever seeing the video or knowing the content, but he was able to anticipate certain parts of the content without remembering how he learned them." Honestly, that's a pretty messy state of consciousness and I wouldn't proudly crow that my AI is conscious if that's as good as it got
- krupan 5mo agoThey are conscious because even for short periods of time they do form memories and those change them even if only briefly. They think on their own too. It is a very limited level of consciousness though.
- Taek 5mo agoIs that any different from an LLM having a context window?
- nozzlegear 5mo agoYes, LLMs don't think on their own, for one; they think when you invoke them.
- layer8 5mo agoThat could be easily fixed by providing the AI with a constant stream of input. For humans, part of the input of the human mind comes from the continuous processes and clocks within the human body, so it’s questionable whether the brain could “think on its own” without such input either.
- nozzlegear 5mo agoThe continuous input for the human arises naturally, it doesn't arise naturally for an LLM unless we direct it so. Our consciousness is bootstrapped, the LLM isn't.
- layer8 5mo agoWe have virtually no idea how consciousness arises in the human brain. Furthermore, what is “natural” supposed mean here, and why should it matter for consciousness whether some prerequisite arises naturally or not?
- nozzlegear 5mo agoI was literally only responding to "Is that any different from an LLM having a context window" lol. Let's keep that in our context windows. I'm not interested in discussing how human consciousness is different from supposed LLM consciousness; it's enough for me to know that humans are conscious, and in an obvious, clearly distinct way, even if we can't define it. Sniffing our own farts about whether LLMs are or aren't is just that – nerds larping as philosophers while practicing fart sniffing. It's a machine, periodt, we can quit roleplaying.
- micromacrofoot 5mo agothey still have memory, just not new ones - they lived experiences
- layer8 5mo agoAn LLM’s training could be seen as lived experience, and the fact that LLMs can output long sequences from their training material can be interpreted as them remembering those parts. Also, how does that relate to consciousness? I don’t think that past episodic memory is necessary for consciousness.
- micromacrofoot 5mo agoyou can't be conscious about your decisions if you don't incorporate their effects into your corpus — knowing the results of other people's actions secondhand isn't the same because you're not those people
- seizethecheese 5mo agoInteresting point but even those people’s brains aren’t immutable. The have habit change without memory.
- bulhabulha 5mo agoThe starting file may be immutable, but the whole processing of that file is very dynamic and intense. Maybe, if there is some consciousness, it lies somewhere during that processing.
- layer8 5mo agoTrue, but I don’t see how that relates to consciousness. An LLM being continuously RLHF-trained also changes its habits; that alone doesn’t make it conscious.
- knollimar 5mo agoI was like this for a bit and you still have memories from like 30 seconds to minutes ago, but after that you have a cliff where you don't remember. I don't think LLMs structurally even get the 30 seconds part. It's literally 0 for them.
- mft_ 5mo agoI'd argue that the context window is analogous to short-term memory. It's functional but limited in duration, and if you overload it, it starts to fail. It's the long-term memory (i.e. learned experiences feeding back and directly altering the content of the core brain, or model) that is missing.
- knollimar 5mo agoThe context window is so flawed that I wouldn't consider it memory. It feels like notes about the situation rather than it being in memory. Memory has more "attention". I think that "it starts to fail" is load bearing here. I feel like memory has like 5 parts, and LLMs are missing 2 of them: current working memory short term what is immediately happening without it being in "RAM". I differentiate here vs working in like thinking fast and slow. Keeping things in working memory is work! You can vibe away short term memory. I had excellent short term memory while I was messed up, I could keep time well. I think LLMs can do this with notes. mid term: Vague awareness of things like what day a week it is or what you did 2 hours ago. This is where my memory personally failed long term memory of experiences. You can fake this with memory.md generalized wisdom for pattern matching long term memories LLMs seem to be missing that part I was missing. Im probably projecting and anthropomorphizing. But i relate: I would confabulate a ton and didn't know anything was wrong for a while but things seemed off. Context is like working memory but not short term or mid term. I think you can imply short term with big enough context. My categories are purely anthropomorphic to me but just wanted to say where I disagreed.
- mft_ 5mo agoThanks for sharing your experience. It's really interesting that you describe a loss of some 'middle' parts but not others. The 'classic' medical/psychological model of memory has three parts (sensory/short-term/long-term), but it's also worth noting that that model was first devised in 1968! > long term memory of experiences. You can fake this with memory.md Not sure about this; to my mind, memory.md is analogous to humans making lists of things to not forget to do, or notes from a lecture to learn (i.e. cram into long-term memory) later on. LLMs use it as a short cut to bring important facts back into their context window; but it's not the same as them already 'knowing' the information via the original training process. --- My consistent (hot?) take is that a (the?) major piece holding LLMs back (maybe even from AGI?) is continual learning. Humans have systems for continually updating their long-term memory from their lived experience - new facts, processes, skills, successes, mistakes, etc. (Sleep and dreaming are centrally involved in this process.) The current architecture of LLMs makes this practically impossible, as it would presumably require the level of power currently necessary for training to be continually applied for continual learning, and demonstrates the huge efficiency advantage of the biological brain.
- MagicMoonlight 5mo agoSomeone getting in an accident that chops their leg off doesn’t mean humans don’t have legs. Come on man.
- flashman 5mo agovast oversimplification of the experience of brain damage