9 ms·
The overfitted brain: Dreams evolved to assist generalization (2021)
- api 4y agoAvoiding overfitting has struck me as being possibly behind the creativity and psychological benefits that some people experience from psychedelics too. Psychedelics seem to be capable of kicking the brain out of some kind of local maximum in which it is stuck. Being "stuck in a rut," having depression or PTSD, or just being uncreative could be symptoms of the brain having overfit to some past set of stimuli.
- ccvannorman 4y agoThis resonates with me; nice way of putting our psychological pitfalls into machine learning terms!
- Jensson 4y ago> Dreaming remains a mystery to neuroscience. While various hypotheses of why brains evolved nightly dreaming have been put forward, many of these are contradicted by the sparse, hallucinatory, and narrative nature of dreams Dreams are your subconsciousness asking you, the consciousness, about different scenarios or problems it can think of. That is the most obvious explanation to me. When you are awake then disturbing you could be dangerous, so that has to happen when you sleep. And since you the consciousness has to answer those questions you will experience them as fragments, ie "dreams". I don't see why dreams would have to be more complicated than that. It is very simple and explains everything.
- endisneigh 4y agothat doesn't explain everything. why would you subconsciousness ask anything to begin with? amazing how people state things like it's an absolute fact based on absolutely nothing.
- Jensson 4y ago> why would you subconsciousness ask anything to begin with? Because it is learning? If you had this supersmart function that could evaluate situations, why not ask it things when you are training another algorithm that tries to evaluate things cheaper? Edit: Or just to categorise things in general.
- malteof 4y ago[Citation needed]
- Jensson 4y agoWhy would I need a citation to state a theory?
- qup 4y agoBecause the internet is in a fad where it's very fashionable to require evidence for any claims, and most opinions.
- dazc 4y agoCitation?
- IshKebab 4y agoThe title is stated as fact, not a theory. You have to get all the way to the last sentence in the abstract before they say it's a hypothesis. Misleading title at best. I would say disingenuous.
- Jensson 4y agoIts clickbait, clickbait works, you see lots of discussion here thanks to it (or due to if you don't like it).
- bumby 4y agoBecause they stated a hypothesis. In order to be a theory, it needs to be rooted in data.
- AlecSchueler 4y agoIt's everyday English though, theory is the common word for what would be called an hypothesis in scientific circles.
- chresko 4y agoWhat you're proposing is a dialog between a conscious and subconscious self, while in a dream state, where one or the other (or both) forms of being are fully aware of what questions to ask, and what the answers to those questions are. This doesn't sound simple. It sounds extremely complex. And it doesn't explain everything, like, for example, why some people don't dream or can't remember their dreams. Or why other animals dream, unless all animals have a conscious and subconscious self and are attempting to resolve internal conflict.
- Jensson 4y ago> unless all animals have a conscious and subconscious You don't believe animals have a conscious? I thought all humans believes that a dog for example had a consciousness. > What you're proposing is a dialog between a conscious and subconscious self, while in a dream state, where one or the other (or both) forms of being are fully aware of what questions to ask You say this is complex, but there is definitely a subroutine that determines when you should sleep, when to wake you up etc. That routine calling you to do some bookkeeping while you are sleeping doesn't add any complexity at all. I don't see why that routine would be smart enough to do everything else, but not smart enough to ask some questions about things when you sleep...
- bumby 4y ago>You don't believe animals have a conscious? I think it’s a matter of degree. We don’t know what level of consciousness they have relative to our own experience. Can you say with certainty, for example, that a mollusk or sea cucumber has conscious experience?
- jklinger410 4y agoWhile we're posting conjecture. I believe dreams are just hallucinations due to chemicals that the brain uses to wash and reset itself every night. The dream itself is the remnants of your conscious mind trying to make sense of the state of hallucination it is in. It is as if there is something damaging about the accumulation of memories, both micro (cellular, nervous) and macro (brain memory), that is incompatible with the way animals evolved. I believe we would see similar negative effects of why we need sleep if we tried to replicate it without the actual rest, or if human brains were able to survive on long passed 100 years.
- Jensson 4y agoBut that can't be it, daydreaming also happens where the subconsciousness takes over even when you are awake. Daydreams doesn't do that washing. But we do know that our subconsciousness can learn to drive or to walk home etc, since we can do that without thinking about it. It has to learn those things somehow, and asking our consciousness about how all the things works, how to react or think about things when we sleep, that is the obvious ways for it to learn and test things from its control. I don't believe you can run the consciousness functions without us remembering it in some way, and that becomes the dreams. Edit: Maybe the hard part to understand is that people don't want to imagine themselves as a small part of a larger system. We aren't the main thing in control of ourselves, instead there is something that can shut us down, turn us up, invoke parts of us at will etc, that is in control. It just reuses some of the code that runs our consciousness to run other evaluations.
- bumby 4y agoIsn’t that incompatible by the very nature that we are aware of our daydreams? I.e., it can’t be subconscious if we’re aware of it. Perhaps you mean daydreaming is when subconscious thoughts transition to conscious awareness? I think this is part of what the other commenter was alluding to with imprecise language.
- Jensson 4y ago> Isn’t that incompatible by the very nature that we are aware of our daydreams? I.e., it can’t be subconscious if we’re aware of it. Perhaps you mean daydreaming is when subconscious thoughts transition to conscious awareness? I don't see how that is different from you being aware of your dreams. Daydreams aren't controlled by you, there is another process you aren't aware of that is feeding you the daydream. Also usually when you daydream you stop being aware of your surroundings, like when walking etc, so some process has to take over that as well. I'm talking about those processes.
- mtlmtlmtlmtl 4y ago"Subconscious" is not a well defined technical term in psychology anymore, and hasn't been used since the days of Freud when nothing was well-defined anyway. And even Freud regretted his use of the term: "If someone talks of subconsciousness, I cannot tell whether he means the term topographically – to indicate something lying in the mind beneath consciousness – or qualitatively – to indicate another consciousness, a subterranean one, as it were. He is probably not clear about any of it. The only trustworthy antithesis is between conscious and unconscious." The "unconscious" is not understood as being a "hidden consciousness" with its own agency. It's simply understood as the collection of processes in the brain that you are not consciously aware of. All sorts of sensory processing and integration. Things like balance, the syncing of visual and auditory inputs. It's like the brain's standard library. There's very little support for the idea of some sort of dialogue between conscious and unconscious.
- Jensson 4y agoThe name doesn't matter, but it is a fact that there are processes that we don't control that are very intelligent in our brain. They can find their way home, or drive your car etc, without you having to think about driving or where to go, you just zone out and before you realize it you are home. Any assumption that such a process doesn't exist is therefore disproven immediately, we know that none of those can be true. So, since we know that such a process exists, it asking us questions just like we ask it questions seems like a very useful property to have to handle all sorts of bookkeeping and learning. It is a very obvious mechanic in a brain containing two intelligent systems interacting so tightly as these two does.
- mtlmtlmtlmtl 4y agoThe name matters a lot if you want to clearly communicate your ideas. Your use of terminology leaves so many possible interpretations that it's hard to know exactly what you're trying to communicate. Which makes meaningful discussion nigh-on impossible.
- Jensson 4y ago
- coldtea 4y ago>Dreams are your subconsciousness asking you, the consciousness, about different scenarios or problems it can think of. That is the most obvious explanation to me. One can't explain the workings of the brain as some Q&A between "conscious" and "unconscious" except as a high level abstraction/metaphor. Else this implies two tiny "thinking" entities inside your brain, where one asks another something.
- hungryforcodes 4y agoThe most credible explanation to me -- and one that makes logical sense, is that dreaming is about processing the day's events -- possibly for learning, or committing to memory (same thing basically?). Since hearing about this theory, I've been able to pin point my dreams 90% of the time to themes that were discussed or experienced during the day I fell asleep. Including topics I had only worried about but never mentioned to anyone.
- Sharlin 4y agoI’m much more inclined to believe that dreams are pure System 1 stuff and essentially what we perceive is just how it feels inside when the part of the brain that thinks it’s in charge is 100% riding shotgun and the part that’s the actual driver is doing whatever dreams are for. Especially given that dreams are mostly the polar opposite of what System 2 is good at. This hypothesis is at least as good as yours, and requires fewer assumptions. For the record, I’m also very inclined to believe that even in the waking life the “conscious mind” has vastly less command authority than it likes to think, mostly receiving “executive memos” when the actual decision has already been made by the faster parts of the brain.
- parineum 4y agoI'm not sure this is a new idea as much as it is a new analogy. It might be useful to some segment of people but I'm not sure it's better than the existing idea of just being conditioned to some stimuli/pattern and dreams being a way to explore new ideas.
- ftxbro 4y agoI wonder can videos of your dreams be recorded by https://sites.google.com/view/stablediffusion-with-brain/ https://sites.google.com/view/stablediffusion-with-brain/
- version_five 4y agoThat article was discussed here yesterday, from what I gather, its likely smoke and mirrors and not in any way decoding what our mind sees Edit: https://news.ycombinator.com/item?id=35012981 https://news.ycombinator.com/item?id=35012981
- rubicon33 4y agoI used to dream a lot as a kid. Somewhere along the way that basically stopped happening. It’s come back somewhat in mid life but not with the intensity and sticking power that dreams in youth had.
- avereveard 4y agoTry studying a subject in real depth for six month, that caused me resume dreaming
- Llamamoe 4y agoThis is sort of what I've been dreading myself, that my loss of dreams has been the consequence of a monotone, miserable life.
- avereveard 4y agoI have no way nor intention to judge your life. I think it's more about knowledge pressure than monotonous life. you can be havein an exciting life and never exercise your thinking muscles, and a miserable life while learning new things every day, and all variation in between. as a matter of fact, the six month where I had to learn thousand pages per week were some of the most miserable I had in the last two years because job kept me away from my newborn, it did a number on my health, and I'm barely returning to normal right now.
- detourdog 4y agoI definitely associate hard thinking with a tighter connection to my subconscious.
- coldtea 4y agoDreaming also depends on sleep quality and REM cycles. Bad sleep, caffeine, etc, can all reduce or eliminate dreaming...
- technocratius 4y ago
- etiam 4y agoAt least this paper cites Hinton et al. (1995). The wake-sleep algorithm for unsupervised neural networks which conjectures just the same roles for dreaming. But as far as I can see there is absolutely nothing new provided here about experiments on brain or psyche. You'd think 28 years is a bit long a delay to try to claim a pilfered engineering theory as your own before the medical community? Friston was at least more timely.
- etiam 4y agoIt seems to be just the editorial that's erroneously claiming that "Erik Hoel proposes a novel hypothesis". The actual review article appears to be a perfectly serviceable review of the topic. https://www.cell.com/patterns/fulltext/S2666-3899(21)00064-7 https://www.cell.com/patterns/fulltext/S2666-3899(21)00064-7
- adampk 4y agoCould you elaborate a bit more about what you think about Friston? All I know of him was from this article: https://www.wired.com/story/karl-friston-free-energy-principle-artificial-intelligence/ https://www.wired.com/story/karl-friston-free-energy-princip... And I took it at face value, what am I missing?
- etiam 4y agoI don't have access to a WIRED subscription currently, so I only see the first few lines, but then again a flavor for some problems with that article seems to start right at the title and opening paragraphs. "The Genius Neuroscientist Who Might Hold the Key to True AI" Karl Friston's free energy principle might be the most all-encompassing idea since the theory of natural selection. But to understand it, you need to peer inside the mind of Friston himself. The gist of the problem is Karl Friston didn't come up with the idea in question, yet somehow he's often promoted publicly much as if he did. Variational free energy methods generally are pretty amazing. Thanks to mathematicians, physicist, chemists and so on over the course of the last three centuries or so for giving us that. Free energy methods in perception and learning are also very interesting, and that goes back at least to Geoff Hinton, Richard Zemel and Radford Neal in the mid 90s (arguably there is relevant prior work in psychology and energy based ANN before that too). It's hard to believe Friston's application of the theory to brain function (as opposed to earlier in brain imaging) would be independent of having arrived at University College London, where people at Gatsby Unit (Hinton, Peter Dayan, etc) had been working with the methods in innovative and and effective ways for machine learning/artificial intelligence/theoretical neuroscience since the 90s. Yet if you entered the subfield through Friston's citations, which many people now do, due in no small part to glowing personality portraits like the one presented by WIRED, you might easily come away with the impression that the whole field suddenly manifested out of whole cloth between the ears of this Genius Neuroscientist, some time in the 00s. Friston does have one real distinguishing contribution to his name though, in being the first person to reach wide attention making vocal and bold claims that the "free energy principle" amounts to a largely comprehensive explanation for how the human brain works. Good for him if that turns out to be the case I guess, but personally I'm not aware of any demonstrations that it actually is such a powerful explanation for any mechanisms of brain function. I have no personal knowledge of Karl Friston, but I've heard he's often well liked in personal contacts. I don't really know enough about the medical imaging methods era of his work to comment on it.
- warning26 4y agoMy question is: how could this idea be adapted to our current ML models? For example, an LLM just tries to predict the next token when requested. How does one go from "next token please" to something that "dreams"?
- jgilias 4y agoLoks like the current iteration of LLMs are very good when it comes to being able to hallucinate!
- midlightdenight 4y agoI’m not sure if dreaming is the right word here, but the recent ML model Dreamer v3 [0] “dreams”. This is more akin to thinking ahead since it’s really just predicting outcomes using its world model. It’s possible they could try training the model on some hallucinated scenarios to avoid over fitting, but I’m no ML researcher. [0] https://danijar.com/project/dreamerv3/ https://danijar.com/project/dreamerv3/
- jonnycomputer 4y agoWell, the idea comes from ML in the first place, and has been incorporated into some theories of brain function.
- konfusinomicon 4y agoI like to think that dreams are our brains tuning in to the many lives of ourselves across the multiverse. there are infinite possibilities, and while sleeping in this dimension, we are experiencing our life on a different timeline. the theory is impossible to prove and is a highly unlikely reality, but it's fun as hell to think about it that way
- runlaszlorun 4y agoI’m no expert on multiverses and I only skimmed the article but that theory and the article are supportive, no?
- konfusinomicon 4y agosame for me and it ya it does in the sense that in waking life the brain experiences the what is(or what we think is), where in dreams we experience the what could be. the thought that we are learning and growing in this life by actually experiencing the x amount of lives we live in other dimensions while our physical self lays dormant in the one we are bound to is pretty interesting. and that our brains actually evolved to make use of that data meaningful even more so. perhaps we are experiencing but a dream of another version of ourselves when we are awake
- pstuart 4y agoWhat if dreams don't have a purpose? Perhaps they're merely scrambled bits of memories and thoughts that are being reprogrammed when we sleep and sometimes our "self" gets a peek at the sausage being made?
- kupopuffs 4y agoThen stil there is some greater purpose tht theyre a symptom of
- pstuart 4y agoI believe the greater purpose is the "garbage collection" of the brain's metabolic waste as well as the optimization of learning.
- standardUser 4y agoWell, yes, sleeping! Our brains have to do something while we sleep, but to assume that "something* has a functional purpose seems like a possibility, not a forgone conclusion.
- gmarx 4y agoI have wondered this too. What if dreams are a side-effect of a maintenance process, like if you were always kind and rewind your VHS but choose to watch them while doing so. Also occurred to me (thanks to chatGPT) that dreams might be produced in the same way chatGPT produces narratives, by running through a series of "what is likely to be next according to my trained network" decisions
- pstuart 4y agoExactly! Psychedelics can give a taste of this with the experience of separation of self, where it feels like you're an observer behind the one way mirror watching a focus group deliberate on various topics.
- suzzer99 4y agoEvolutionarily speaking, nature generally doesn't expend effort on something without a benefit attached to it.
- fear_and_coffee 4y ago[dead]
- college_physics 4y agoIs it a good thing to remember ones dreams or more of an indication of some sort of leakage between brain states?
- psychphysic 4y agoIt's just a thing. The brains pretty primed to forget dreams, but if you consciously think about them you can encourage recall. At the same time, who knows how accurate that recall is...
- standardUser 4y agoIt seems like a bold assumption to me that dreams have a functional purpose.
- Gunax 4y agoHuh, funny because I am thinking the opposite (that it's bold to suggest they dont have a purpose). I'm not basing it on any facts or data. It's just funny how our gut instincts are so different.
- Gunax 4y agoThis is only tangentially related, but when I was a teenager I practiced remembering my dreams. And now I cannot forget them. This is actually negativity impacting me. I often get reality confused with something I have dreamed. I cannot always remember if a conversation I had occurred in my dreams or in reality.
- jonnycomputer 4y agoI've had this somewhat while taking prozac. My dreams are so vividly remembered, and the contents of those dreams often so mundane, that its not always obvious to me which is which (e.g. dreaming that I bought eggs and milk). I think this is why we're actually supposed to forget memories; they are counterfactuals used to train the brain to avoid overfitting, but not supposed to interfere with our actual recollection of what has and has not occurred.
- dawatchusay 4y agoSo this is interesting. It’s not like this pseudo-randomization of inputs is necessarily a good thing because you’re actually introducing non-reality into the brain (like dreaming I’m in prison or can fly) but instead we’re being prepared for possibilities so that I can wake up and think, “what if that actually had happened?” Still, not sure if this is a productive thing at this stage in human existence or not, i.e. do I need to train for if I could fly or should I focus on reality?
- ordu 4y ago> do I need to train for if I could fly or should I focus on reality? I believe the answer is yes, you need to leave reality sometimes to undersrand reality. It is a very general pattern, to understand something you need to move out of it. You cannot see a forest while being in the forest, because trees are obscuring your sight. You cannot prove all true statements in arithmetic system without extending it. For example it is impossible to find a general solution for a cubic equation in real numbers, you need to go imaginary, find imaginary roots and toss away not real ones. The remaining roots will be real roots of real cubic equation. You cannot understand limitations of a scientific theory without considering an alternative theory. (Or a more general, or a narrower theory) You cannot prove hypothesis without disproving an alternative one. This pattern is everywhere. So it wouldn't surprise me a bit, if it turned out that you cannot digest a real experience without getting some unreal experience as well.
- suzzer99 4y agoDoes this explain why like 1/4 of my dreams involve me jumping into water with my cell phone in my pocket? I need to be prepared in case I get near water?
- popcalc 4y agoThis dream is sponsored by OtterBox.
- realjhol 4y agoOh so that's why I keep dreaming I'm at an important work function with no pants on. It must be because my brain is overfitted on wearing pants
- MagicMoonlight 4y agoYeah so imagine if tomorrow you forgot to wear pants. Now you'd know what would happen.