16 ms·
> So the ancient mystery of why we need sleep might have just been answered. I would be very surprised if sleep would serve only one purpose. In complex interc
by kbrkbr 1y ago
> So the ancient mystery of why we need sleep might have just been answered.
I would be very surprised if sleep would serve only one purpose. In complex interconnected systems you usually don't get far with monocausal explanations.
- lolive 1y agoAI frenzy almost convinced me that sleep was the training of our neural network with all the prompts of the day. And now this /o\
- ozgung 1y agoThat's what I still 'believe'. Wake-sleep algorithm [1] is a good start for speculation. I think brain needs to be in a different mode to reorganize its weights and to forget unnecessary things to prevent overfitting. In this mode we happen to be unconscious. I also believe dreams are just hallucinations caused by random noise input to the system. The brain converts noise distribution to a meaningful distribution and samples from that. I have zero evidence btw, but I believe these are related. [1] https://en.wikipedia.org/wiki/Wake-sleep_algorithm https://en.wikipedia.org/wiki/Wake-sleep_algorithm
- dr_dshiv 1y agoWhen we don’t sleep, we can lose sensory and cognitive coherence. Mild visual hallucinations begin and reality can start slipping. Sleep itself is characterized by coherent neural activity— the large number of brain regions with synchronized neural activity. The slow waves where huge numbers are all firing close together in a rhythm. Low frequency and high amplitude delta brainwaves (1-2 hertz). Complex adaptive brain activity requires more complex firing than a simple rhythmic frequency. So, in a way, the complex activity must be stopped in order to support global synchrony. Why would our neurons want to all fire synchronously? Well, it is healthy for neurons to fire together in a causal manner— neurons release growth hormones then. That neural growth during synchronized firing is the basis of “neurons that fire together wire together.” And it doesn’t seem coincidental that a successfully predicted model feels good, as in the case of successfully throw a ball in a basket. Neurons are trying to predict other neuron firing and respond to it. If they are unable to effectively, they may become like the 1/3 of our baby neurons in the cortex — they will be pruned and die. Good feelings is positive reinforcement—behaviors leading to good feelings get reinforcement. The feeling of harmony or harmonization, where we have to balance a broad set of internal neural impulses, feels good when we do it well. We feel harmony in music — and in our own internal sensory resonance to the world. Hypothesis 1: the harmonization of neural activity might cause conscious feelings due to the convergence of the activity to platonic forms (see Platonic Representation Hypothesis in LLM research). Returning to sleep — this is a proposal for why sleep feels good. Synchronization might intrinsically feel good. But because the sleep also disrupts your working memory contextual attunements (ie, whatever your day was about) - taking your brain into deep synchrony — it strengthens the overall dendritic connections between the synchronizing neurons. And, because it wears off the edges of your previous experiences — you can return refreshed. In this way, sleep seems to contribute to the overall integrity of the operation of our intelligence. Without it, we lose integrity and internal harmony. And yet, not sleeping is one of my favorite drugs. Can be a major performance enhancer, even if it is variable. Hypothesis 2: Not sleeping increases the (statistical) temperature of the brain.
- skirmish 1y ago> not sleeping is one of my favorite drugs. Can be a major performance enhancer, even if it is variable. Sleep deprivation is a well known treatment for depression [1]. Maybe you lean to the depressive side, that would explain positive effects. [1] https://en.wikipedia.org/wiki/Sleep_deprivation#Treating_depression https://en.wikipedia.org/wiki/Sleep_deprivation#Treating_dep...
- incognito124 1y agoIt's not training as much as it's discarding bad examples. Sort of.
- andrepd 1y agoJesus christ, not even a biology thread is safe in the orange website.
- gitremote 1y agoIndeed. Animals without linguistic ability (like fruit flies) need sleep, but after ChatGPT's release in 2022, now tech bros think LLMs specifically might model the animal brain in general because of anthropocentrism and anthropomorphism. It's also a fundamental misunderstanding of how LLMs work, mixing up inference with training.
- immibis 1y agoIt was applicable to all neural networks, not just LLMs. Can we say that after ChatGPT's release in 2022, now antitech bros think everything is about LLMs specifically?
- gitremote 1y agoThe statement was "AI frenzy almost convinced me that sleep was the training of our neural network with all the prompts of the day." Prompts are specific to LLMs. Most neural networks don't have prompts. Additionally, prompts happen during LLM inference, not LLM training. There are many non-technical people who claim they have experience "training" LLMs, when they are just an end user who added a lot of tokens to the context window during inference.
- sva_ 1y ago> Additionally, prompts happen during LLM inference, not LLM training. It is pretty common during the fine-tuning phase.
- gitremote 1y agoSure. Foundation models aren't fine-tuned, and companies fine-tune foundation models to optimize user experience. So they are modeling the animal brain on an even more specific type of LLM that happens to be related to being a consumer of AI products.
- dspillett 1y ago> sleep was the training of our neural network with all the prompts of the day Periods of sleep certainly seem to be used in that sort of way, but that is an extra use evolution found for the sleep cycle once it existed rather than the reason sleep developed in the first place. There are a number of things that seem tied to, or at least aligned with, our wake/sleep cycle that likely didn't exist when sleep first came about.
- nahuel0x 1y agoCurious how the zeitgeist changes, on a previous AI cycle we could thought sleep was required/generated by a semi-space garbage collection brain-LISP process :)
- AIPedant 1y agoYou didn’t need this study to realize that this was wrong: jellyfish and hydras also sleep despite not having a central nervous system. There are indications that sponges sleep too, despite not having any neurons (though obviously it’s somewhat ambiguous): https://www.science.org/content/article/if-alive-sleeps-brainless-creatures-shed-light-why-we-slumber https://www.science.org/content/article/if-alive-sleeps-brai...
- xgkickt 1y agoRebalancing the weights.
- yreg 1y agoIt would make sense if there was a monocausal explanation of why ancient ancestors started sleeping, but then other body functions started making use of the sleeping system since it was at hand.
- hhjinks 1y agoHey, that's Hyrum's Law!
- Waterluvian 1y agoThis is why I implemented private static readonly final sleep()
- baq 1y agoWhat if you dream about reflections?
- Filligree 1y agoSleep is still detectable via CPU load, so I added a thread that checks for load and runs some critical cleanup processes when it drops below a preset threshold. Hope you don’t mind.
- tux3 1y agoThe obligatory related XKCD: https://xkcd.com/1172/ https://xkcd.com/1172/
- tomrod 1y agoHyrum would be so proud!
- deleted 1y ago[deleted]
- vendiddy 1y agosounds like legacy code
- eutropia 1y agoTFA also acknowledges this: > There could well be many other functions that have since joined in with the sleep cycle (such as memory consolidation), but the authors hypothesize that mitochondrial function is the process that underlies all of them. If you need oxygen, then you need sleep!
- deleted 1y ago[deleted]
- hearsathought 1y ago> If you need oxygen, then you need sleep! Do plants sleep? Don't some insects, like flies, live without any sleep?
- mock-possum 1y agoPlants breathe out oxygen, like we breathe out the other one.
- andy99 1y agoThat's true for photosynthesis but don't they still have oxygen respiration (i.e. oxidizing sugar for energy?)
- tingletech 1y agoyes, at night they breath oxygen. Maybe they sleep during the day.
- wongarsu 1y agoBut their cells still consume oxygen during the day, don't they? In sunshine they produce more oxygen than they consume, but the cells are still fundamentally powered by mitochondria oxidizing glucose
- 1y ago
- steve1977 1y agoIt might have one evolutionary root cause and then got hijacked for other uses as well.
- ge96 1y agoWhen I'm awake for a very long time (32hrs+) it feels like there is poison built up in my mind, then sleep clears it up/feel better. Also if you lift in the mornings you feel lack of sleep/alcohol sleep disruption.
- xnx 1y ago> it feels like there is poison built up in my mind, then sleep clears it up/feel better. I'm not sure how common this is, but I feel this acutely after sustained mental exertion (e.g. reading informational material for a few hours). A deep 15 minute nap takes the feeling away completely without any grogginess.
- skirmish 1y ago> A deep 15 minute nap takes the feeling away Almost the same here but it's not a deep nap for me. I relax, start seeing dream-like images in my mind (yet still drifting into-out of conscious awareness), then in ~15 minutes I feel energy build up and am ready to jump up and go. I would say that the darn alarm clock prevented me from completing a sleep cycle properly in the morning, and now I did complete it and made my brain happy.
- bruce343434 1y agoHow do you ensure you are asleep for 15 minutes? Do you have a smart watch that detects when you drift asleep and can start a timer then? Or are you not losing consciousness, but are you simply closing your eyes and meditating?
- xnx 1y agoFor these instances where I get urgently fatigued ("brain tired") in the daytime, I close my eyes and fall asleep in 1-2 minutes. I'm definitely unconscious. I don't set any alarm and naturally wake up in ~15 minutes. It's been as short as 8 minutes, or as long as 30, but probably averages around 15. "Body tired" is different and requires the normal multiple hours of sleep.
- legohead 1y agoI wouldn't. The current theories on sleep and "brain needs sleep" always struck me as a stopgap theory. Even spent some time with GPT arguing about it and never felt fully convinced, like the real reason was still missing.