2 ms·

The brain does a lot of hierarchical prediction with sense data. When new information comes in, it references "something" and adjusts the sense data likewise.

The brain does a lot of hierarchical prediction with sense data. When new information comes in, it references "something" and adjusts the sense data likewise. That's why when you shift your eyes quickly, you see blurred images pass by. In reality, you should be seeing complete black because the brain doesn't actually process visual information that shifts so quickly. But your brain "knows" it should see...well something. And so it fits that blurred passthrough as compensation. Completely made up data. But not ungrounded data, data that seems like it should fit according to that "something". That "something" being referenced is a world model. That's a sub-conscious example but you can think of a lot of times people consciously reference "something" to better take actions or make decisions. For Othello-GPT, training seemed to induce a learned board state that the model referenced at every Inference to make moves. It seems that fully accurate or not, deep predictions might require "something" to reference. Else perhaps you might need to derive everything from first principles at every turn. Not very efficient.

SanderNL 3y ago
It always seemed sensible to me that we are looking at a dreamlike construction all the time, not just holes filled in but literally everything you see is made up, but grounded in sensory data (when not psychotic or dreaming). This also nicely explains the clarity and realism of lucid dreaming, because it’s just doing whatever you are doing during the day but without (external) grounding sensory data.
- emrah 3y ago
  > but without (external) grounding sensory data. Or perhaps with external data that is normally surpressed by data that comes through sensory organs (e.g. other parts of EM spectrum).
astrange 3y ago
> That "something" being referenced is a world model. It doesn't need to be a whole understanding of the nearby world to build an image. Basically it's an event camera (https://en.wikipedia.org/wiki/Event_camera https://en.wikipedia.org/wiki/Event_camera) and just needs to integrate recent visual "samples". But if you do need to understand objects near you in detail (say, to move them), you don't do it just by looking. You investigate them by moving closer to them, picking them up, etc. If you need to "create a world model", then it's an activity rather something your senses just do.

SanderNL 3y ago

It always seemed sensible to me that we are looking at a dreamlike construction all the time, not just holes filled in but literally everything you see is made up, but grounded in sensory data (when not psychotic or dreaming). This also nicely explains the clarity and realism of lucid dreaming, because it’s just doing whatever you are doing during the day but without (external) grounding sensory data.

emrah 3y ago
> but without (external) grounding sensory data. Or perhaps with external data that is normally surpressed by data that comes through sensory organs (e.g. other parts of EM spectrum).

astrange 3y ago

> That "something" being referenced is a world model. It doesn't need to be a whole understanding of the nearby world to build an image. Basically it's an event camera (https://en.wikipedia.org/wiki/Event_camera https://en.wikipedia.org/wiki/Event_camera) and just needs to integrate recent visual "samples". But if you do need to understand objects near you in detail (say, to move them), you don't do it just by looking. You investigate them by moving closer to them, picking them up, etc. If you need to "create a world model", then it's an activity rather something your senses just do.