I think this nips at the core. The reality we live in is an unlimited stream of inputs. Their notion of attention is important. Modeling weight of inputs seems

I think this nips at the core. The reality we live in is an unlimited stream of inputs. Their notion of attention is important. Modeling weight of inputs seems to be an interactive componation of bidding the value of past and future across infinite dimensions. By encoding them as a single input stream they are nudging at what a brain does well when is works right: contextulization.