There's something called "attentional neural networks" that attempt to do this. They tend to do very well in reading natural language, IIRC, but I've also seen

There's something called "attentional neural networks" that attempt to do this. They tend to do very well in reading natural language, IIRC, but I've also seen them applied to video.