Right, so the point is, you have your vectors and now you have a way to find 4-tuples representing analogous sets of words. Great! You built an analogizing mach

Right, so the point is, you have your vectors and now you have a way to find 4-tuples representing analogous sets of words. Great! You built an analogizing machine! Now what do you do with the analogies? Neural networks can't get bored because they don't decide what to think about. They are like a total functional programming language with no control flow constructs. This is why they always give an answer in the same amount of time, regardless of the input vector. Surprisal is only a measure of information (given a distribution) and is only distantly related to what I'm talking about. However, if a neural network could choose to think more, choose to get new data and reconsider, choose to go back and look at that one from a couple minutes ago... then you'd also want it to have the ability to get bored. AKA to recognize when it is spending resources on an unprofitable line of inquiry.