The most remarkable take-away from this whole genre of word-embedding is that just by doing 'dumb averages' of word contexts and then optimizing the 'vector[wor

The most remarkable take-away from this whole genre of word-embedding is that just by doing 'dumb averages' of word contexts and then optimizing the 'vector[word]' on the input (and output sides), you end up with a SEMANTIC understanding of the English language in the word vectors. This paper is the latest in the series (across multiple researchers), and seems to boil the task down to its bare minimum : Just a raw least-squares optimization works. And instead of the 'linguistic knowledge' being smuggled into the problem set-up increasing (initially, people used tree-embeddings, and WordNet bootstrapping, in the 2003 papers), this is getting rid of almost all structure. And ending up with better results. So, instead of semantics being a naturally very deep problem, apparently common sense understanding can be derived from surface statistics. IMHO, more people should be excited about this (from an AI standpoint).