This is awesome! Especially because word2vec and it's derivatives are much more useful in some cases than transformers are. For instance, they store the vocabu

This is awesome! Especially because word2vec and it's derivatives are much more useful in some cases than transformers are. For instance, they store the vocabulary. I can query for similar words, or do vector math and convert it back to words. That is much harder to do with transformers. Also, not surprised at all that this kind of bug made it through inspire of how popular word2vec is. NLP is chalk full of tiny bugs like this and there is all sorts of low hanging fruit for interested enough researchers...