3 ms·
Don't know if you tried it on 16.09 when this was harder, but now you can just strip off the /c/en/ and the rest has exactly the same normalization as word2vec.
by rspeer 9y ago
Don't know if you tried it on 16.09 when this was harder, but now you can just strip off the /c/en/ and the rest has exactly the same normalization as word2vec. Even including the weird thing with changing multiple digits to # signs.
There is also a Python function included that does the normalization. But it's pretty simple and if you're already using word2vec you're already doing it.
- nl 9y agoReally? I'll check what version we were using, but the problem I'm referring to was vectors for concepts with the underscore in them. We did work out removing /c/<lang>/