4 ms·
Played a bit with fastText months ago. The issue I had with it is that unlike CNNs/RNNs, the relative position of a word doesn't matter as much (only as a part
by minimaxir 10y ago
Played a bit with fastText months ago. The issue I had with it is that unlike CNNs/RNNs, the relative position of a word doesn't matter as much (only as a part of a context window during training the embeds), and so results can be worse depending on the case. However, for CPU-workloads, fastText is certainly faster, especially since subword information is also incorporated.
I have Python code to process text files into a fastText/friendly format so I may clean that up and see how these pre-trained embeds work.
(although, the English embeds are 10.36 GB; that might be a tough pill to swallow for training on machines with only 8 GB of RAM)
- exgrv 10y agoRegarding the size of the word vectors files: the text files are sorted by frequency, so it is possible to easily load the top k words only. We might also release smaller models in the future, for training on machines without large memory.
- morenoh149 10y agofwiw I have 32gb on my workstation and my personal laptop is maxed out at 16gb. Keeping within these thresholds may be useful to others.