3 ms·
Yes, I've not looked at fasttext but word2vec is a simple 1 hidden-layer network to learn word embeddings, which can then be used as pre-trained word embeddings
by aglionby 9y ago
Yes, I've not looked at fasttext but word2vec is a simple 1 hidden-layer network to learn word embeddings, which can then be used as pre-trained word embeddings in other tasks.
CNNs and RNNs would be used in the next stage of the pipeline for whatever your task is (machine translation etc), probably as some way of combining the word vectors. RNNs are especially nice as they can deal with variable length sentences. Note also that it's possible for systems to learn their own systems as part of training, rather than using pre-trained ones from word2vec etc.
- wodenokoto 9y agoI meant for generating wordvectors. Word2vec and fasttext use nn that are so shallow that the can be described as types of linear regression.