3 ms·
I’m confused by the choice of word2vec for the embeddings, for “simplicity”. Using a transformer based model like Universal Sentence Encoder or SBERT is just as
by Xenoamorphous 3y ago
I’m confused by the choice of word2vec for the embeddings, for “simplicity”. Using a transformer based model like Universal Sentence Encoder or SBERT is just as easy and the results will be considerably better.
- fzliu 3y agoword2vec is seminal and outputs static embeddings - fairly easy for beginners to understand. The embeddings for each token change relative to context in attention-based MLMs and I figured it might be confusing for an introductory blog.
- cma 3y agoGood enough for Google: https://www.google.com/search?q=book+of+megadrive https://www.google.com/search?q=book+of+megadrive In the TPUv4 paper they say they are doing word based embeddings for search, maybe explaining why Google search has gotten so bad.