Likely not, embeddings are very crude. Embeddings of a text is just an average of "meanings" of words. As is embeddings lack a lot of tricks that made transfor

Likely not, embeddings are very crude. Embeddings of a text is just an average of "meanings" of words. As is embeddings lack a lot of tricks that made transformers so efficient.