3 ms·The context comes from the attention mechanism, not from word embeddings.by eggdaft 2y agoThe context comes from the attention mechanism, not from word embeddings.mjburgess 2y agoRun attention on an ordinal word embedding and see what happenseggdaft 2y agoWell yes, necessary but not sufficient, obviously.