3 ms·
the original models can only generate sentence embeddings, correct? can a token prediction model use this?
by billconan 2y ago
the original models can only generate sentence embeddings, correct?
can a token prediction model use this?
- stephantul 2y agoHey, thanks for the question. You are right, this can only be used to distill models for producing embeddings, although it's not restricted to encoder-only models. For example, you could use it on Llama, but you'd just get a bunch of embeddings out, not a model that can be used to do next-token prediction.