It's not surprising that an ensemble of embeddings beats single embeddings. I can't imagine it beats a single embedding fine-tuned to the task at hand, given s

It's not surprising that an ensemble of embeddings beats single embeddings. I can't imagine it beats a single embedding fine-tuned to the task at hand, given sufficient training data. Where ensemble embeddings might shine is when there is sufficient data to determine how to weight the ensemble, but insufficient data to effectively fine-tune. I'm not sure how often this case comes up.