In your experience, in general when comparing vector embeddings, what layer of the net works best? Are the embeddings from the first hidden layer, the second,

In your experience, in general when comparing vector embeddings, what layer of the net works best? Are the embeddings from the first hidden layer, the second, the last, etc?