I mean, the simplest answer is a matmul... Given embedding x, y, find M such that Mx ~= y. Easy to train so long as you've got access to both models to compute

I mean, the simplest answer is a matmul... Given embedding x, y, find M such that Mx ~= y. Easy to train so long as you've got access to both models to compute embedding over whatever you're interested in... (easy to extend to two layers mlp as needed. maybe ensure that x and y are zero mean and unit length to make training the matmul a bit easier.)