I think that's kind of the point of the paper. The model is based on un-normalized dot products, and wasn't deliberately designed to produce meaningful cosine s

I think that's kind of the point of the paper. The model is based on un-normalized dot products, and wasn't deliberately designed to produce meaningful cosine similarities. They are showing that, in that case, cosine similarities might be arbitrary and not as useful as people might assume or hope.