I don't think it's a legit use of t-SNE. Distances in its space don't really provide anything except maybe local structure and are non-deterministic. After all,

I don't think it's a legit use of t-SNE. Distances in its space don't really provide anything except maybe local structure and are non-deterministic. After all, you can query raw vectors for a list of most similar ones.