4 ms·
Very cool! I like the visualizations a lot. Did you try to get an interpretation for what each quantized vector dimension means (have just skimmed, not read)?
by andreyk 9y ago
Very cool! I like the visualizations a lot. Did you try to get an interpretation for what each quantized vector dimension means (have just skimmed, not read)?
Also, I am curious why you chose to go straight to publishing on Arxiv? I am actually also in CS224N right now and have a project me and my collaborator feel is publication worthy, but our plan is to go the normal route of submitting to a conference and only putting it on Arxiv after the review process (though our code is open source, so now that I think about it maybe that's not that useful a plan...).
- maxlam 9y agoDefinitely tried to figure out if the dimensions mean anything -- as far as I can tell they don't really mean much :(
- Maybestring 9y agoIf you want them to be meaningful without changing the model... Couldn't you rotate the basis to minimize the distance between each basis vector and it's nearest neighbor?
- maxlam 9y agoHaven't tried this but this is definitely a good idea for visualizing what's going on!
- loxias 9y agoOne of the fascinating things about neural embedding such as these is that the individual component dimensions have NO "real" semantic meaning to us humans. It's better to think of them as single points in a higher dimensional space. (of course, with a clever network design you could probably FORCE "meaning" onto some components)