3 ms·
I like the analogy to multi-head attention. Is there work on something like it for collaborative filtering embeddings, eg splitting up the vector into multiple
by phreeza 5y ago
I like the analogy to multi-head attention. Is there work on something like it for collaborative filtering embeddings, eg splitting up the vector into multiple subvectors and calculating distances separately?
- zwaps 5y agoNot sure about CF but in terms of representation learning there are, of course, transformer approaches to embedding graphs ( the article essentially talks about the predictive power of closed triads in a graph) So, a good starting point might be SDNE type graph embeddings with a transformer architecture, since these are already encoder/decoder setups. There’s a ton of ways to get node similarity in a graph, but it turns out that all of them have a substantive theory basis (walks versus neighborhoods, higher order structure vs proximity etc) and none of them are really just “NN, embedding and done“ Perhaps also because it is really simple to see ex ante that a flat Euclidian embedding can not work.