4 ms·
Yes The embedding needs to embed a social graph with similarity weights and predict ties. Of course, such partly cyclical graphs can not be guaranteed to embed
by zwaps 5y ago
Yes
The embedding needs to embed a social graph with similarity weights and predict ties. Of course, such partly cyclical graphs can not be guaranteed to embed in a low dimensional (aka far away from N) Euclidian or even Hyperbolic space.
Which means, among other things, that throwing any old Pytorch NN at the problem will not work, similar to how you can’t just Word2Vec to GPT performance since the latter employs many parallel Euclidian embeddings (or, in other words, any selection or aggregation of layer embedding is contextual).
Not that you can’t do it on Graphs, it’s just that understanding the issues is very helpful.
For example, understanding that Euclidian embeddings imply a triangle inequality while social graphs of similarity do not, is a crucial thing to know before throwing stuff at pytorch.
- phreeza 5y agoI like the analogy to multi-head attention. Is there work on something like it for collaborative filtering embeddings, eg splitting up the vector into multiple subvectors and calculating distances separately?
- zwaps 5y agoNot sure about CF but in terms of representation learning there are, of course, transformer approaches to embedding graphs ( the article essentially talks about the predictive power of closed triads in a graph) So, a good starting point might be SDNE type graph embeddings with a transformer architecture, since these are already encoder/decoder setups. There’s a ton of ways to get node similarity in a graph, but it turns out that all of them have a substantive theory basis (walks versus neighborhoods, higher order structure vs proximity etc) and none of them are really just “NN, embedding and done“ Perhaps also because it is really simple to see ex ante that a flat Euclidian embedding can not work.