3 ms·
It's a generalization of a fully connected layer, where you give the network an adjacency matrix for data routing between nodes. This allows for designing spars
by visarga 6y ago
It's a generalization of a fully connected layer, where you give the network an adjacency matrix for data routing between nodes. This allows for designing sparse data flows and learning relations between pairs of input elements.
Graph NN layer: f(X, A) = g(AXW), where g is an activation
Fully connected layer: f(X) = g(XW), like GNN where the adj. matrix A=I
To put it in context a few years ago GNNs became a hot field. They are very similar in a way to transformers because both do pairwise interactions between elements, the difference being that GNNs use explicit and transformers implicit graphs.
- benibraz 6y agoIs this a simplified interpretation of GNNs? because I couldn't find a reference for that. EDIT: found it! https://arxiv.org/pdf/1609.02907.pdf https://arxiv.org/pdf/1609.02907.pdf