3 ms·
Yeah, I guess this is fine if by "Transformers are Graph Neural Networks" we mean Transformers < GNN, rather than Tranformers == GNN. "Sentences are a fully co
by lexpar 7y ago
Yeah, I guess this is fine if by "Transformers are Graph Neural Networks" we mean Transformers < GNN, rather than Tranformers == GNN.
"Sentences are a fully connected graph". Ok fine, but that's a graph with basically no information embedded in its structure. GNNs are supposed to be useful for graphs that have interesting structure, right?
- chaitjo 7y agoNot necessarily. In fact, applying the Transformer/GNN on a full graph is seen by some people as 'discovering' or identifying some useful underlying graph structure. We had an interesting discussion on this on Twitter: https://twitter.com/PetarV_93/status/1233820829852618754 https://twitter.com/PetarV_93/status/1233820829852618754