3 ms·
Most transformer models require that the data is euclidian (e.g. a sequence or image). Non-Euclidean deep learning methods like graph neural networks (GNNs) are
by grandmczeb 6y ago
Most transformer models require that the data is euclidian (e.g. a sequence or image). Non-Euclidean deep learning methods like graph neural networks (GNNs) are much more general and can accept almost arbitrarily structured data (social networks, point clouds, trees, etc). You can actually model transformers as a special case of GNNs[1].
[1] https://towardsdatascience.com/transformers-are-graph-neural-networks-bca9f75412aa https://towardsdatascience.com/transformers-are-graph-neural...
- bollu 6y agoAll smooth manifolds can be realised as submanifolds of euclidean space. This is the famous Nash embedding theorem: https://en.wikipedia.org/wiki/Nash_embedding_theorem https://en.wikipedia.org/wiki/Nash_embedding_theorem Hence, it's unclear if the power gained by GNNs is actually 'useful'.
- grandmczeb 6y agoWhat do you mean by useful? GNNs get better performance than other methods on lots of real tasks. It's the same as CNNs being useful in practice despite not having more "theoretical" power than MLPs.
- lostmsu 6y agoWhich tasks where GNNs get better performance have standardized benchmarks?