4 ms·
By no means I am an expert in deep learning, but lately I've been considering graph NNs snake oil of neural networks. There are no impressive results on any com
by lostmsu 6y ago
By no means I am an expert in deep learning, but lately I've been considering graph NNs snake oil of neural networks. There are no impressive results on any common tasks and that picture is supported by a recent paper claiming transformers to "contain a graph network inside".
- timr 6y agoGraph NNs are widely used in the chemical domain -- molecules are easily modeled as graphs, so it's a natural fit.
- VHRanger 6y agoSo are graph embedding methods, which the post discusses
- dumb1224 6y agoThe post can't be accessed with a Internal Server Error. Or is it just my ISP? I often confuse myself with other representation learning methods such as graph kernels https://en.wikipedia.org/wiki/Graph_kernel https://en.wikipedia.org/wiki/Graph_kernel Are there any relations between them at all?
- VHRanger 6y agoNo, my blog went down from hitting the front page of HN
- enriquto 6y agoAnd for large molecules (e.g., proteins) graph methods really start to shine.
- astrophysician 6y agoIn my opinion there aren't many deep learning applications that are fundamentally snake-oil, it's more that people who don't quite understand the limitations and advantages of particular methods that end up being (witting or unwitting) snake-oil salespeople. Deep graph NN's are very useful for lots of data that are inherently graph-structured (another commenter above mentioned chemistry applications, and there are lots of other examples). Whether or not for the time-being they give SOTA results on common datasets, being able to work directly with graph-structured data is quite appealing in many cases.
- mrfox321 6y agoGoogle maps improved their worse-case residuals of traffic estimation using GNNs (messaging passing Nets, not graph embeddings).
- vladTheInhaler 6y agoA bunch of fully connected layers also "contains a CNN inside", but empirically CNNs lead to better performance on image classification tasks. When you can cut away an enormous, and mostly irrelevant part of the parameter space, that has a lot of value.
- Der_Einzige 6y agoSo wrong oh my God! This beauty got an outstanding paper award at COLING 2020. GNNs are not applied to "common tasks" because they are primarily suited for environments where graph structure is present. In this case, predicting how the Indian congress will vote. https://www.aclweb.org/anthology/2020.coling-main.426/ https://www.aclweb.org/anthology/2020.coling-main.426/