3 ms·
yes, true. there has been progress on NLP SOTA, just not from graph NN architectures. I have thought about using graph NN's where I am currently using RNN's, bu
by huac 7y ago
yes, true. there has been progress on NLP SOTA, just not from graph NN architectures. I have thought about using graph NN's where I am currently using RNN's, but they seem fairly immature / difficult to train over very large datasets so have not made much progress.
I was following the work of https://www.octavian.ai/ https://www.octavian.ai/, but have not seen much recently. http://web.stanford.edu/class/cs224w/info.html http://web.stanford.edu/class/cs224w/info.html also looks interesting. looking briefly over the student projects, there appear to be a couple nlp projects (e.g. http://web.stanford.edu/class/cs224w/project/26418192.pdf http://web.stanford.edu/class/cs224w/project/26418192.pdf) but most leverage knowledge graph concepts.
- nl 7y agoOn the contrary, Graph NN's I've found Graph NNs extraordinarily useful and very easy to train! I'm not sure what kind of problems you are trying to use them for, but generally they are really useful for node classification or recommendation type tasks. So for example in the knowledge graph context, you can do things like give it a tiger, jaguar and panther and it will find nodes like lions and leopards. https://arxiv.org/pdf/1901.00596.pdf https://arxiv.org/pdf/1901.00596.pdf is a survey paper which has a decent overview of the tasks they are useful for. http://research.baidu.com/Public/uploads/5c1c9a58317b3.pdf http://research.baidu.com/Public/uploads/5c1c9a58317b3.pdf is using them for QA, but I'm not familiar enough with the dataset to evaluate it fairly.
- huac 7y agoI spent some time experimenting with https://github.com/facebookresearch/PyTorch-BigGraph https://github.com/facebookresearch/PyTorch-BigGraph and a few other libraries, I ran into some challenges given that my dataset was very large (O(100M) edges) and very sparse; and didn't pursue it further. The papers you linked are very interesting, I will have to dig further! One more recent writeup: https://eng.uber.com/uber-eats-graph-learning/ https://eng.uber.com/uber-eats-graph-learning/ -- a real production use case, seems promising to explore more.
- nl 7y agoI like https://github.com/facebookresearch/Starspace#graphspace-link-prediction-in-knowledge-bases https://github.com/facebookresearch/Starspace#graphspace-lin... for fast and easy graph embedding generation via a GNN.