4 ms·

I’m glad to see support for GNNs with tensorflow. Working with gnns for the past few years, personally for me it gets tiring to roll my own framework.

claytonjy 5y ago
what's the state of GNN support elsewhere? does everyone else also roll their own, or are folks using Pytorch or something else?
- rytill 5y ago
  This is a big one: https://github.com/pyg-team/pytorch_geometric https://github.com/pyg-team/pytorch_geometric
- patagurbon 5y ago
  DGL is the other big one, it supports several frameworks (at least PyTorch and MXNet).
- UncleOxidant 5y ago
  GeometricFlux for GNNs in Julia: https://github.com/FluxML/GeometricFlux.jl https://github.com/FluxML/GeometricFlux.jl
- agentofoblivion 5y ago
  Deep Graph Library (DGL) is the big one, which can use either PyTorch, MXNet or Tensorflow as the backend and is developed by AWS. You also have PyTorch Geometric and Jraph, which is built on top of JAX and used mostly by researchers at DeepMind as far as I can tell.
- thecleaner 5y ago
  The bottleneck in GNN computations is that the aggregation ops cant be expressed as matrix operations and require writing custom kernels. This problem was solved in PyTorch with torch-scatter. The other bottleneck is subsampling (e.g k-hop) which also dont benefit from GPU support. Other than that the embedding aspects can just be written as nn ops.
H8crilA 5y ago
What's an example problem for which such networks work well?
- lmeyerov 5y ago
  Think of it as an ensemble for blending your normal NN features (ex: RNN for time/clickstreams) with a model that can also leverage useful graph features (document citations, app logins, chemicals connecting, social graphs). We think a lot about security/fraud and digital journeys, where NN + xgboost are popular in general, and graph is used seperately (or upstream) for looking at broader structure. GNNs help blend these models. For example, in analyzing malicious user accounts (ex: misinfo on twitter), we already get many time/nlp/etc scores for whatever events/entities we look at, and use the social network structure to ensure better propagation/blending, similar to why boosting and ensemble methods became popular to beginwith. Feel free to DM if interested, we are quite excited by this space and working on some things here.
  - quibono 5y ago
    I remember reading a bit about GNNs circa 2019. At that time it seemed to have mostly to do with point clouds (for LIDAR data and for 3-D modelling mostly) but I imagine things have changed lots on this front. Are there any interesting papers/resources you could recommend for one to get back up to speed?
    - atomflunder 5y ago
      From what I can tell, the field is indeed evolving very rapidly, but I have only worked on a specific application (knowledge graph completion), so I can't give an overview over all the current day applications. I can, however, recommend William Hamilton's excellent text book, which is available online [1]. [1] https://www.cs.mcgill.ca/~wlh/grl_book/files/GRL_Book.pdf https://www.cs.mcgill.ca/~wlh/grl_book/files/GRL_Book.pdf
    - lmeyerov 5y ago
      For enterprise relevance in our world, the exciting things have been handling heterogeneity via things like RGCNs, and handling bigger scales via DGL (GPU tricks, sampling tricks, ...). Imagine fraud, hacks, and entity resolution from everything you've recorded on a user interacting with a system. There are important cases like maps and chemistry that take more specialized techniques, but we focus on events/logs/etc. So less to say on the niche stuff, even if those niches cover big use cases like "how google maps works" or "how google auto-designs their TPUs" For the logs/events/transactions/clicks/devices/users/accounts cases, happy to chat, but maybe not as useful elsewhere :)
ackbar03 5y ago
Any reason for not using pytorch? They have torch geometric

claytonjy 5y ago

what's the state of GNN support elsewhere? does everyone else also roll their own, or are folks using Pytorch or something else?

rytill 5y ago
This is a big one: https://github.com/pyg-team/pytorch_geometric https://github.com/pyg-team/pytorch_geometric
patagurbon 5y ago
DGL is the other big one, it supports several frameworks (at least PyTorch and MXNet).
UncleOxidant 5y ago
GeometricFlux for GNNs in Julia: https://github.com/FluxML/GeometricFlux.jl https://github.com/FluxML/GeometricFlux.jl
agentofoblivion 5y ago
Deep Graph Library (DGL) is the big one, which can use either PyTorch, MXNet or Tensorflow as the backend and is developed by AWS. You also have PyTorch Geometric and Jraph, which is built on top of JAX and used mostly by researchers at DeepMind as far as I can tell.
thecleaner 5y ago
The bottleneck in GNN computations is that the aggregation ops cant be expressed as matrix operations and require writing custom kernels. This problem was solved in PyTorch with torch-scatter. The other bottleneck is subsampling (e.g k-hop) which also dont benefit from GPU support. Other than that the embedding aspects can just be written as nn ops.

H8crilA 5y ago

What's an example problem for which such networks work well?

lmeyerov 5y ago
Think of it as an ensemble for blending your normal NN features (ex: RNN for time/clickstreams) with a model that can also leverage useful graph features (document citations, app logins, chemicals connecting, social graphs). We think a lot about security/fraud and digital journeys, where NN + xgboost are popular in general, and graph is used seperately (or upstream) for looking at broader structure. GNNs help blend these models. For example, in analyzing malicious user accounts (ex: misinfo on twitter), we already get many time/nlp/etc scores for whatever events/entities we look at, and use the social network structure to ensure better propagation/blending, similar to why boosting and ensemble methods became popular to beginwith. Feel free to DM if interested, we are quite excited by this space and working on some things here.
- quibono 5y ago
  I remember reading a bit about GNNs circa 2019. At that time it seemed to have mostly to do with point clouds (for LIDAR data and for 3-D modelling mostly) but I imagine things have changed lots on this front. Are there any interesting papers/resources you could recommend for one to get back up to speed?
  - atomflunder 5y ago
    From what I can tell, the field is indeed evolving very rapidly, but I have only worked on a specific application (knowledge graph completion), so I can't give an overview over all the current day applications. I can, however, recommend William Hamilton's excellent text book, which is available online [1]. [1] https://www.cs.mcgill.ca/~wlh/grl_book/files/GRL_Book.pdf https://www.cs.mcgill.ca/~wlh/grl_book/files/GRL_Book.pdf
  - lmeyerov 5y ago
    For enterprise relevance in our world, the exciting things have been handling heterogeneity via things like RGCNs, and handling bigger scales via DGL (GPU tricks, sampling tricks, ...). Imagine fraud, hacks, and entity resolution from everything you've recorded on a user interacting with a system. There are important cases like maps and chemistry that take more specialized techniques, but we focus on events/logs/etc. So less to say on the niche stuff, even if those niches cover big use cases like "how google maps works" or "how google auto-designs their TPUs" For the logs/events/transactions/clicks/devices/users/accounts cases, happy to chat, but maybe not as useful elsewhere :)

ackbar03 5y ago

Any reason for not using pytorch? They have torch geometric