4 ms·

Correct, but with a DB involved, there are additional benefits. This implementation is still rudimentary, but there are many options for tighter integrations, e

by mbuda 4y ago

Correct, but with a DB involved, there are additional benefits. This implementation is still rudimentary, but there are many options for tighter integrations, e.g., a specialized DB index that will copy data to a GPU continuously, significantly reducing latency in the production environment!

lmeyerov 4y ago
Yes, that was what motivated my comment on for GNN inferencing, or more generally, graph-informed inferencing servers: We generally find it's better to separate DB nodes from pricey GPU parts -- always-on $$$ on db vs during-use GPU (daytime, ...), elastic scaling to multi-GPU, not having heavy GPU analytics jobs bursts break your DB server SLAs (!!!), etc. But not always. Most to mind, we're seeing: - Internal data teams: When a box has ~no one using it on average it's ok to be taken over. Like a data scientist's personal dev box, or a DS team server. In these cases, everything is on the ~same box: notebooks, rapids containers, graphistry runtimes, whatever DB, esp. with bulk Apache Arrow in/out like the new Neo4j support here or RAPIDS zero-copy GPU dataframe pointers, and Spark has had this for years. (I'm guessing memgraph is/will support this too if they're already integrating cugraph.) - Production: The GNN inferencing case is interesting because it is likely much more tightly scoped and steady for being safe in production DBs with basic delivery SLAs. We aren't really seeing production users doing GPU + DB on the same node (except ~small-scale GPU DBs like say omnisci/heavy.ai) due to the above issues. Inferencing is inching closer, with graph "entity 360" context today is looking more like like HBase/Mongo/S3 system <> cpu/gpu inferencer. An interesting thing with memgraph's architecture is we're getting into customer convs where it may make sense to the inferencer compute closer to the DB node. I probably still wouldn't do a big A100 on a production DB node, but attaching some T4s vs running them separately is interesting!
- mbuda 4y ago
  Very good points!

by mbuda 4y ago

lmeyerov 4y ago

Yes, that was what motivated my comment on for GNN inferencing, or more generally, graph-informed inferencing servers: We generally find it's better to separate DB nodes from pricey GPU parts -- always-on $$$ on db vs during-use GPU (daytime, ...), elastic scaling to multi-GPU, not having heavy GPU analytics jobs bursts break your DB server SLAs (!!!), etc. But not always. Most to mind, we're seeing: - Internal data teams: When a box has ~no one using it on average it's ok to be taken over. Like a data scientist's personal dev box, or a DS team server. In these cases, everything is on the ~same box: notebooks, rapids containers, graphistry runtimes, whatever DB, esp. with bulk Apache Arrow in/out like the new Neo4j support here or RAPIDS zero-copy GPU dataframe pointers, and Spark has had this for years. (I'm guessing memgraph is/will support this too if they're already integrating cugraph.) - Production: The GNN inferencing case is interesting because it is likely much more tightly scoped and steady for being safe in production DBs with basic delivery SLAs. We aren't really seeing production users doing GPU + DB on the same node (except ~small-scale GPU DBs like say omnisci/heavy.ai) due to the above issues. Inferencing is inching closer, with graph "entity 360" context today is looking more like like HBase/Mongo/S3 system <> cpu/gpu inferencer. An interesting thing with memgraph's architecture is we're getting into customer convs where it may make sense to the inferencer compute closer to the DB node. I probably still wouldn't do a big A100 on a production DB node, but attaching some T4s vs running them separately is interesting!

mbuda 4y ago
Very good points!