4 ms·

Is there anything similar for network graphs?

jointpdf 8y ago
Perhaps Gephi? It was used for those Graphcore neural network visualizations (millions of edges and nodes) and they look stunning. [1] https://gephi.org/features/ https://gephi.org/features/ [2] https://www.graphcore.ai/posts/what-does-machine-learning-look-like https://www.graphcore.ai/posts/what-does-machine-learning-lo...
- simplyinfinity 8y ago
  I've used both gephi and cytoscape, i have about 2 million nodes with about 3 million edges, it's hard to visualise at once. however i've reduced the graphs to only sub networks with more than 5 connections, the remainder is about 600k, which is still painfully slow to visualise
maliker 8y ago
My team has had luck rendering an SVG from the graph and sending it to a browser. It works well for about 10k vertices and edges. Above that scale we use datashader, and we're investigating a potential to move to QGIS. We tried Gephi a few years ago and it had trouble at these scales.
lmeyerov 8y ago
Yep -- at https://github.com/graphistry/pygraphistry https://github.com/graphistry/pygraphistry, we started by making millions of nodes/edges interactive. If you use notebooks, can signup on our site and get going. The trick is we connect GPUs in the browser to GPUs in the cloud, and encapsulate it enough that you can stick to writing standard SQL/pandas/etc. We've been curious about server-side static tile rendering for larger graphs, but has been on the back-burner. (We already connect to GPUs on the server, so not rocket science.) Currently, we're actively increasing how much can be ingested + computed on, such as for finding influencers, communities & rings, etc. However, visualizing that hasn't been an operational priority for our users. More useful to generate the communities, and then either inspect individual ones, or see how communities stitch together: quickly run out of pixels otherwise due to too many edges. Likewise, we're building connectors to gigascale-petascale graph DBs: titan, janus, aws neptune, tigergraph, spark graphx, etc. We still are interested, but more for when we start supporting geographic maps: you can see that is the primary use for datashader. Also, because data art is fun :)
- jbednar 8y ago
  Geographic maps aren't the primary use for datashader; those are just easy examples that people can appreciate without a lot of explanation. In practice we use it for any large datasets that we don't want to subsample before visualizing them.
  - lmeyerov 8y ago
    Yes, definitely more capable. I've seen them primarily used in scatterplots (x, y, maybe z) + maps. Curious where else you're seeing the 80/20 breakdown if not there...
IanCal 8y ago
You can render networks in datashader, there's a line primitive. I added edge bundling (probably the slowest thing in datashader!) but I know there's examples of flight path rendering in the video I linked in another comment.
jbednar 8y ago
Datashader itself renders networks: http://datashader.org/user_guide/7_Networks.html http://datashader.org/user_guide/7_Networks.html

Perhaps Gephi? It was used for those Graphcore neural network visualizations (millions of edges and nodes) and they look stunning. [1] https://gephi.org/features/ https://gephi.org/features/ [2] https://www.graphcore.ai/posts/what-does-machine-learning-look-like https://www.graphcore.ai/posts/what-does-machine-learning-lo...

simplyinfinity 8y ago
I've used both gephi and cytoscape, i have about 2 million nodes with about 3 million edges, it's hard to visualise at once. however i've reduced the graphs to only sub networks with more than 5 connections, the remainder is about 600k, which is still painfully slow to visualise

maliker 8y ago

My team has had luck rendering an SVG from the graph and sending it to a browser. It works well for about 10k vertices and edges. Above that scale we use datashader, and we're investigating a potential to move to QGIS. We tried Gephi a few years ago and it had trouble at these scales.

lmeyerov 8y ago

Yep -- at https://github.com/graphistry/pygraphistry https://github.com/graphistry/pygraphistry, we started by making millions of nodes/edges interactive. If you use notebooks, can signup on our site and get going. The trick is we connect GPUs in the browser to GPUs in the cloud, and encapsulate it enough that you can stick to writing standard SQL/pandas/etc. We've been curious about server-side static tile rendering for larger graphs, but has been on the back-burner. (We already connect to GPUs on the server, so not rocket science.) Currently, we're actively increasing how much can be ingested + computed on, such as for finding influencers, communities & rings, etc. However, visualizing that hasn't been an operational priority for our users. More useful to generate the communities, and then either inspect individual ones, or see how communities stitch together: quickly run out of pixels otherwise due to too many edges. Likewise, we're building connectors to gigascale-petascale graph DBs: titan, janus, aws neptune, tigergraph, spark graphx, etc. We still are interested, but more for when we start supporting geographic maps: you can see that is the primary use for datashader. Also, because data art is fun :)

jbednar 8y ago
Geographic maps aren't the primary use for datashader; those are just easy examples that people can appreciate without a lot of explanation. In practice we use it for any large datasets that we don't want to subsample before visualizing them.
- lmeyerov 8y ago
  Yes, definitely more capable. I've seen them primarily used in scatterplots (x, y, maybe z) + maps. Curious where else you're seeing the 80/20 breakdown if not there...

IanCal 8y ago

You can render networks in datashader, there's a line primitive. I added edge bundling (probably the slowest thing in datashader!) but I know there's examples of flight path rendering in the video I linked in another comment.

jbednar 8y ago

Datashader itself renders networks: http://datashader.org/user_guide/7_Networks.html http://datashader.org/user_guide/7_Networks.html