3 ms·
Very interesting! I will share it with my DS friends. One thing I have struggled with optimizing is visualization and coordinate calculation of network graphs
by angelmass 8y ago
Very interesting! I will share it with my DS friends.
One thing I have struggled with optimizing is visualization and coordinate calculation of network graphs with 10s of millions of edges + nodes using networkX and most visualization tools. Have you looked into this utility for Vaex? Reading your article it sounds like it would be well-suited for it.
- maartenbreddels 8y agoI have not looked into it, maybe datashader can do this, which is a package purely focussing on viz, while vaex is more allround (although there is overlap). If you think vaex can be useful here, feel free to ask question/open issues https://github.com/vaexio/vaex https://github.com/vaexio/vaex
- bayesian_horse 8y agoThe bigger question is what you want to achieve by visualizing so many nodes. If you want a map that can be zoomed in to view individual nodes, you mainly need to compute coordinates for every node. Finding the arrangement of the node is probably what gets you in trouble, so you probably need a custom algorithm which scales better (and does poorer, probably). More interesting may be to identify clusters and either group them together or visualize these clusters as nodes themselves.
- blattimwind 8y agoGephi?