4 ms·

El Grapho: High Performance WebGL Graph Data Visualization Engine

113 points by ArtWomb 8y ago

jarym 8y ago
This is very impressive stuff - thank you! I used three.js to PoC something like this in the past (nowhere near as polished though). As great as WebGL is it takes quite a bit of effort to get something polished so kudos to you.
holoduke 8y ago
Nice. does it support animations as well? Would be nice to have somekind of mouse hover effects
- jcims 8y ago
  I’d like graph animations where you can fix nodes and have edges come and go (think tcp connections in a cluster).
  - atlassubbed 8y ago
    I found this pretty easy to do with ngraph.forcelayout[0]. I used it to build a basic VDOM-diff visualizing demo[1], which involved dynamically animating edge changes to a graph. If you need something like that, it's pretty easy with ngraph in case this library doesn't support it. If you end up using this library, please let me know how it turns out! I'd love to see a demo of exactly what you're talking about. I ended writing quite a bit of mouse-event and panning/zooming code on top of ngraph so maybe this might be easier? [0] https://github.com/anvaka/ngraph.forcelayout https://github.com/anvaka/ngraph.forcelayout [1] https://github.com/atlassubbed/play-relax-visualized https://github.com/atlassubbed/play-relax-visualized
    - jcims 8y ago
      Will check this out thank you!!
- immortalloom 8y ago
  Yes it does. You can subscribe to different events like node-mouseover, node-click, node-mouseout, idle, etc. You can also customize tooltips. See the docs here: https://github.com/ericdrowell/ElGrapho https://github.com/ericdrowell/ElGrapho
pilooch 8y ago
I guess webgl makes things faster with hw acceleration ? We re using this https://github.com/alx/parasol https://github.com/alx/parasol that is based on sigma.js, usually along with ML. Does webgl allow to get a higher number of nodes and edges running fine in the browser ? Ability to get forceatlas within the browser is a plus in practuve weve found.
- lmeyerov 8y ago
  We do millions of edges in the browser fine via webgl (graphistry.com / https://github.com/graphistry/pygraphistry https://github.com/graphistry/pygraphistry ) . For forceatlas and other interactive & iterative analytics algs (t-sne, ...), it's hard to keep 100K+ nodes interactive via webgl. Most webgl attempts go naive all-to-all O(n^2), so the graph size quickly hits a ceiling. We rewrote to use GPUs end-to-end in the server so we can do trickier opencl/cuda algs that achieve O(nlogn) and O(n). Funny enough, users don't notice round trip costs -- or get even faster perf -- as our 2.0 engine does live streaming between client/cloud to hit ~20fps, similar to how streaming movies keep up frame rate. Our problem is now that JS only supports pointers up to 2GB :)
  - pilooch 8y ago
    Very cool! The docker install is proprietary as well or require an API key right ? Looks like a very finetuned tool, I'll recommend it around.
    - lmeyerov 8y ago
      Yep! Either a key to our SaaS for non-profit analysts or a docker for commercial, which supports the kinds of requests you see here as they add up, esp when supporting bigger scales. We're launching additional accessible options, if relevant. We started in pretty deep gov/enterprise, so fun to finally be emerging for simpler stuff :)
- immortalloom 8y ago
  That is correct. WebGL outperforms HTML5 Canvas 2d context, and HTML5 Canvas 2d context outperforms SVG. With WebGL, you are loading up the vertices of all the nodes and edges into the GPU, and they stay there. All the transformations like panning and zooming happen on the GPU. El Grapho uses picking for hit detection via ConcreteJS, so hit detection is nearly instantaneous for any number of nodes and edges, i.e. millions of them.
atlassubbed 8y ago
This is a funny coincidence. I stumbled upon this library just yesterday from the Big-O cheatsheet[0] site, but didn't think to post it here. Has anyone here done a detailed comparison between ElGrapho and ngraph.pixel[1] or vivagraph[2]? I currently use ngraph for an interactive VDOM diffing demo[3] and it works great for large amounts of nodes, but I guess I should check this out in more detail now. [0] http://bigocheatsheet.com/ http://bigocheatsheet.com/ [1] https://github.com/anvaka/ngraph.pixel https://github.com/anvaka/ngraph.pixel [2] https://github.com/anvaka/VivaGraphJS https://github.com/anvaka/VivaGraphJS [3] https://github.com/atlassubbed/play-relax-visualized https://github.com/atlassubbed/play-relax-visualized
sbr464 8y ago
I’ve yet to find a better visualization/usage of webgl (wrt graph networks) than this demo: https://frewsxcv.github.io/graphosaurus/examples/eve-universe/index.html https://frewsxcv.github.io/graphosaurus/examples/eve-univers...
- divan 8y ago
  Try out anvaka's projects: https://anvaka.github.io/pm/#/?_k=bk8ot0 https://anvaka.github.io/pm/#/?_k=bk8ot0

113 points by ArtWomb 8y ago

This is very impressive stuff - thank you! I used three.js to PoC something like this in the past (nowhere near as polished though). As great as WebGL is it takes quite a bit of effort to get something polished so kudos to you.

holoduke 8y ago

Nice. does it support animations as well? Would be nice to have somekind of mouse hover effects

jcims 8y ago
I’d like graph animations where you can fix nodes and have edges come and go (think tcp connections in a cluster).
- atlassubbed 8y ago
  I found this pretty easy to do with ngraph.forcelayout[0]. I used it to build a basic VDOM-diff visualizing demo[1], which involved dynamically animating edge changes to a graph. If you need something like that, it's pretty easy with ngraph in case this library doesn't support it. If you end up using this library, please let me know how it turns out! I'd love to see a demo of exactly what you're talking about. I ended writing quite a bit of mouse-event and panning/zooming code on top of ngraph so maybe this might be easier? [0] https://github.com/anvaka/ngraph.forcelayout https://github.com/anvaka/ngraph.forcelayout [1] https://github.com/atlassubbed/play-relax-visualized https://github.com/atlassubbed/play-relax-visualized
  - jcims 8y ago
    Will check this out thank you!!
immortalloom 8y ago
Yes it does. You can subscribe to different events like node-mouseover, node-click, node-mouseout, idle, etc. You can also customize tooltips. See the docs here: https://github.com/ericdrowell/ElGrapho https://github.com/ericdrowell/ElGrapho

pilooch 8y ago

I guess webgl makes things faster with hw acceleration ? We re using this https://github.com/alx/parasol https://github.com/alx/parasol that is based on sigma.js, usually along with ML. Does webgl allow to get a higher number of nodes and edges running fine in the browser ? Ability to get forceatlas within the browser is a plus in practuve weve found.

lmeyerov 8y ago
We do millions of edges in the browser fine via webgl (graphistry.com / https://github.com/graphistry/pygraphistry https://github.com/graphistry/pygraphistry ) . For forceatlas and other interactive & iterative analytics algs (t-sne, ...), it's hard to keep 100K+ nodes interactive via webgl. Most webgl attempts go naive all-to-all O(n^2), so the graph size quickly hits a ceiling. We rewrote to use GPUs end-to-end in the server so we can do trickier opencl/cuda algs that achieve O(nlogn) and O(n). Funny enough, users don't notice round trip costs -- or get even faster perf -- as our 2.0 engine does live streaming between client/cloud to hit ~20fps, similar to how streaming movies keep up frame rate. Our problem is now that JS only supports pointers up to 2GB :)
- pilooch 8y ago
  Very cool! The docker install is proprietary as well or require an API key right ? Looks like a very finetuned tool, I'll recommend it around.
  - lmeyerov 8y ago
    Yep! Either a key to our SaaS for non-profit analysts or a docker for commercial, which supports the kinds of requests you see here as they add up, esp when supporting bigger scales. We're launching additional accessible options, if relevant. We started in pretty deep gov/enterprise, so fun to finally be emerging for simpler stuff :)
immortalloom 8y ago
That is correct. WebGL outperforms HTML5 Canvas 2d context, and HTML5 Canvas 2d context outperforms SVG. With WebGL, you are loading up the vertices of all the nodes and edges into the GPU, and they stay there. All the transformations like panning and zooming happen on the GPU. El Grapho uses picking for hit detection via ConcreteJS, so hit detection is nearly instantaneous for any number of nodes and edges, i.e. millions of them.

atlassubbed 8y ago

This is a funny coincidence. I stumbled upon this library just yesterday from the Big-O cheatsheet[0] site, but didn't think to post it here. Has anyone here done a detailed comparison between ElGrapho and ngraph.pixel[1] or vivagraph[2]? I currently use ngraph for an interactive VDOM diffing demo[3] and it works great for large amounts of nodes, but I guess I should check this out in more detail now. [0] http://bigocheatsheet.com/ http://bigocheatsheet.com/ [1] https://github.com/anvaka/ngraph.pixel https://github.com/anvaka/ngraph.pixel [2] https://github.com/anvaka/VivaGraphJS https://github.com/anvaka/VivaGraphJS [3] https://github.com/atlassubbed/play-relax-visualized https://github.com/atlassubbed/play-relax-visualized

sbr464 8y ago

I’ve yet to find a better visualization/usage of webgl (wrt graph networks) than this demo: https://frewsxcv.github.io/graphosaurus/examples/eve-universe/index.html https://frewsxcv.github.io/graphosaurus/examples/eve-univers...

divan 8y ago
Try out anvaka's projects: https://anvaka.github.io/pm/#/?_k=bk8ot0 https://anvaka.github.io/pm/#/?_k=bk8ot0