7 ms·

BioFabric: New way to visualize large interaction networks without any hairballs

74 points by samuell 12y ago

samuell 12y ago
This is among the cooler (as in more useful) visualizations I've seen in a long time.
samuell 12y ago
Btw, if you want to play with some own data, you can run it with Java web start on [1]. As something simple to try out, I figured that I could create a file 'test.sif' (in SIF file format [2]) like this, just to get some example: item1 conn item2 item1 conn item3 item2 conn item1 item2 conn item3 item3 conn item1 item3 conn item2 ... (yea, you get the idea) And then visualize it by going: "File > Import > Import from SIF" Turns out there is more "demo SIF files" in [3] though. Hey, this is kinda fun :) [1] http://www.biofabric.org/#launch http://www.biofabric.org/#launch [2] "Simple Interaction Format" http://wiki.cytoscape.org/Cytoscape_User_Manual/Network_Formats http://wiki.cytoscape.org/Cytoscape_User_Manual/Network_Form... [3] http://www.biofabric.org/#sif http://www.biofabric.org/#sif , http://www.biofabric.org/sifFiles.html http://www.biofabric.org/sifFiles.html
irickt 12y ago
Compare this earlier genealogy visualization: http://www.aviz.fr/geneaquilts/ http://www.aviz.fr/geneaquilts/
astanway 12y ago
These seems pretty cool. My worry is that you lose intuitions of centrality with this, with the exception of degree centrality, since nodes with the highest degree are sorted from left to right. How can we capture other measures of centrality? What about weak ties?
- wjrl 12y ago
  Sorry, newbie at commenting, so my replies are actually in separate comments below. Summary: you can create a layout that orders nodes, and edges, to fit what you are trying to look at.
wjrl 12y ago
BioFabric layouts are just a linear ordering of the nodes and edges, and you can specify those with input files. For example, you can break out clusters and show intra- and inter- cluster links separately. See http://www.biofabric.org/gallery/index.html#LesMiz http://www.biofabric.org/gallery/index.html#LesMiz
wjrl 12y ago
Note that since the nodes-as-lines approach gives you a new degree of freedom to work with, you can organize edges into semantically useful groups by placing similar links next to each other. E.g. think about link communities instead of node communities.
untilHellbanned 12y ago
As a systems biologist and web developer, I say major thumbs up for the presentation of the idea, but a thumbs sideways for the actual visualization method. I'm not sure this makes biology clearer. Seems like rearranging deck chairs.
- Fomite 12y ago
  This was my thought as well - I work with large contact networks for epidemiology, and while getting away from hairballs would be good in theory, I'm not seeing how a figure built using this technique will make me go "Oh! Yes, now I see it!"
  - wjrl 12y ago
    Placing infected individuals in the top rows, ranking uninfected individuals by probability of infection, lay out the network. Overall shape of network might give clues to network structure? I find that with contact networks, dynamic networks (links appear and disappear) tend to have problems with force-directed layouts bouncing around a lot.
    - Fomite 12y ago
      It's possible. My issue is I'm not sold that in networks of any size (the Le Mis network is pretty small) that this doesn't end up in a similarly difficult to interpret mess. I may try it out, but the initial demo didn't trigger much beyond "Neat". But to be honest, I don't tend to work in .sif format networks, and I don't have time tonight to try to get something I actually use in that format.
      - wjrl 12y ago
        Of course, YMMV, but the Stanford web network (http://www.biofabric.org/gallery/index.html#Stanford http://www.biofabric.org/gallery/index.html#Stanford) has 2.3 million edges but you can glean structure from it. You can visually estimate things like the network radius and 90-percentile effective diameter from the global view. There is a very simple R implementation that takes igraph networks as input. There is also a simple Python version from A. Mazurie at https://github.com/ajmazurie/biofabric https://github.com/ajmazurie/biofabric, though I have not tried it myself.
aw3c2 12y ago
I am currently trying to visualize the network of a moderately big forum: Users who post to threads. In Gephi I only managed to created blobby hairballs, any suggestions for other free tools? It's a couple of thousand users and a couple of thousand threads but pretty dense.
- wjrl 12y ago
  BioFabric should be able to handle that OK. Import as tab-delimited sif file. Default layout is breadth-first search, in order of decreasing degree, starting from highest degree node. If that is not what you want, can specify node order. See this post: http://biofabric.blogspot.com/2013/05/banish-bipartite-blues.html http://biofabric.blogspot.com/2013/05/banish-bipartite-blues...
_prometheus 12y ago
Dear Author of BioFabric: thank you. thank you for being willing to experiment with graph layouts when so many others just fail too soon. Great work, and keep going! It's vastly applicable work. visualizing graphs (and dags specifically) shows up all over the place)
- wjrl 12y ago
  You're welcome, and thanks! Anybody who wants to can play with different layouts, which are just linear orderings of nodes and edges, which you specify using node and edge attribute files. Features like that are discussed in the blog posts at http://biofabric.blogspot.com/ http://biofabric.blogspot.com/ Unfortunately, the blog remains the best documentation at the moment for various features...
JunkDNA 12y ago
I've made my fair share of hairball networks over the years. but I don't think this fixes the issue. The problem is that biological network diagrams of sufficient detail to be accurate, are far too complicated to be useful. Plus, biologists are often sloppy with their data model for these things: mixing genes and proteins for example. In those cases, what does a line between two nodes even mean?
- wjrl 12y ago
  If you have a complicated system, the best way to deal with the complexity is to organize it in some fashion. BioFabric provides a way to do that, since it allows the network to be arranged in a linear fashion. Kinda like the advantages of organized adjacency matrix views, but with the intuitive link-hopping model of node-link diagrams. I can't argue that if you put garbage data in, you get garbage out. But, over time, very large but high-quality biological networks will emerge.
deleted 12y ago
[deleted]
deleted 12y ago
[deleted]
benjaminjackman 12y ago
If this was a bi directional graph you would need to keep the implied 2d grid column header row straight rather than dropping down the column headers to skip already listed people. Also all those lines are noisy why not just have a 2d grid drop the headers for locality as was done then just place colored squares at the intersection points. Hmm might be worth prototyping something in d3 later.
- wjrl 12y ago
  Not sure what you mean in first sentence? I have found the node lines to be useful for navigation, and the edge lines (drawn darker and in front) create "edge wedges" that make nodes with similar neighborhoods stand out. Without lines, you end up with just an incidence (as opposed to an adjacency) matrix.

74 points by samuell 12y ago

This is among the cooler (as in more useful) visualizations I've seen in a long time.

Btw, if you want to play with some own data, you can run it with Java web start on [1]. As something simple to try out, I figured that I could create a file 'test.sif' (in SIF file format [2]) like this, just to get some example: item1 conn item2 item1 conn item3 item2 conn item1 item2 conn item3 item3 conn item1 item3 conn item2 ... (yea, you get the idea) And then visualize it by going: "File > Import > Import from SIF" Turns out there is more "demo SIF files" in [3] though. Hey, this is kinda fun :) [1] http://www.biofabric.org/#launch http://www.biofabric.org/#launch [2] "Simple Interaction Format" http://wiki.cytoscape.org/Cytoscape_User_Manual/Network_Formats http://wiki.cytoscape.org/Cytoscape_User_Manual/Network_Form... [3] http://www.biofabric.org/#sif http://www.biofabric.org/#sif , http://www.biofabric.org/sifFiles.html http://www.biofabric.org/sifFiles.html

irickt 12y ago

Compare this earlier genealogy visualization: http://www.aviz.fr/geneaquilts/ http://www.aviz.fr/geneaquilts/

astanway 12y ago

These seems pretty cool. My worry is that you lose intuitions of centrality with this, with the exception of degree centrality, since nodes with the highest degree are sorted from left to right. How can we capture other measures of centrality? What about weak ties?

wjrl 12y ago
Sorry, newbie at commenting, so my replies are actually in separate comments below. Summary: you can create a layout that orders nodes, and edges, to fit what you are trying to look at.

wjrl 12y ago

BioFabric layouts are just a linear ordering of the nodes and edges, and you can specify those with input files. For example, you can break out clusters and show intra- and inter- cluster links separately. See http://www.biofabric.org/gallery/index.html#LesMiz http://www.biofabric.org/gallery/index.html#LesMiz

wjrl 12y ago

Note that since the nodes-as-lines approach gives you a new degree of freedom to work with, you can organize edges into semantically useful groups by placing similar links next to each other. E.g. think about link communities instead of node communities.

untilHellbanned 12y ago

As a systems biologist and web developer, I say major thumbs up for the presentation of the idea, but a thumbs sideways for the actual visualization method. I'm not sure this makes biology clearer. Seems like rearranging deck chairs.

Fomite 12y ago
This was my thought as well - I work with large contact networks for epidemiology, and while getting away from hairballs would be good in theory, I'm not seeing how a figure built using this technique will make me go "Oh! Yes, now I see it!"
- wjrl 12y ago
  Placing infected individuals in the top rows, ranking uninfected individuals by probability of infection, lay out the network. Overall shape of network might give clues to network structure? I find that with contact networks, dynamic networks (links appear and disappear) tend to have problems with force-directed layouts bouncing around a lot.
  - Fomite 12y ago
    It's possible. My issue is I'm not sold that in networks of any size (the Le Mis network is pretty small) that this doesn't end up in a similarly difficult to interpret mess. I may try it out, but the initial demo didn't trigger much beyond "Neat". But to be honest, I don't tend to work in .sif format networks, and I don't have time tonight to try to get something I actually use in that format.
    - wjrl 12y ago
      Of course, YMMV, but the Stanford web network (http://www.biofabric.org/gallery/index.html#Stanford http://www.biofabric.org/gallery/index.html#Stanford) has 2.3 million edges but you can glean structure from it. You can visually estimate things like the network radius and 90-percentile effective diameter from the global view. There is a very simple R implementation that takes igraph networks as input. There is also a simple Python version from A. Mazurie at https://github.com/ajmazurie/biofabric https://github.com/ajmazurie/biofabric, though I have not tried it myself.

aw3c2 12y ago

I am currently trying to visualize the network of a moderately big forum: Users who post to threads. In Gephi I only managed to created blobby hairballs, any suggestions for other free tools? It's a couple of thousand users and a couple of thousand threads but pretty dense.

wjrl 12y ago
BioFabric should be able to handle that OK. Import as tab-delimited sif file. Default layout is breadth-first search, in order of decreasing degree, starting from highest degree node. If that is not what you want, can specify node order. See this post: http://biofabric.blogspot.com/2013/05/banish-bipartite-blues.html http://biofabric.blogspot.com/2013/05/banish-bipartite-blues...

_prometheus 12y ago

Dear Author of BioFabric: thank you. thank you for being willing to experiment with graph layouts when so many others just fail too soon. Great work, and keep going! It's vastly applicable work. visualizing graphs (and dags specifically) shows up all over the place)

wjrl 12y ago
You're welcome, and thanks! Anybody who wants to can play with different layouts, which are just linear orderings of nodes and edges, which you specify using node and edge attribute files. Features like that are discussed in the blog posts at http://biofabric.blogspot.com/ http://biofabric.blogspot.com/ Unfortunately, the blog remains the best documentation at the moment for various features...

JunkDNA 12y ago

I've made my fair share of hairball networks over the years. but I don't think this fixes the issue. The problem is that biological network diagrams of sufficient detail to be accurate, are far too complicated to be useful. Plus, biologists are often sloppy with their data model for these things: mixing genes and proteins for example. In those cases, what does a line between two nodes even mean?

wjrl 12y ago
If you have a complicated system, the best way to deal with the complexity is to organize it in some fashion. BioFabric provides a way to do that, since it allows the network to be arranged in a linear fashion. Kinda like the advantages of organized adjacency matrix views, but with the intuitive link-hopping model of node-link diagrams. I can't argue that if you put garbage data in, you get garbage out. But, over time, very large but high-quality biological networks will emerge.

deleted 12y ago

[deleted]

deleted 12y ago

[deleted]

benjaminjackman 12y ago

If this was a bi directional graph you would need to keep the implied 2d grid column header row straight rather than dropping down the column headers to skip already listed people. Also all those lines are noisy why not just have a 2d grid drop the headers for locality as was done then just place colored squares at the intersection points. Hmm might be worth prototyping something in d3 later.

wjrl 12y ago
Not sure what you mean in first sentence? I have found the node lines to be useful for navigation, and the edge lines (drawn darker and in front) create "edge wedges" that make nodes with similar neighborhoods stand out. Without lines, you end up with just an incidence (as opposed to an adjacency) matrix.