So believe it or not, a lot of genomic research uses text-based formats. When I first heard that my jaw dropped to the floor - we're talking about an alphabet w

So believe it or not, a lot of genomic research uses text-based formats. When I first heard that my jaw dropped to the floor - we're talking about an alphabet with four letters, and you're using ASCII? Really? Similarly, for gene expression, CSV files were pretty common, probably because it's the most "neutral" format for tabular data. Obviously, this doesn't scale if you are going to measure so much data that your table consists of tens of thousands of genes by hundreds of thousands of cells. So my boss, professor Sten Linnarsson decided that his group needed a more efficient file format. And ideally, it would become adopted by other labs, so that everyone could easily exchange information. The result is the .loom file format[0]. It's open source, BSD licensed, and it comes with a SciPy support library out of the box. It is HDF5-based[1], which has the benefit of being an old, battle-tested system, and platform-independent. Another problem to address is that a lot of gene data is out there, but when it comes to asking simple questions you need to do a lot of work. Researchers have to "anticipate" what kind of questions people might ask about their data and put in the effort of creating a website for it. For example, for one of the groups recent papers, they created a simple website that lets you explore the gene expression of a single gene[2]. It turned out to be very popular. So he wanted a more generic viewer for the Loom format, that lets you quickly explore the metadata en gene data and ask simple questions (for complicated stuff you will always need to download the whole dataset). That is what I have been building. The Loom Viewer[0] is an SPA lets you explore Loom files without having to download the whole data set. If, for example, you are interested in a few dozen genes, but the data set is a hundred thousands cells and tens of thousands of genes in size, it is both wasteful and time-consuming to have to download all of that. You can run it off-line with Loom files you created yourself, and you can use it to serve loom files to others. The app is still very rough around the edges, but feel free to have a look. It's a really fun project, my first dive into web-dev, which was a bit frustrating at times because I basically had to teach myself everything without any guidance or help. But I learned all kinds of cool things: the app makes use of the canvas and typed arrays for relatively fast rendering[3] and minimal bandwidth overhead (most comparable viewers download image data from the server), has off-line storage for fast gene retrieval, stores the state regarding the "view settings" in the URL so you can inherently share links, etc. The great thing about working for a professor is that he lets you indulge in building cool stuff for them. [0] https://github.com/linnarsson-lab/loompy https://github.com/linnarsson-lab/loompy [1] https://en.wikipedia.org/wiki/Hierarchical_Data_Format https://en.wikipedia.org/wiki/Hierarchical_Data_Format [2] http://linnarssonlab.org/cortex/ http://linnarssonlab.org/cortex/ [2] https://github.com/linnarsson-lab/loom-viewer https://github.com/linnarsson-lab/loom-viewer [3] In case you're wondering: "why not WebGL?" Well, because my target audience includes old professors who print out websites. I actually need to render 100 separate canvases; faking it with one WebGL overlay like regl does here: http://idyll-lang.org/idyll-regl-component/ http://idyll-lang.org/idyll-regl-component/ would not survive the print command.