> Instead of parsing data, it might be better to store it as a blob of some sort with a binary index. This is exactly something I did for JSON, I call it semi-

> Instead of parsing data, it might be better to store it as a blob of some sort with a binary index. This is exactly something I did for JSON, I call it semi-indexing: instead of parsing it into a tree of pointers, I create a succinct representation of the parsing tree, which is orders of magnitude smaller than the original JSON. Construction is much faster than parsing because there are basically no memory allocations, and access is not that much slower. About performance of succinct data structures in general, it is true that they have shown poor practical performance, but things are changing, both because we have better CPUs (while memory latency is pretty much unchanged), and better algorithms are being found. I did my Ph.D. on practical succinct data structures. We found that in some applications, the access times are competitive or faster than the non-succinct counterparts, while the space is much smaller. One example is tries: in my thesis [2] there are experiments for string dictionaries, and for query autocompletion (for example for search engines). Another area where (quasi-)succinct data structures are having some success is inverted indexes: recently proposed posting lists based on Elias-Fano [3] have been shown to outperform standard delta-encoded posting lists for queries with sparse intersection, and are used in Facebook's graph search [4]. Finally, the biggest success story of SDS has historically been molecular biology, because the size of the DNA sequences processed is so large that non-succinct data structures are impractical. Many sequence assemblers/aligners use variants of FM-indexes and Compressed Suffix Arrays, that are self-indexes based on the Burrows-Wheeler Transform and Wavelet Trees. [1] https://github.com/ot/semi_index https://github.com/ot/semi_index [2] http://www.di.unipi.it/~ottavian/files/phd_thesis.pdf http://www.di.unipi.it/~ottavian/files/phd_thesis.pdf [3] http://vigna.di.unimi.it/ftp/papers/QuasiSuccinctIndices.pdf http://vigna.di.unimi.it/ftp/papers/QuasiSuccinctIndices.pdf [4] http://www.vldb.org/pvldb/vol6/p1150-curtiss.pdf http://www.vldb.org/pvldb/vol6/p1150-curtiss.pdf