3 ms·
Since the order of the samples doesn't matter, could you sort them somehow so the Gzipped stream of samples can be compressed better? (E.g. sort the color index
by pjtr 11y ago
Since the order of the samples doesn't matter, could you sort them somehow so the Gzipped stream of samples can be compressed better? (E.g. sort the color index by component average, by red, by maximum component, ... and sort the point index by color.)
Have you tried struct-of-array (AAA...BBB...) instead of array-of-struct (ABABAB...) layouts?
- ruarai 11y agoI tried your struct-of-array idea, and that's produced an okay improvement ~1%. Sorting them seems tough as the index of each value must match for each channel, so any sorting would have to occur beforehand. Except that is already sorted by x-y values, and my attempts otherwise have failed to produce results.
- pjtr 11y agoI'm missing where they are sorted by x-y values. I did a quick and dirty experiment: http://pastebin.com/NjZNRjw1 http://pastebin.com/NjZNRjw1 Seems about 30% smaller than before on http://i.imgur.com/5zwCEF5.png http://i.imgur.com/5zwCEF5.png
- ruarai 11y agoWow, that works pretty well. I was mistaken in thinking that either the Dictionary class or the process of sampling would sort them. Mind if I merge that? Or you could submit a pull request. Either would be great! Also, do you know of any resources for learning about how to optimise for gzip compression? Google is just telling me about compression for websites.
- pjtr 11y agoSure, feel free to merge. I don't know any resources specifically about gzip compression. Demosceners have very practical and fun compression know how, so maybe look into: http://www.farbrausch.com/~fg/seminars/workcompression.html http://www.farbrausch.com/~fg/seminars/workcompression.html
- willvarfar 11y agoCompression gurus hang out at http://encode.ru http://encode.ru You'll likely find a compressor much more suited to your particular data than gzip.