4 ms·

Some of the workarounds he mentions at the end of his Trello in 4096 bytes[1] post seem really interesting: - I optimized for compression by doing things the s

by jc4p 13y ago

Some of the workarounds he mentions at the end of his Trello in 4096 bytes[1] post seem really interesting: - I optimized for compression by doing things the same way everywhere; e.g. I always put the class attribute first in my tags - I wrote a utility that tried rearranging my CSS, in an attempt to find the ordering that was the most compressible [1] http://danlec.com/blog/trello-in-4096-bytes http://danlec.com/blog/trello-in-4096-bytes

baddox 13y ago
> - I optimized for compression by doing things the same way everywhere; e.g. I always put the class attribute first in my tags Compression algorithms can do a better job when they're domain-aware. An HTML-aware algorithm could compress HTML much better than a general-use plain-text compression algorithm, without requiring the user to do things like put the class attribute first. Of course, that also requires the decompression algorithm to be similarly aware, which can be a problem if you're distributing the compressed bits widely.
- MasterScrat 13y ago
  > that also requires the decompression algorithm to be similarly aware, which can be a problem if you're distributing the compressed bits widely. Well not necessarily... An HTML-aware algorithm could for example rearrange attributes in the same order everywhere because it knows it doesn't matter. Actually that would be a nice addition to the HTML "compressors" out there.
  - baddox 13y ago
    That's a good point. You could have an HTML-aware "precompressor" prepare the HTML for a general-use compression algorithm. However, with end-to-end HTML awareness I think you could do even better.
    - MasterScrat 13y ago
      Actually that's what Google has done with Courgette: http://www.chromium.org/developers/design-documents/software-updates-courgette http://www.chromium.org/developers/design-documents/software... > Courgette transforms the input into an alternate form where binary diffing is more effective, does the differential compression in the transformed space, and inverts the transform to get the patched output in the original format. With careful choice of the alternate format we can get substantially smaller updates.
  - speleding 13y ago
    I actually do just that with a pre-processor on my site, it's only a single line of ruby with a regex, a split, a sort and a join. The reason for doing it wasn't so much the compression benefit but some of the nanoc code that generates the site did not always order the tags the same way and then it had to rsync up more than it needed to
- lucian1900 13y ago
  What that means in practice is a binary encoding, which would be really nice. At least HTTP is getting a sane binary encoding, hopefully more protocols/formats will follow.
julian37 13y ago
On a related note, Google Closure Compiler has some optimizations that increase code size before compression in order to achieve gains after compression. http://code.google.com/p/closure-compiler/wiki/FAQ#Closure_Compiler_inlined_all_my_strings,_which_made_my_code_size http://code.google.com/p/closure-compiler/wiki/FAQ#Closure_C...
nulagrithom 13y ago
After browsing St4koverflow for a while, the amount of time it took to load the Trello auth screen was jarring. I could get used to that kind of speed.
bane 13y ago
It sounds like the goal was to reduce the entropy of the code as much as possible, to within whatever the windows of the algorithm were set to. I've seen similar ideas in the demoscene 4k competition world, where code and music is arranged to have as many repeating self-similar patterns as possible so the executable compressors can shrink them optimally.
- pointernil 13y ago
  ...and reordering vertex data in chunks of only x-coords followed by chunks of y-coords followed by z-coords for the same reasons. "Farbrausch" and their series of Fr-X "small" demos would be one example of this kind of "entropy trickery" http://www.pouet.net/groups.php?which=322 http://www.pouet.net/groups.php?which=322 The next target for crunching would be to minimize the actual amount of code given to the browser to execute, versus maximizing the compression ratio only (which is "just" correlated to the running "code" size)

by jc4p 13y ago

baddox 13y ago

> - I optimized for compression by doing things the same way everywhere; e.g. I always put the class attribute first in my tags Compression algorithms can do a better job when they're domain-aware. An HTML-aware algorithm could compress HTML much better than a general-use plain-text compression algorithm, without requiring the user to do things like put the class attribute first. Of course, that also requires the decompression algorithm to be similarly aware, which can be a problem if you're distributing the compressed bits widely.

MasterScrat 13y ago
> that also requires the decompression algorithm to be similarly aware, which can be a problem if you're distributing the compressed bits widely. Well not necessarily... An HTML-aware algorithm could for example rearrange attributes in the same order everywhere because it knows it doesn't matter. Actually that would be a nice addition to the HTML "compressors" out there.
- baddox 13y ago
  That's a good point. You could have an HTML-aware "precompressor" prepare the HTML for a general-use compression algorithm. However, with end-to-end HTML awareness I think you could do even better.
  - MasterScrat 13y ago
    Actually that's what Google has done with Courgette: http://www.chromium.org/developers/design-documents/software-updates-courgette http://www.chromium.org/developers/design-documents/software... > Courgette transforms the input into an alternate form where binary diffing is more effective, does the differential compression in the transformed space, and inverts the transform to get the patched output in the original format. With careful choice of the alternate format we can get substantially smaller updates.
- speleding 13y ago
  I actually do just that with a pre-processor on my site, it's only a single line of ruby with a regex, a split, a sort and a join. The reason for doing it wasn't so much the compression benefit but some of the nanoc code that generates the site did not always order the tags the same way and then it had to rsync up more than it needed to
lucian1900 13y ago
What that means in practice is a binary encoding, which would be really nice. At least HTTP is getting a sane binary encoding, hopefully more protocols/formats will follow.

julian37 13y ago

On a related note, Google Closure Compiler has some optimizations that increase code size before compression in order to achieve gains after compression. http://code.google.com/p/closure-compiler/wiki/FAQ#Closure_Compiler_inlined_all_my_strings,_which_made_my_code_size http://code.google.com/p/closure-compiler/wiki/FAQ#Closure_C...

nulagrithom 13y ago

After browsing St4koverflow for a while, the amount of time it took to load the Trello auth screen was jarring. I could get used to that kind of speed.

bane 13y ago

It sounds like the goal was to reduce the entropy of the code as much as possible, to within whatever the windows of the algorithm were set to. I've seen similar ideas in the demoscene 4k competition world, where code and music is arranged to have as many repeating self-similar patterns as possible so the executable compressors can shrink them optimally.

pointernil 13y ago
...and reordering vertex data in chunks of only x-coords followed by chunks of y-coords followed by z-coords for the same reasons. "Farbrausch" and their series of Fr-X "small" demos would be one example of this kind of "entropy trickery" http://www.pouet.net/groups.php?which=322 http://www.pouet.net/groups.php?which=322 The next target for crunching would be to minimize the actual amount of code given to the browser to execute, versus maximizing the compression ratio only (which is "just" correlated to the running "code" size)