5 ms·
Many webservers allow you to serve a compressed file (stored on disk) and _decompress_ when a client specifically can't support the compressed encoding. Since m
by sethhochberg 3y ago
Many webservers allow you to serve a compressed file (stored on disk) and _decompress_ when a client specifically can't support the compressed encoding. Since most clients should support compression, this means you only use the CPU for the rarer case where the uncompressed data is required.
Eg, http://nginx.org/en/docs/http/ngx_http_gunzip_module.html http://nginx.org/en/docs/http/ngx_http_gunzip_module.html
- nikau 3y agoOr you can just zip it explicitly and remove complexity and future issues it they have to move to another web server platform.
- MengerSponge 3y agoBut what if an analyst needs to access this data and run their regressions on a potato? Surely that use case is worth adding a few libraries to handle.
- CJefferson 3y agoWhat machine could anyone be running an analysis on where unzip is a limiting factor?
- londonReed 3y agoWhoosh
- adbachman 3y agoyou heard them, a literal potato from the ground. even a big old russet only has, what, like 32 bites?
- pantalaimon 3y agoI mean there are some potatoes that take more then four bytes to eat, but they are rare. For some, if you slice them, you might even end up with only 16 bits.
- innocenat 3y agoI would think running regression should be more demanding on the potato then decompressing ZIP file.
- nikau 3y agohmm good point, can you draft up an architecture plan using multiple microservices and redundancy via a kubernetes cluster and have it on my desk by Monday please.