4 ms·

Forcing Gzip Compression

47 points by gthank 16y ago

pjscott 16y ago
I'm curious what proxies or antivirus software would mangle HTTP headers to break gzip compression. I want to avoid buying them, or anything made by the companies responsible for them, ever. Does anybody know?
- ars 16y ago
  I wonder if they themself can't handle gzip, and that's why they remove the header. Although if true, that's a pretty sneaky way of bypassing the virus scanner. Caching proxy servers probably disable it since they need to be able to server uncompressed pages sometimes (and presumably don't want to deal with compressing/uncompressing).
- dabeeeenster 16y ago
  Proxies and antivirus software are the scourge of the production website. The absolute scourge. I cant count the number of times one of our clients has complained that something has broken, accused us of being useless because we cant fix it, and we subsequently discover they are running some ancient stinky version of MS proxy on a Windows 98 box somewhere. Happens ALL THE TIME. Ignoring cache headers as well - that's a great one. Tell the browser never to remember the page, but it goes ahead and remembers it for the next two weeks. Marvellous. Yep, I hate that shit more than IE6. At least you know where you are with IE6.
  - henrikschroder 16y ago
    There was a time, before AJAX, when Norton Antivirus thought it was a great idea to protect users by rewriting javascript on the pages according to god knows what rules. I was working on a website which used a pretty complex javascript client, and had users which were... on the low end of the computer-saviness scale. The only advice that worked for these users was "disable Norton AV completely". Good times. :-)
- pilif 16y ago
  Between a windows-client and a web server, we are using bzip2 compression to squeeze off some more bytes than by just using standard gzip (the transferred data can be huge depending on the time of the month). The client correctly sends "Accept-Encoding: x-ps-bzip2" and the server responds with "Content-Encoding: x-ps-bzip2". GData AntiVirus had a bug once in which it didn't strip out the Accept-Encoding header, tried to uncompress the servers answer (but failed) and then returned the data to the client still compressed, but with the header stripped off The client assumed uncompressed data and tried to handle the answer that way - needless to say that it had mild difficulties at parsing the compressed XML as if it were uncompressed. Of course the customer blamed us "GData is perfect - an uncle of an aunt of a friend of mine works there and it's perfect. Fix your crappy software". This was the last straw of broken AV software altering our data. We stopped using Accept-Encoding or any other additional request header (those are sometimes stripped too) and we use the User-Agent header of the client and the Server-Header of the server to signal compression. Sadly, but true, this works so far. I hate, hate, hate AV programs doing stuff like this.
  - IgorPartola 16y ago
    AV software is a very bad idea. It's like having a blacklist of spammers' e-mails: you just can't blacklist them all. Combined with the fact that the whole AV industry is built on the customers' fear of "hackers", I find the whole thing quite gross.
- amartone 16y ago
  Tony Gentilcore's presentation at Velocity 2009 identified some proxies and antivirus software packages that do this: http://en.oreilly.com/velocity2009/public/schedule/detail/9072 http://en.oreilly.com/velocity2009/public/schedule/detail/90...
lennysan 16y ago
First @souders gets to the front page of TechMeme, now he's on the front page of Hacker News. He's on a roll! BTW, if you are interested in web performance, we could use your vote for this StackExchange community proposal: http://area51.stackexchange.com/proposals/6960/web-performance http://area51.stackexchange.com/proposals/6960/web-performan...
nopal 16y ago
Does anyone know how to get off-the-shelf servers like Apache or nginx to serve gzipped content based on a cookie and hot the Accept-Encoding header?
- pquerna 16y ago
  This should work inside Apache 2: SetEnvIf Cookie (.*)force_gzip(.*) force-gzip Any request with the "force-gzip" variable set in the request env will ignore the Accept headers and send deflated content to the browser.

47 points by gthank 16y ago

I'm curious what proxies or antivirus software would mangle HTTP headers to break gzip compression. I want to avoid buying them, or anything made by the companies responsible for them, ever. Does anybody know?

ars 16y ago
I wonder if they themself can't handle gzip, and that's why they remove the header. Although if true, that's a pretty sneaky way of bypassing the virus scanner. Caching proxy servers probably disable it since they need to be able to server uncompressed pages sometimes (and presumably don't want to deal with compressing/uncompressing).
dabeeeenster 16y ago
Proxies and antivirus software are the scourge of the production website. The absolute scourge. I cant count the number of times one of our clients has complained that something has broken, accused us of being useless because we cant fix it, and we subsequently discover they are running some ancient stinky version of MS proxy on a Windows 98 box somewhere. Happens ALL THE TIME. Ignoring cache headers as well - that's a great one. Tell the browser never to remember the page, but it goes ahead and remembers it for the next two weeks. Marvellous. Yep, I hate that shit more than IE6. At least you know where you are with IE6.
- henrikschroder 16y ago
  There was a time, before AJAX, when Norton Antivirus thought it was a great idea to protect users by rewriting javascript on the pages according to god knows what rules. I was working on a website which used a pretty complex javascript client, and had users which were... on the low end of the computer-saviness scale. The only advice that worked for these users was "disable Norton AV completely". Good times. :-)
pilif 16y ago
Between a windows-client and a web server, we are using bzip2 compression to squeeze off some more bytes than by just using standard gzip (the transferred data can be huge depending on the time of the month). The client correctly sends "Accept-Encoding: x-ps-bzip2" and the server responds with "Content-Encoding: x-ps-bzip2". GData AntiVirus had a bug once in which it didn't strip out the Accept-Encoding header, tried to uncompress the servers answer (but failed) and then returned the data to the client still compressed, but with the header stripped off The client assumed uncompressed data and tried to handle the answer that way - needless to say that it had mild difficulties at parsing the compressed XML as if it were uncompressed. Of course the customer blamed us "GData is perfect - an uncle of an aunt of a friend of mine works there and it's perfect. Fix your crappy software". This was the last straw of broken AV software altering our data. We stopped using Accept-Encoding or any other additional request header (those are sometimes stripped too) and we use the User-Agent header of the client and the Server-Header of the server to signal compression. Sadly, but true, this works so far. I hate, hate, hate AV programs doing stuff like this.
- IgorPartola 16y ago
  AV software is a very bad idea. It's like having a blacklist of spammers' e-mails: you just can't blacklist them all. Combined with the fact that the whole AV industry is built on the customers' fear of "hackers", I find the whole thing quite gross.
amartone 16y ago
Tony Gentilcore's presentation at Velocity 2009 identified some proxies and antivirus software packages that do this: http://en.oreilly.com/velocity2009/public/schedule/detail/9072 http://en.oreilly.com/velocity2009/public/schedule/detail/90...

lennysan 16y ago

First @souders gets to the front page of TechMeme, now he's on the front page of Hacker News. He's on a roll! BTW, if you are interested in web performance, we could use your vote for this StackExchange community proposal: http://area51.stackexchange.com/proposals/6960/web-performance http://area51.stackexchange.com/proposals/6960/web-performan...

nopal 16y ago

Does anyone know how to get off-the-shelf servers like Apache or nginx to serve gzipped content based on a cookie and hot the Accept-Encoding header?

pquerna 16y ago
This should work inside Apache 2: SetEnvIf Cookie (.*)force_gzip(.*) force-gzip Any request with the "force-gzip" variable set in the request env will ignore the Accept headers and send deflated content to the browser.