8 ms·

Google Releases mod_pagespeed for Apache

290 points by mike-cardwell 16y ago

mike-cardwell 16y ago
There are some very interesting filters: http://code.google.com/speed/page-speed/docs/filters.html http://code.google.com/speed/page-speed/docs/filters.html It will automatically strip inline css and js out of a html page and then create links to an external version. It will automatically set very long cache times. It minifies css and javascript on the fly, strips whitespace and comments from the html. Then for images, it automatically rescales them, strips out meta data and recompresses. It even converts img tags to use data: uris when it's safe and efficient to do so. I love it.
jellicle 16y ago
That's pretty cool, though the biggest slowdown for most sites are all of the third-party scripts being loaded from remote servers. If people really realized how much page speed matters in usability, they would be loathe to add in all the scripts from facebook and digg and google and 12 different analytics services.
- user24 16y ago
  addthis is a particularly guilty culprit. pageSpeed is always giving me hassle about their CSS. grumpywebmaster. You listening addthis?
- al_james 16y ago
  Ad networks are a particular culprit. For many (banner) ad networks, HTTP requests get bounced around so many servers that it can really slow things down.
  - SanjayUttam 16y ago
    Might be obvious, but this is why you should use some asynchronous code to load your ads.
    - tropin 16y ago
      Could you please give us some links or info on loading the ads asynchronously? Is it compatible with Adsense policy?
      - SanjayUttam 16y ago
        Here are some links. I don't have a rock solid answer regarding Google's TOS for this, my apologies. Async Adsense: http://webmasters.stackexchange.com/questions/2506/is-it-possible-to-use-asynchronous-google-adsense-i-e-non-blocking http://webmasters.stackexchange.com/questions/2506/is-it-pos... Async Analytics: http://www.ditii.com/2009/12/01/google-analytics-asynchronous-tracking-code-snippet-launched/ http://www.ditii.com/2009/12/01/google-analytics-asynchronou...
        tropin 16y ago
        From the first link: "this practice might not be welcome by Google". I don't know what's the second link for, as analytics are asynchronous per se? EDIT: By the date on the second url, very probably is a hack for an older version of Analytics. EDIT 2: First link is an unaccepted answer on an stackoverflow-like site on the topic for using an against the rules of Google hack. And Google doesn't need much help for closing your account and your money.
        SanjayUttam 16y ago
        Re: Analytics; I'm not sure if the default code google gives you for analytics is async _now_ but based on this http://code.google.com/apis/analytics/docs/tracking/asyncTracking.html http://code.google.com/apis/analytics/docs/tracking/asyncTra... it certainly seems like it, at least at one point, it was not. Again, that may have changed. Here is another link for async adsense with code included. The author has also posted his e-mail conversation with a rep. from the Google Adsense team. There is nothing fancy going on here, just iframes, really. http://geoland.org/2007/01/adsense-in-iframe/ http://geoland.org/2007/01/adsense-in-iframe/ You could just reach out to adsense yourself and report back what you find regarding violation of the Google Adsense TOS.
- catshirt 16y ago
  You're right, but seems irrelevant. mod_pagespeed likely has a very different audience.
- tropin 16y ago
  My adsense scripts almost always freeze my webs waiting for them to load. Do you fellow HNers know any way to speed up them or at least load them in the background?
  - metachris 16y ago
    You could inject the analytic script after the page has loaded instead of having it part of it.
    - tropin 16y ago
      I'm talking about ads, not analytics.
      - aiurtourist 16y ago
        It's been a while, but I think you can choose between IFRAME and DOM-injection methods. Researching that might get you started.
  - asolove 16y ago
    Put script tags at the end of the body, not in the header. If you still have a problem, dynamically add the script tags after the page has already loaded and displayed.
    - tropin 16y ago
      ??? Could you share any sample of adsense with scripts in the header? Also, do you know any way to load dynamically the scripts? Google seems to be full of people trying to do it to no avail.
      - bentlegen 16y ago
        Google's asynchronous tracking snippet: http://code.google.com/apis/analytics/docs/tracking/asyncTracking.html http://code.google.com/apis/analytics/docs/tracking/asyncTra...
jamesbressi 16y ago
Ask HN: I'm sure there is going to be tons of positives to this from the community, many I can identify myself, but can someone list some NEGATIVEs that may arise by using this?
- mike-cardwell 16y ago
  If you click on the individual filters to read their descriptions, they all have a "Risks" section at the bottom of the page. For example, the inline css removal filter has this: The 'Inline CSS' filter is low to moderate risk. It should be safe for most pages, but it could potentially break scripts that walk the DOM looking for and examining <link> or <style> tags.
- pornel 16y ago
  Each of the filters mentions risks: http://code.google.com/speed/page-speed/docs/filters.html http://code.google.com/speed/page-speed/docs/filters.html This is a tool that mindlessly applies optimisations that are lossy or change behavior of the page. Sometimes it may break things (e.g. I use `input[type=text]` selector in my CSS, and removal of "redundant" `type="text"` attribute from source would break it). It also wastes some processing time on applying on-the-fly code changes you could do yourself in the source. If you follow performance best practices, you may optimize pages better yourself (you need to judge what is better to inline, what can be made async. Simple heuristics of this module uses may be too crude).
  - mike-cardwell 16y ago
    Would it? I thought the type defaulted to "text" if missing. I'd be surprised if css selectors didn't recognise that...
    - pornel 16y ago
      CSS sees DOM attributes (what you get from element.getAttribute('type')), and not DOM properties (element.type). These are different things kept roughly in sync by setters/getters (called "reflected properties" in HTML spec). The `type` attribute is also supposed to have default value implied by the DTD, but that could only work in "validating" SGML/XML parsers (which browsers aren't).
qeorge 16y ago
Regarding 'Outlining CSS', there's also a risk of breaking sites that depend on the specificity rules of CSS[1]. I think this should be added to the risks list for this filter[2]. Does anyone know how to ping Google about such things? [1] http://www.molly.com/2005/10/06/css2-and-css21-specificity-clarified/ http://www.molly.com/2005/10/06/css2-and-css21-specificity-c... [2] http://code.google.com/speed/page-speed/docs/filter-css-outline.html http://code.google.com/speed/page-speed/docs/filter-css-outl...
- paulirish 16y ago
  http://groups.google.com/group/mod-pagespeed-discuss http://groups.google.com/group/mod-pagespeed-discuss or http://code.google.com/p/modpagespeed/issues/list http://code.google.com/p/modpagespeed/issues/list
  - qeorge 16y ago
    Great, thank you! Reported here: http://groups.google.com/group/mod-pagespeed-discuss/t/6bdf27584dced4f1 http://groups.google.com/group/mod-pagespeed-discuss/t/6bdf2...
hippich 16y ago
I would use this to quickly test any possible enhancements could be done to site in order to improve performance, but later - turn it off and implement everything in code.
ovi256 16y ago
Google has been accused by free software luminaries that, while it respects the letter of the GPL, it does not respect the spirit of the license, as it does not share its optimized versions of the Linux kernel, Apache server, and many other GPL software. Of course, as it's GPL, not AGPL, they are not legally in the wrong. This nice piece of infrastructure is a step forward, let's hope they release more of their custom infrastructure.
- wmf 16y ago
  Does Google use this internally? I got the impression that it was written specifically for external use.
  - stanleydrew 16y ago
    I would think they almost certainly don't use this internally given that they don't run Apache for any of their user-facing applications.
mootothemax 16y ago
Somewhat ironically, one of the biggest slowdowns on my websites has been using Google Analytics. I only realised this after I switched completely to my own installation of Piwik - give it a try, the load times can be quite surprising.
- trickjarrett 16y ago
  Were you using the asyncronous Analytics code?
  - mootothemax 16y ago
    Nah, and to be fair, this was a while ago - when the async stuff had barely been released in beta. I hope the situation's improved somewhat now because of it :)
    - trickjarrett 16y ago
      The async code makes a world of difference, it definitely improved things for me and my sites.
- deleted 16y ago
  [deleted]
- hippich 16y ago
  As another option - you can cache GA js code locally and combine it with the rest site's js code. That's what Google Analytics module for Drupal is doing. Result - no third-party requests at all.
dchest 16y ago
Does it cache optimized images or regenerates them for every request? Edit: Yes, it does: In order to rewrite resources, mod_pagespeed must cache them on the server.
Yrlec 16y ago
This looks very promising! Microsoft has similar features (although not officially supported yet) for ASP.NET: http://aspnet.codeplex.com/releases/view/50869 http://aspnet.codeplex.com/releases/view/50869 http://aspnet.codeplex.com/releases/view/35893 http://aspnet.codeplex.com/releases/view/35893
bwooceli 16y ago
All great ideas. I'm in a bit of a knowledge-base limbo where I spend just enough time setting up my server so that it survives a load test. I use nginx as my daily driver; I think I'll try to level up some admin skill and try to apply some of these principles to my setup. Who knows, maybe I'll do a write up.
robk 16y ago
It's sad/embarrassing they don't have a Chrome plugin for this but do offer Firefox.
- pjscott 16y ago
  Firefox has a more powerful plugin system than Chrome, and I would bet that most web developers have Firefox installed, even if only for testing and for Firebug.
- blasdel 16y ago
  Screw that, they've got their priorities straight. It's awesome that they don't treat 'uncaptured synergy' as a release blocker like Microsoft would have.
Goosey 16y ago
This made me realize just how strongly Google is tied to the success of 'the internet' vs the traditional desktop. There isn't much in their strategy that doesn't make sense if you stop trying to figure out how it ties into seach/advertising and if you see how it just enhances the internet as a whole. edit: of course enhancing the internet increases the number and frequency of people interacting with their search, adwords on pages, and feeling comfortable with web apps (helping create lead for Google Apps B2B)
gawker 16y ago
Would love something like this for nginx - Apache's just using too much memory.
- necro 16y ago
  Most of these are micro optimizations that matter most on a larger site. Most large site do not use Apache for static content. I agree, nginx would be a better first choice for a module. One thing that may not be obvious about minimization is that it does not yield much size reduction when coupled with gzip. You may see 30% decrease in size with minimization on the plain file, but when you gzip both the plain and minimized you may only see a difference for a couple of hundred bytes. If you are worried about speed, you should be using gzip for static content.
  - 56quarters 16y ago
    I've found using minimization to give up 40% smaller files even after gzip. Even so, combining multiple JS or CSS files into single one and using image sprites will probably be your biggest win.
  - gawker 16y ago
    I've seen similar results. It's nice to see it everything do it on the fly but I'm guessing doing all these optimizations on a large website will take up more processor cycles?
- easp 16y ago
  Put nginx in front of apache and allow it to buffer responses. That way apache can move on to rendering the next request while nginix empties the buffer. Each apache process still uses just as much memory, but they can get more work done, because they aren't tying up memory while they push the buffered response out.
  - gawker 16y ago
    True that. But it's just a wish from my perspective since I'm running on a VPS with limited memory so I can't really afford to have both running.
andrewingram 16y ago
I love the idea of this, but these days I always have something else sitting in front of apache (usually nginx), it seems like this is where i'd want a module like this to be running. If apache rewrites the html with such things like temporary optimized versions of files, nginx presumably won't know anything about serving them, so it'll break (date urls being an exception here I guess). For things like media, I really want to keep apache out of the loop completely.
lamnk 16y ago
Doesn't matter. I ditched Apache completely for nginx and didn't look back since then. Every apps get proxied through nginx: PHP scripts run as FastCGI (php-fpm), Rails and Sinatra apps run on Unicorn ...
CharlesPal 16y ago
Has anyone been able to get this working with a web server running cPanel?
wanderr 16y ago
Has anyone found/done any benchmarks showing what performance impact running mod_pagespeed will have on your /server/? I'm all for optimizing the crap out of page serving, but if this is going to double our Apache load, a) I'd like to know about it ahead of time and b) it might be worth just applying optimizations by hand.
apgwoz 16y ago
For those worried that it might break something on their site, a word of advice. You shouldn't expect anything to "just work," and should, like any technology that you make use of, evaluate and test it out to see if it's right for you. There's no details on it, but I wonder if GoDaddy has some plan to provide a way to turn specific things off. If not, this sounds like it could be a disaster.
ochronus 16y ago
Some concerns about this module here: http://blog.mostof.it/mod_pagespeed/ http://blog.mostof.it/mod_pagespeed/ Don't let developers be blinded by an automatic optimizer.

290 points by mike-cardwell 16y ago

There are some very interesting filters: http://code.google.com/speed/page-speed/docs/filters.html http://code.google.com/speed/page-speed/docs/filters.html It will automatically strip inline css and js out of a html page and then create links to an external version. It will automatically set very long cache times. It minifies css and javascript on the fly, strips whitespace and comments from the html. Then for images, it automatically rescales them, strips out meta data and recompresses. It even converts img tags to use data: uris when it's safe and efficient to do so. I love it.

jellicle 16y ago

That's pretty cool, though the biggest slowdown for most sites are all of the third-party scripts being loaded from remote servers. If people really realized how much page speed matters in usability, they would be loathe to add in all the scripts from facebook and digg and google and 12 different analytics services.

user24 16y ago
addthis is a particularly guilty culprit. pageSpeed is always giving me hassle about their CSS. grumpywebmaster. You listening addthis?
al_james 16y ago
Ad networks are a particular culprit. For many (banner) ad networks, HTTP requests get bounced around so many servers that it can really slow things down.
- SanjayUttam 16y ago
  Might be obvious, but this is why you should use some asynchronous code to load your ads.
  - tropin 16y ago
    Could you please give us some links or info on loading the ads asynchronously? Is it compatible with Adsense policy?
    - SanjayUttam 16y ago
      Here are some links. I don't have a rock solid answer regarding Google's TOS for this, my apologies. Async Adsense: http://webmasters.stackexchange.com/questions/2506/is-it-possible-to-use-asynchronous-google-adsense-i-e-non-blocking http://webmasters.stackexchange.com/questions/2506/is-it-pos... Async Analytics: http://www.ditii.com/2009/12/01/google-analytics-asynchronous-tracking-code-snippet-launched/ http://www.ditii.com/2009/12/01/google-analytics-asynchronou...
      - tropin 16y ago
        From the first link: "this practice might not be welcome by Google". I don't know what's the second link for, as analytics are asynchronous per se? EDIT: By the date on the second url, very probably is a hack for an older version of Analytics. EDIT 2: First link is an unaccepted answer on an stackoverflow-like site on the topic for using an against the rules of Google hack. And Google doesn't need much help for closing your account and your money.
        SanjayUttam 16y ago
        Re: Analytics; I'm not sure if the default code google gives you for analytics is async _now_ but based on this http://code.google.com/apis/analytics/docs/tracking/asyncTracking.html http://code.google.com/apis/analytics/docs/tracking/asyncTra... it certainly seems like it, at least at one point, it was not. Again, that may have changed. Here is another link for async adsense with code included. The author has also posted his e-mail conversation with a rep. from the Google Adsense team. There is nothing fancy going on here, just iframes, really. http://geoland.org/2007/01/adsense-in-iframe/ http://geoland.org/2007/01/adsense-in-iframe/ You could just reach out to adsense yourself and report back what you find regarding violation of the Google Adsense TOS.
catshirt 16y ago
You're right, but seems irrelevant. mod_pagespeed likely has a very different audience.
tropin 16y ago
My adsense scripts almost always freeze my webs waiting for them to load. Do you fellow HNers know any way to speed up them or at least load them in the background?
- metachris 16y ago
  You could inject the analytic script after the page has loaded instead of having it part of it.
  - tropin 16y ago
    I'm talking about ads, not analytics.
    - aiurtourist 16y ago
      It's been a while, but I think you can choose between IFRAME and DOM-injection methods. Researching that might get you started.
- asolove 16y ago
  Put script tags at the end of the body, not in the header. If you still have a problem, dynamically add the script tags after the page has already loaded and displayed.
  - tropin 16y ago
    ??? Could you share any sample of adsense with scripts in the header? Also, do you know any way to load dynamically the scripts? Google seems to be full of people trying to do it to no avail.
    - bentlegen 16y ago
      Google's asynchronous tracking snippet: http://code.google.com/apis/analytics/docs/tracking/asyncTracking.html http://code.google.com/apis/analytics/docs/tracking/asyncTra...

jamesbressi 16y ago

Ask HN: I'm sure there is going to be tons of positives to this from the community, many I can identify myself, but can someone list some NEGATIVEs that may arise by using this?

mike-cardwell 16y ago
If you click on the individual filters to read their descriptions, they all have a "Risks" section at the bottom of the page. For example, the inline css removal filter has this: The 'Inline CSS' filter is low to moderate risk. It should be safe for most pages, but it could potentially break scripts that walk the DOM looking for and examining <link> or <style> tags.
pornel 16y ago
Each of the filters mentions risks: http://code.google.com/speed/page-speed/docs/filters.html http://code.google.com/speed/page-speed/docs/filters.html This is a tool that mindlessly applies optimisations that are lossy or change behavior of the page. Sometimes it may break things (e.g. I use `input[type=text]` selector in my CSS, and removal of "redundant" `type="text"` attribute from source would break it). It also wastes some processing time on applying on-the-fly code changes you could do yourself in the source. If you follow performance best practices, you may optimize pages better yourself (you need to judge what is better to inline, what can be made async. Simple heuristics of this module uses may be too crude).
- mike-cardwell 16y ago
  Would it? I thought the type defaulted to "text" if missing. I'd be surprised if css selectors didn't recognise that...
  - pornel 16y ago
    CSS sees DOM attributes (what you get from element.getAttribute('type')), and not DOM properties (element.type). These are different things kept roughly in sync by setters/getters (called "reflected properties" in HTML spec). The `type` attribute is also supposed to have default value implied by the DTD, but that could only work in "validating" SGML/XML parsers (which browsers aren't).

qeorge 16y ago

Regarding 'Outlining CSS', there's also a risk of breaking sites that depend on the specificity rules of CSS[1]. I think this should be added to the risks list for this filter[2]. Does anyone know how to ping Google about such things? [1] http://www.molly.com/2005/10/06/css2-and-css21-specificity-clarified/ http://www.molly.com/2005/10/06/css2-and-css21-specificity-c... [2] http://code.google.com/speed/page-speed/docs/filter-css-outline.html http://code.google.com/speed/page-speed/docs/filter-css-outl...

paulirish 16y ago
http://groups.google.com/group/mod-pagespeed-discuss http://groups.google.com/group/mod-pagespeed-discuss or http://code.google.com/p/modpagespeed/issues/list http://code.google.com/p/modpagespeed/issues/list
- qeorge 16y ago
  Great, thank you! Reported here: http://groups.google.com/group/mod-pagespeed-discuss/t/6bdf27584dced4f1 http://groups.google.com/group/mod-pagespeed-discuss/t/6bdf2...

hippich 16y ago

I would use this to quickly test any possible enhancements could be done to site in order to improve performance, but later - turn it off and implement everything in code.

ovi256 16y ago

Google has been accused by free software luminaries that, while it respects the letter of the GPL, it does not respect the spirit of the license, as it does not share its optimized versions of the Linux kernel, Apache server, and many other GPL software. Of course, as it's GPL, not AGPL, they are not legally in the wrong. This nice piece of infrastructure is a step forward, let's hope they release more of their custom infrastructure.

wmf 16y ago
Does Google use this internally? I got the impression that it was written specifically for external use.
- stanleydrew 16y ago
  I would think they almost certainly don't use this internally given that they don't run Apache for any of their user-facing applications.

mootothemax 16y ago

Somewhat ironically, one of the biggest slowdowns on my websites has been using Google Analytics. I only realised this after I switched completely to my own installation of Piwik - give it a try, the load times can be quite surprising.

trickjarrett 16y ago
Were you using the asyncronous Analytics code?
- mootothemax 16y ago
  Nah, and to be fair, this was a while ago - when the async stuff had barely been released in beta. I hope the situation's improved somewhat now because of it :)
  - trickjarrett 16y ago
    The async code makes a world of difference, it definitely improved things for me and my sites.
deleted 16y ago
[deleted]
hippich 16y ago
As another option - you can cache GA js code locally and combine it with the rest site's js code. That's what Google Analytics module for Drupal is doing. Result - no third-party requests at all.

dchest 16y ago

Does it cache optimized images or regenerates them for every request? Edit: Yes, it does: In order to rewrite resources, mod_pagespeed must cache them on the server.

Yrlec 16y ago

This looks very promising! Microsoft has similar features (although not officially supported yet) for ASP.NET: http://aspnet.codeplex.com/releases/view/50869 http://aspnet.codeplex.com/releases/view/50869 http://aspnet.codeplex.com/releases/view/35893 http://aspnet.codeplex.com/releases/view/35893

bwooceli 16y ago

All great ideas. I'm in a bit of a knowledge-base limbo where I spend just enough time setting up my server so that it survives a load test. I use nginx as my daily driver; I think I'll try to level up some admin skill and try to apply some of these principles to my setup. Who knows, maybe I'll do a write up.

robk 16y ago

It's sad/embarrassing they don't have a Chrome plugin for this but do offer Firefox.

pjscott 16y ago
Firefox has a more powerful plugin system than Chrome, and I would bet that most web developers have Firefox installed, even if only for testing and for Firebug.
blasdel 16y ago
Screw that, they've got their priorities straight. It's awesome that they don't treat 'uncaptured synergy' as a release blocker like Microsoft would have.

Goosey 16y ago

This made me realize just how strongly Google is tied to the success of 'the internet' vs the traditional desktop. There isn't much in their strategy that doesn't make sense if you stop trying to figure out how it ties into seach/advertising and if you see how it just enhances the internet as a whole. edit: of course enhancing the internet increases the number and frequency of people interacting with their search, adwords on pages, and feeling comfortable with web apps (helping create lead for Google Apps B2B)

gawker 16y ago

Would love something like this for nginx - Apache's just using too much memory.

necro 16y ago
Most of these are micro optimizations that matter most on a larger site. Most large site do not use Apache for static content. I agree, nginx would be a better first choice for a module. One thing that may not be obvious about minimization is that it does not yield much size reduction when coupled with gzip. You may see 30% decrease in size with minimization on the plain file, but when you gzip both the plain and minimized you may only see a difference for a couple of hundred bytes. If you are worried about speed, you should be using gzip for static content.
- 56quarters 16y ago
  I've found using minimization to give up 40% smaller files even after gzip. Even so, combining multiple JS or CSS files into single one and using image sprites will probably be your biggest win.
- gawker 16y ago
  I've seen similar results. It's nice to see it everything do it on the fly but I'm guessing doing all these optimizations on a large website will take up more processor cycles?
easp 16y ago
Put nginx in front of apache and allow it to buffer responses. That way apache can move on to rendering the next request while nginix empties the buffer. Each apache process still uses just as much memory, but they can get more work done, because they aren't tying up memory while they push the buffered response out.
- gawker 16y ago
  True that. But it's just a wish from my perspective since I'm running on a VPS with limited memory so I can't really afford to have both running.

andrewingram 16y ago

I love the idea of this, but these days I always have something else sitting in front of apache (usually nginx), it seems like this is where i'd want a module like this to be running. If apache rewrites the html with such things like temporary optimized versions of files, nginx presumably won't know anything about serving them, so it'll break (date urls being an exception here I guess). For things like media, I really want to keep apache out of the loop completely.

lamnk 16y ago

Doesn't matter. I ditched Apache completely for nginx and didn't look back since then. Every apps get proxied through nginx: PHP scripts run as FastCGI (php-fpm), Rails and Sinatra apps run on Unicorn ...

CharlesPal 16y ago

Has anyone been able to get this working with a web server running cPanel?

wanderr 16y ago

Has anyone found/done any benchmarks showing what performance impact running mod_pagespeed will have on your /server/? I'm all for optimizing the crap out of page serving, but if this is going to double our Apache load, a) I'd like to know about it ahead of time and b) it might be worth just applying optimizations by hand.

apgwoz 16y ago

For those worried that it might break something on their site, a word of advice. You shouldn't expect anything to "just work," and should, like any technology that you make use of, evaluate and test it out to see if it's right for you. There's no details on it, but I wonder if GoDaddy has some plan to provide a way to turn specific things off. If not, this sounds like it could be a disaster.

ochronus 16y ago

Some concerns about this module here: http://blog.mostof.it/mod_pagespeed/ http://blog.mostof.it/mod_pagespeed/ Don't let developers be blinded by an automatic optimizer.