5 ms·

So I work on real estate websites that typically consume 50-80GB for images so articles such as this are really interesting to me. I am using PHP7 BTW... Curr

by cubano 7y ago

So I work on real estate websites that typically consume 50-80GB for images so articles such as this are really interesting to me. I am using PHP7 BTW... Currently, I'm converting all the jpegs to webp at 55 compression at the time of import and I love both the quality and the sizes of the files, and so far couldn't be happier with the results[0] I managed to shave some 20GB from the disk usage as well. The 800lb gorilla is, however, Safari. We all know that Safari does not currently support webp and due to internal politics, may never do so. So the first "solution" is to keep copies of both formats and use the <picture> tag. This just won't do I have zero interest in doubling my already prodigal disk usage. There are some rather complicated OS libs that will take a jpg and convert on the fly to webp...I want to do it the other way around. One thing I hate are "kitchen sink" libs that do SO much that you are often left confused on how to do the specific thing you want to do, so I am in the process of rolling my own solution by using GD and intelligently converting the 11% of all requests that are Safari and using nginx caching to keep these converted images ready to go. I would like to hear of anyone with better solutions, if possable, since as you can see, I haven't yet implemented my ideas. [0]https://lasvegasluxerealty.com https://lasvegasluxerealty.com

daveslash 7y ago
I don't have a solution, but you may find this of interest: I was snooping around on AirBnB with my network analyzer. (I like to quasi-reverse-engineer sites). I noticed that if you hit the site with Chrome/FF, you get served webp images; if you hit the site with IE, you get served JPEGs. It would be really great if there was an on-the-fly conversion library -- basically, reference the image once and let the run-time deliver up the right image type. What I don't like about that is that I generally shy away from doing any sort of "if then" based on browsers' user-agent-string.Edit/Update: I don't like that that because it got us all into a lot of trouble during the Browser Wars[0] and is now generally considered bad practice. [1] This might get me some downvotes... but hold my beer and watch this: I wonder if there is a javascript solution? Maybe you serve up the webp images, always, and write your own jquery plug-in that converts the webp to jpeg in memory and then writes it to an HTML5 canvas, but only if it's safari/IE? [0] https://youtu.be/yRkQOw1uRrw?t=164 https://youtu.be/yRkQOw1uRrw?t=164 [1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Browser_detection_using_the_user_agent https://developer.mozilla.org/en-US/docs/Web/HTTP/Browser_de...
- simcop2387 7y ago
  I've never used it but there does appear to be something like that called webp-hero https://github.com/chase-moskal/webp-hero https://github.com/chase-moskal/webp-hero
- johnday 7y ago
  > What I don't like about that is that I generally shy away from doing any sort of "if then" based on browsers' user-agent-string. Can't you just base it on the "Accepts" header? That's the whole reason it exists, after all.
  - social_quotient 7y ago
    I think this is the right answer and semantically better than parsing user agents, guessing, or other magic.
- MildlySerious 7y ago
  Wouldn't it be possible to combine the approaches? Say, using a <picture> tag where the webp is requested when supported, and if not a jpg is requested, which is generated from the webp on the backend.
lkiernan 7y ago
I haven't dealt with this volume of images but Cloudinary will auto convert and serve the correct image format
thijsvandien 7y ago
Being in a luxury market, is 20GB (or even 60GB for that matter) more disk space really considered too expensive, especially given the fact that it's going to need engineering effort? Or maybe you're talking traffic, but that isn't so costly either.
cactus2093 7y ago
It's hard to give advice without knowing more detail but I would suggest putting a CDN (like cloudfront) in front of it to reduce bandwidth costs, and potentially storing the images themselves in S3 instead of on your server to reduce storage costs. Other things like making thumbnails of all the images can also help reduce bandwidth but depends on how the site is structured.
- vidarh 7y ago
  AWS outbound bandwidth costs are insanely high compared to most alternatives, so this very rarely makes any sense unless your images are only very rarely accessed.
chrisweekly 7y ago
It's been quite a few years now, but back in the day I had success leveraging `mod_pagespeed` in Apache, to rewrite server-generated markup on the fly, and to respond to raster image requests with webp assets for browsers that supported them. For other similarly conditional responses I relied on a proper WURFL integration -- not just shallow UA-sniffing -- but IIRC mod_pagespeed handled the webp conditionality on its own. Which is to say, I think mod_pagespeed was able to generate and cache a webp version of my jpg assets, and it would simply serve them in response to requests that included the relevant value in their "Accepts" HTTP headers. If you go that route, be aware that HTTP caching mechanics can get tricky, and some CDN service providers (deliberately and self-interestedly) violate the rfc's (eg refuse to cache anything w/ a "Vary" response header at all -- looking at you Akamai)... In retrospect, it was great fun to effectively piece together a web performance optimization service layer to support and accelerate the heck out of a traditional 3-tier web application... but doing it all by hand, these days, might not have great ROI.

by cubano 7y ago

daveslash 7y ago

I don't have a solution, but you may find this of interest: I was snooping around on AirBnB with my network analyzer. (I like to quasi-reverse-engineer sites). I noticed that if you hit the site with Chrome/FF, you get served webp images; if you hit the site with IE, you get served JPEGs. It would be really great if there was an on-the-fly conversion library -- basically, reference the image once and let the run-time deliver up the right image type. What I don't like about that is that I generally shy away from doing any sort of "if then" based on browsers' user-agent-string.Edit/Update: I don't like that that because it got us all into a lot of trouble during the Browser Wars[0] and is now generally considered bad practice. [1] This might get me some downvotes... but hold my beer and watch this: I wonder if there is a javascript solution? Maybe you serve up the webp images, always, and write your own jquery plug-in that converts the webp to jpeg in memory and then writes it to an HTML5 canvas, but only if it's safari/IE? [0] https://youtu.be/yRkQOw1uRrw?t=164 https://youtu.be/yRkQOw1uRrw?t=164 [1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Browser_detection_using_the_user_agent https://developer.mozilla.org/en-US/docs/Web/HTTP/Browser_de...

simcop2387 7y ago
I've never used it but there does appear to be something like that called webp-hero https://github.com/chase-moskal/webp-hero https://github.com/chase-moskal/webp-hero
johnday 7y ago
> What I don't like about that is that I generally shy away from doing any sort of "if then" based on browsers' user-agent-string. Can't you just base it on the "Accepts" header? That's the whole reason it exists, after all.
- social_quotient 7y ago
  I think this is the right answer and semantically better than parsing user agents, guessing, or other magic.
MildlySerious 7y ago
Wouldn't it be possible to combine the approaches? Say, using a <picture> tag where the webp is requested when supported, and if not a jpg is requested, which is generated from the webp on the backend.

lkiernan 7y ago

I haven't dealt with this volume of images but Cloudinary will auto convert and serve the correct image format

thijsvandien 7y ago

Being in a luxury market, is 20GB (or even 60GB for that matter) more disk space really considered too expensive, especially given the fact that it's going to need engineering effort? Or maybe you're talking traffic, but that isn't so costly either.

cactus2093 7y ago

It's hard to give advice without knowing more detail but I would suggest putting a CDN (like cloudfront) in front of it to reduce bandwidth costs, and potentially storing the images themselves in S3 instead of on your server to reduce storage costs. Other things like making thumbnails of all the images can also help reduce bandwidth but depends on how the site is structured.

vidarh 7y ago
AWS outbound bandwidth costs are insanely high compared to most alternatives, so this very rarely makes any sense unless your images are only very rarely accessed.

chrisweekly 7y ago

It's been quite a few years now, but back in the day I had success leveraging `mod_pagespeed` in Apache, to rewrite server-generated markup on the fly, and to respond to raster image requests with webp assets for browsers that supported them. For other similarly conditional responses I relied on a proper WURFL integration -- not just shallow UA-sniffing -- but IIRC mod_pagespeed handled the webp conditionality on its own. Which is to say, I think mod_pagespeed was able to generate and cache a webp version of my jpg assets, and it would simply serve them in response to requests that included the relevant value in their "Accepts" HTTP headers. If you go that route, be aware that HTTP caching mechanics can get tricky, and some CDN service providers (deliberately and self-interestedly) violate the rfc's (eg refuse to cache anything w/ a "Vary" response header at all -- looking at you Akamai)... In retrospect, it was great fun to effectively piece together a web performance optimization service layer to support and accelerate the heck out of a traditional 3-tier web application... but doing it all by hand, these days, might not have great ROI.