8 ms·

Being able to resize images via url parameters is one of those great ideas, but I've always been surprised by how long it's taken to catch on. Awesome.

chrisfarms 13y ago
I've got a feeling it's something that many people have been doing on even mildly complex sites for a long time, but people seem to be trying to SaaS it up recently. protip: Dynamically generating content by URLs is also a nice DOS attack surface if you don't rate limit in some way or try to detect abuse.
- agschwender 13y ago
  Definitely, true. The purpose of the signature and allowed hosts functionality is there to help mitigate that risk. Also, since the image service would be run separate from the main application, it shouldn't impact the performance of your site. Likewise, in a real production environment, this would run behind a CDN or Varnish, the DOS would just prevent new images from being generated until it could be addressed.
na85 13y ago
Honest question: Why is it such a great idea? Wouldn't it be easier/quicker to either: a) Have a command-line tool to resize images? Alternatively, one that's integrated into the shell so that you can access it from context menus? I.e. right-click -> "resize image" b) Just open it in mspaint or your open-source image editor of choice?
- OhHeyItsE 13y ago
  Pretty sure the idea is to make this available as a service, for UGC-type sites.
- canterburry 13y ago
  These services are becoming more interesting as responsive web design takes off and everyone tries to format their images based on client agent needs. Often the exact dimensions and treatment of the image can't be determined before request time depending on the user agent's screen size, resolution etc. Also, why spend time as a designer cropping and scaling images when you can automate it all and make it on demand? This thread caught by eye since I have been working on the very same idea for the last year. http://www.pixtulate.com http://www.pixtulate.com
- joevandyk 13y ago
  It's simpler to store the original image, then generate URLs like http://image-server.com/resize?url=http://s3.com/original.jpg&width=300&height=200 http://image-server.com/resize?url=http://s3.com/original.jp... This lets you change the size of thumbnails by only changing the URLs. I've been doing it for years, works great.
  - brandon272 13y ago
    Doesn't this mean that the server needs to manually process the image on each request?
    - michaelmior 13y ago
      You should stick a CDN in front so this doesn't happen. Ideally once the server generates the image, it would also be cached there.
      - joevandyk 13y ago
        You probably want nginx in front of pillbox. Nginx setup as a basic http proxy cache. So, CDN -> nginx -> pillbox. This is because CDNs can send lots of concurrent requests to the origin server if lots of people are requesting the same new image simultaneously.
bjt 13y ago
PHP sites have been doing this with TimThumb for quite a while. https://code.google.com/p/timthumb/ https://code.google.com/p/timthumb/ I wrote https://bitbucket.org/btubbs/thumpy https://bitbucket.org/btubbs/thumpy at work and we've been using it in production for a couple years. It's very similar to Pilbox.
gaelenh 13y ago
I wrote a similar image service (resize, crop, border, color analysis, format, and other fun features) a few years ago. It served about 6M image requests a day from 4 servers, plus a whole lot more from the CDN. If you're using a CDN, just make sure they cache on query string parameters. Our resize controls were part of the URI, but the other commands were part of the QS. Our CDN stripped queries, so the first image would be cached and any calls with different QS would return the first hit. A possible alternative that I never implemented was using the request header instead of the query string for additional commands. Edit to add: Also, if you have lots of resized images on a page, be careful when Google or Bing scraped you. Your CPU and IO will go through the roof as your servers go crazy trying to dynamically generate all the images.
- canterburry 13y ago
  So what happened to it? Are you still running it? If not, why not?
  - gaelenh 13y ago
    Not running (probably). We were acquired by a competitor. Last I heard, they upgrade their image server code with the bare essential features to migrate over our clients. A lot of good software was buried in that acquisition.
- lnanek2 13y ago
  Kind of strange the bots would spike you. Wouldn't anything on a page have been used before and thus cached on your server as a file? I've written all this stuff before as well, including overlaying site logo on images that were hotlinked to boot, but I always cached to the file system anything I had to generate the first time it was requested and simply served it after.
  - gaelenh 13y ago
    Short version: We had direct access to all the large news image providers and analyzed the images for topic and story identification. We used this data to dynamically generate hundreds of thousands of pages. Way too much data to cache to file (our storage cluster was many many terabytes). We used a CDN to cache the images. When a bot would scrape, it would hit all the old pages that fell out of the cache. Perhaps our fault for having such a large sitemap.xml.
Terretta 13y ago
> Being able to resize images via url parameters is one of those great ideas, but I've always been surprised by how long it's taken to catch on. Our video CDN customers can use a custom URL to get a thumbnail of any size from any key frame in their video files, fantastic for building "trick play" responsive HTML5 video player UIs without having to manage a single image. Just one of the countless little nice-to-haves that sets a VDN apart from a CDN.
- samstave 13y ago
  Who are you using as a VDN?
  - ihsw 13y ago
    His message indicated that he works for a VDN provider, rather than be a user of one. This would seem to be his employer: http://www.advection.net/ http://www.advection.net/

chrisfarms 13y ago

I've got a feeling it's something that many people have been doing on even mildly complex sites for a long time, but people seem to be trying to SaaS it up recently. protip: Dynamically generating content by URLs is also a nice DOS attack surface if you don't rate limit in some way or try to detect abuse.

agschwender 13y ago
Definitely, true. The purpose of the signature and allowed hosts functionality is there to help mitigate that risk. Also, since the image service would be run separate from the main application, it shouldn't impact the performance of your site. Likewise, in a real production environment, this would run behind a CDN or Varnish, the DOS would just prevent new images from being generated until it could be addressed.

na85 13y ago

Honest question: Why is it such a great idea? Wouldn't it be easier/quicker to either: a) Have a command-line tool to resize images? Alternatively, one that's integrated into the shell so that you can access it from context menus? I.e. right-click -> "resize image" b) Just open it in mspaint or your open-source image editor of choice?

OhHeyItsE 13y ago
Pretty sure the idea is to make this available as a service, for UGC-type sites.
canterburry 13y ago
These services are becoming more interesting as responsive web design takes off and everyone tries to format their images based on client agent needs. Often the exact dimensions and treatment of the image can't be determined before request time depending on the user agent's screen size, resolution etc. Also, why spend time as a designer cropping and scaling images when you can automate it all and make it on demand? This thread caught by eye since I have been working on the very same idea for the last year. http://www.pixtulate.com http://www.pixtulate.com
joevandyk 13y ago
It's simpler to store the original image, then generate URLs like http://image-server.com/resize?url=http://s3.com/original.jpg&width=300&height=200 http://image-server.com/resize?url=http://s3.com/original.jp... This lets you change the size of thumbnails by only changing the URLs. I've been doing it for years, works great.
- brandon272 13y ago
  Doesn't this mean that the server needs to manually process the image on each request?
  - michaelmior 13y ago
    You should stick a CDN in front so this doesn't happen. Ideally once the server generates the image, it would also be cached there.
    - joevandyk 13y ago
      You probably want nginx in front of pillbox. Nginx setup as a basic http proxy cache. So, CDN -> nginx -> pillbox. This is because CDNs can send lots of concurrent requests to the origin server if lots of people are requesting the same new image simultaneously.

bjt 13y ago

PHP sites have been doing this with TimThumb for quite a while. https://code.google.com/p/timthumb/ https://code.google.com/p/timthumb/ I wrote https://bitbucket.org/btubbs/thumpy https://bitbucket.org/btubbs/thumpy at work and we've been using it in production for a couple years. It's very similar to Pilbox.

gaelenh 13y ago

I wrote a similar image service (resize, crop, border, color analysis, format, and other fun features) a few years ago. It served about 6M image requests a day from 4 servers, plus a whole lot more from the CDN. If you're using a CDN, just make sure they cache on query string parameters. Our resize controls were part of the URI, but the other commands were part of the QS. Our CDN stripped queries, so the first image would be cached and any calls with different QS would return the first hit. A possible alternative that I never implemented was using the request header instead of the query string for additional commands. Edit to add: Also, if you have lots of resized images on a page, be careful when Google or Bing scraped you. Your CPU and IO will go through the roof as your servers go crazy trying to dynamically generate all the images.

canterburry 13y ago
So what happened to it? Are you still running it? If not, why not?
- gaelenh 13y ago
  Not running (probably). We were acquired by a competitor. Last I heard, they upgrade their image server code with the bare essential features to migrate over our clients. A lot of good software was buried in that acquisition.
lnanek2 13y ago
Kind of strange the bots would spike you. Wouldn't anything on a page have been used before and thus cached on your server as a file? I've written all this stuff before as well, including overlaying site logo on images that were hotlinked to boot, but I always cached to the file system anything I had to generate the first time it was requested and simply served it after.
- gaelenh 13y ago
  Short version: We had direct access to all the large news image providers and analyzed the images for topic and story identification. We used this data to dynamically generate hundreds of thousands of pages. Way too much data to cache to file (our storage cluster was many many terabytes). We used a CDN to cache the images. When a bot would scrape, it would hit all the old pages that fell out of the cache. Perhaps our fault for having such a large sitemap.xml.

Terretta 13y ago

> Being able to resize images via url parameters is one of those great ideas, but I've always been surprised by how long it's taken to catch on. Our video CDN customers can use a custom URL to get a thumbnail of any size from any key frame in their video files, fantastic for building "trick play" responsive HTML5 video player UIs without having to manage a single image. Just one of the countless little nice-to-haves that sets a VDN apart from a CDN.

samstave 13y ago
Who are you using as a VDN?
- ihsw 13y ago
  His message indicated that he works for a VDN provider, rather than be a user of one. This would seem to be his employer: http://www.advection.net/ http://www.advection.net/