4 ms·

I'm curious, how would the HN community tackle optimizing (and/or re- optimizing) ~200k images sitting in an S3 bucket?

technotarek 8y ago
Edit: Was thinking more of what self-implemented workflows/precessing tools one might consider (node packages etc) as opposed to paid SAAS options.
yjftsjthsd-h 8y ago
Look at AWS batch - https://aws.amazon.com/batch/ https://aws.amazon.com/batch/
craz8 8y ago
I would think about using Cloudinary to process these on demand as needed if your usage can support this Cloudinary Fetch pulls images from your S3, caches them, and serves the size and format you request all without touching the original files
- technotarek 8y ago
  Cloudinary is nice, but it gets expensive really fast on a high traffic site that's image heavy and bandwidth optimized. For example, a single image might have 5-6 versions on top of the original -- low quality thumbnail, high quality thumbnail, default, default 2x retina, default 3x retina. Then activate their webp service and perhaps offer another image aspect ratio. Each is a conversion that counts against your quota. Now you've got 15+ (con)versions and you're shelling out hundreds of dollars a month.
  - dmitrymukhin 8y ago
    Cloudinary, Imgix, Uploadcare (which I' working for) and others save you money because you don't have to develop and maintain these moving parts. You have to constantly check what's happening with browsers, what formats/encoders are available and which are the best etc. It's classic build vs. buy dilemma. In majority of cases it's much more cost effective to buy. BTW, Uploadcare doesn't charge for file processing at all, only for CDN traffic. So you can create as much image variants as you need.
adityapatadia 8y ago
One option is to use https://www.gumlet.com https://www.gumlet.com It image processing servers located across the world and ensures very low latency image processing.

Edit: Was thinking more of what self-implemented workflows/precessing tools one might consider (node packages etc) as opposed to paid SAAS options.

yjftsjthsd-h 8y ago

Look at AWS batch - https://aws.amazon.com/batch/ https://aws.amazon.com/batch/

craz8 8y ago

I would think about using Cloudinary to process these on demand as needed if your usage can support this Cloudinary Fetch pulls images from your S3, caches them, and serves the size and format you request all without touching the original files

technotarek 8y ago
Cloudinary is nice, but it gets expensive really fast on a high traffic site that's image heavy and bandwidth optimized. For example, a single image might have 5-6 versions on top of the original -- low quality thumbnail, high quality thumbnail, default, default 2x retina, default 3x retina. Then activate their webp service and perhaps offer another image aspect ratio. Each is a conversion that counts against your quota. Now you've got 15+ (con)versions and you're shelling out hundreds of dollars a month.
- dmitrymukhin 8y ago
  Cloudinary, Imgix, Uploadcare (which I' working for) and others save you money because you don't have to develop and maintain these moving parts. You have to constantly check what's happening with browsers, what formats/encoders are available and which are the best etc. It's classic build vs. buy dilemma. In majority of cases it's much more cost effective to buy. BTW, Uploadcare doesn't charge for file processing at all, only for CDN traffic. So you can create as much image variants as you need.

adityapatadia 8y ago

One option is to use https://www.gumlet.com https://www.gumlet.com It image processing servers located across the world and ensures very low latency image processing.