3 ms·

I'd love for there to be an AWS photo transcoding service as well. If you have a large site, creating your own photo transcoder workflow is not a big deal, and

by DenisM 13y ago

I'd love for there to be an AWS photo transcoding service as well. If you have a large site, creating your own photo transcoder workflow is not a big deal, and you can make it elastic by combining a bunch of other AWS services, but for a small web site it's a different picture. For me photo transcoding comes in batches that take 5-15 minutes to process, a few times per week. This work bogs down the tiny instance, so I have to keep around a medium instance just for the occasional conversion work, but 99% of the time it's a waste of hardware. It's also a waste of my time having to write code to deal with things like iPhone picture image rotation and so on.

IanCal 13y ago
Have you looked at PiCloud? It looks like exactly the base you want to build on. They run AWS instances, but charge by the millisecond. You can run one up just for the 5-15 minutes it takes. Or if you want to get fancier, try their queues [0], they've got an example of doing image processing to explain how it works. Essentially, you set up your processing pipeline attached to a queue. You then drop messages on the queue whenever you want to do work, and they spin up one or many machines (controllable by you) to process the data. Then after 20 seconds of nothing new appearing on the queue, they turn them all off. They'll even give you 20 hours of compute time free, every month. Edit - if you don't want to build it, drop me a message, I can do it. [0] http://blog.picloud.com/2013/04/03/introducing-queues-creating-a-pipeline-in-the-cloud/ http://blog.picloud.com/2013/04/03/introducing-queues-creati...
- DenisM 13y ago
  I'll spend some more time on it, but right now it looks like another platform I need to learn, which I don't have time for. What I want is a service that takes this URL and returns a resized JPG: http://SimpleImageService.com/resize ?account=DenisM &source=http://s3.amazonaws.com/DenisMsPictures/Image1.TIFF &size=400x400 That's it. I'll drop this into my app and forget about it. Set the prices at your AWS cost + 25%, and send me the bill at the end of each month.
  - georgemcbay 13y ago
    When I was working at Technicolor/Medinavi (after they acquired most of the chumby employees), I wrote basically this exact service (massively parallel with goroutines and with filesystem-based caching, and they were running it on an AWS instance) in about 100 lines of Go code using ImageMagick through CGO (because ImageMagick has a lot more image operation and resample filter options than the built-in go image/draw package). Sadly, I obviously can't just share the code because I don't own the copyright.
  - madlag 13y ago
    That's exactly what Stupeflix Tasks will do in a few weeks, along with video thumbnails strips, thumbnailing of videos at any point in time, video uploads, audio waveform, text to speech, video reverse etc. The beta doc is at https://stupeflix-tasks-api.readthedocs.org/en/latest/versions/index.html https://stupeflix-tasks-api.readthedocs.org/en/latest/versio... , feel free to give me some feedback on it ! And if you want to do way more than just transcode, but create completely your video from images / music / videos , you can use the video rendering service at http://developer.stupeflix.com/ http://developer.stupeflix.com/ . It will migrated to the Tasks api too. (Disclaimer : I am the CTO )
  - IanCal 13y ago
    Well I need a side project at the moment :) I'll try and put something together and drop you a message.
  - omfg 13y ago
    Cloudinary does that. We used it for a bit but ultimately decided to roll our own. Their service worked great though. We've tried a bunch of options since hosting on Heroku makes large scale image processing a chore. Cloudinary was best of the bunch.
  - ruckusing 13y ago
    Check out imgix.com
georgemcbay 13y ago
Photo transcoding is a fast enough operation that for most sites doing it on-demand with a cache of previous results makes the most sense. Eg. store only the original to start, when 1024p wide version is requested, check filesystem or database blob cache for 1024p version, if it exists, serve it, if not, transcode then serve it, cache it so next time it is requested you'll have it. This way you don't spend cycles transcoding photos at resolutions that are subsequently never requested as various sizes. And existing APIs like ImageMagick make all the normal image processing features a breeze to implement. Granted, photo transcoding is still a good idea for a service for people who really don't care about it and just want something that works out of the box, but video transcoding is a much tougher nut to crack (despite ffmpeg/libav and other solutions, there's still a lot of black magic people who haven't gotten their hands dirty with this sort of thing don't realize is an issue until they start transcoding videos and noticing audio sync and other such issues) and thus I'd be way more likely to pay for a service to do it rather than roll my own with video as opposed to photos.

by DenisM 13y ago

IanCal 13y ago

Have you looked at PiCloud? It looks like exactly the base you want to build on. They run AWS instances, but charge by the millisecond. You can run one up just for the 5-15 minutes it takes. Or if you want to get fancier, try their queues [0], they've got an example of doing image processing to explain how it works. Essentially, you set up your processing pipeline attached to a queue. You then drop messages on the queue whenever you want to do work, and they spin up one or many machines (controllable by you) to process the data. Then after 20 seconds of nothing new appearing on the queue, they turn them all off. They'll even give you 20 hours of compute time free, every month. Edit - if you don't want to build it, drop me a message, I can do it. [0] http://blog.picloud.com/2013/04/03/introducing-queues-creating-a-pipeline-in-the-cloud/ http://blog.picloud.com/2013/04/03/introducing-queues-creati...

DenisM 13y ago
I'll spend some more time on it, but right now it looks like another platform I need to learn, which I don't have time for. What I want is a service that takes this URL and returns a resized JPG: http://SimpleImageService.com/resize ?account=DenisM &source=http://s3.amazonaws.com/DenisMsPictures/Image1.TIFF &size=400x400 That's it. I'll drop this into my app and forget about it. Set the prices at your AWS cost + 25%, and send me the bill at the end of each month.
- georgemcbay 13y ago
  When I was working at Technicolor/Medinavi (after they acquired most of the chumby employees), I wrote basically this exact service (massively parallel with goroutines and with filesystem-based caching, and they were running it on an AWS instance) in about 100 lines of Go code using ImageMagick through CGO (because ImageMagick has a lot more image operation and resample filter options than the built-in go image/draw package). Sadly, I obviously can't just share the code because I don't own the copyright.
- madlag 13y ago
  That's exactly what Stupeflix Tasks will do in a few weeks, along with video thumbnails strips, thumbnailing of videos at any point in time, video uploads, audio waveform, text to speech, video reverse etc. The beta doc is at https://stupeflix-tasks-api.readthedocs.org/en/latest/versions/index.html https://stupeflix-tasks-api.readthedocs.org/en/latest/versio... , feel free to give me some feedback on it ! And if you want to do way more than just transcode, but create completely your video from images / music / videos , you can use the video rendering service at http://developer.stupeflix.com/ http://developer.stupeflix.com/ . It will migrated to the Tasks api too. (Disclaimer : I am the CTO )
- IanCal 13y ago
  Well I need a side project at the moment :) I'll try and put something together and drop you a message.
- omfg 13y ago
  Cloudinary does that. We used it for a bit but ultimately decided to roll our own. Their service worked great though. We've tried a bunch of options since hosting on Heroku makes large scale image processing a chore. Cloudinary was best of the bunch.
- ruckusing 13y ago
  Check out imgix.com

georgemcbay 13y ago

Photo transcoding is a fast enough operation that for most sites doing it on-demand with a cache of previous results makes the most sense. Eg. store only the original to start, when 1024p wide version is requested, check filesystem or database blob cache for 1024p version, if it exists, serve it, if not, transcode then serve it, cache it so next time it is requested you'll have it. This way you don't spend cycles transcoding photos at resolutions that are subsequently never requested as various sizes. And existing APIs like ImageMagick make all the normal image processing features a breeze to implement. Granted, photo transcoding is still a good idea for a service for people who really don't care about it and just want something that works out of the box, but video transcoding is a much tougher nut to crack (despite ffmpeg/libav and other solutions, there's still a lot of black magic people who haven't gotten their hands dirty with this sort of thing don't realize is an issue until they start transcoding videos and noticing audio sync and other such issues) and thus I'd be way more likely to pay for a service to do it rather than roll my own with video as opposed to photos.