8 ms·

Shrine – A new solution for handling file uploads in Ruby

52 points by lucisferre 11y ago

anonova 11y ago
The creator of carrierwave started the new project refile [1] as a better alternative. Is there a comparison between the two approaches, i.e, refile vs shrine? [1]: https://github.com/refile/refile https://github.com/refile/refile
- _neil 11y ago
  I'd also be interested in a brief comparison. This is the first I've heard of both Shrine and Refile. I'm really excited that backgrounding and direct uploads are being simplified in both projects.
- janko-m 11y ago
  I was actually contributing actively to Refile before, so in a lot of ways Shrine is similar to Refile (the idea of backends is basically the same). However, the main difference is that Refile does processing on-the-fly, while Shrine (and CarrierWave) do it upfront, so it goes in that direction. Basically in Shrine I added all of the features I was missing and wanted to change, which I wasn't able to add to Refile in a compatible way. One major addition is the support for background jobs, which I just couldn't see how it could be added to Refile. Also, I wanted Shrine to be far less opinionated than Refile, that you can have much more options how to do uploads.
  - jrochkind1 11y ago
    Can you explain what you mean by "upfront" vs "on-the-fly"? I'm somehow not getting it. This looks very nice. It'll need lots of docs with examples to understand how to put together the nicely flexible building blocks you provide.
    - janko-m 11y ago
      "Upfront" processing means that you're doing the processing during upload, and on the "on-the-fly" processing means that you first upload the file, and then it's processed dynamically through the URL. For example, in Refile you make an URL to the uploaded file like this: /attachments/sdkd/resize/300/300 That will make Refile trigger the specified processing at the moment the URL is requested. You should then put a CDN or a reverse proxy in front to kind of cache these processed files, so that they're not processed each time. About the documentation, I wrote documentation for each of the plugins, and I linked them all on http://shrinerb.com http://shrinerb.com. The plugin names should hopefully indicate what feature they accomplish, and since they're all individual, it's up to the user to choose what features they want (i.e. combination of plugins).
      - jrochkind1 11y ago
        So "up front" still includes doing the processing in a background job async, as long as it's triggered at point of upload, not at point of request? "upfront" but async background does seem to be the best way to approach it to me, at least if you had to pick one -- that's what Shrine does?
        janko-m 11y ago
        Yes, you got it right, this is exactly what it means. Yeah, there are some rare use cases for on-the-fly processing (think a WYSIWYG editor where the user can choose the size of the image), but for me upfront definitely works best.
        jrochkind1 11y ago
        Cool. I'd call those two choices "upon upload" (instead of 'upfront'), and "upon first view" (instead of "on the fly"), but I'm sure my terminology would confuse someone else too.
michaelmior 11y ago
I haven't done much Web work in Ruby, but in my head the first thing that comes to mind for file uploads is Paperclip[0]. I'd be curious to hear thoughts on the various alternatives (e.g. CarrierWave, Refile, Shrine). [0] https://github.com/thoughtbot/paperclip https://github.com/thoughtbot/paperclip
- gonyea 11y ago
  I've used Paperclip in production. It's good and does what it's supposed to. The main issue is if you have a lot of different image sizes. It will basically process all of the images inline + store them to s3, etc. My ideal flow for image processing is to create the thumbnail (which the user sees right away) and to throw the rest of the image processing work on a background queue. This minimizes the time spent processing a request and keeps the front-end snappy.
  - hiphopyo 11y ago
    You could do that with https://github.com/jrgifford/delayed_paperclip/ https://github.com/jrgifford/delayed_paperclip/ I guess.
    - janko-m 11y ago
      The part that I mind about both delayed_paperclip and carrierwave_backgrounder is that it doesn't really allow you to write your own job classes. This can be quite limiting for example if you want your jobs to send information to some monitoring services. Another part that I mind is the user experience. Ok, both of these gems allow you to know when the backgrond job is in process, and then you can display some placeholder image until the processing is done. But that's not really nice user experience. In Shrine the user immediately sees the image they uploaded, because it has that image already cached (usually on the filesystem), and then from the user's point of view the uploading is finished, before the background job even started. CarrierWave also has the cached image, but by design it's not possible to use it, while Paperclip doesn't cache images.
      - 11y ago
gonyea 11y ago
Great work! This looks like it addresses my issues with image uploads, specifically the backgrounding. Thanks for open sourcing this. I'll throw it in my stack and give some feedback.

52 points by lucisferre 11y ago

The creator of carrierwave started the new project refile [1] as a better alternative. Is there a comparison between the two approaches, i.e, refile vs shrine? [1]: https://github.com/refile/refile https://github.com/refile/refile

_neil 11y ago
I'd also be interested in a brief comparison. This is the first I've heard of both Shrine and Refile. I'm really excited that backgrounding and direct uploads are being simplified in both projects.
janko-m 11y ago
I was actually contributing actively to Refile before, so in a lot of ways Shrine is similar to Refile (the idea of backends is basically the same). However, the main difference is that Refile does processing on-the-fly, while Shrine (and CarrierWave) do it upfront, so it goes in that direction. Basically in Shrine I added all of the features I was missing and wanted to change, which I wasn't able to add to Refile in a compatible way. One major addition is the support for background jobs, which I just couldn't see how it could be added to Refile. Also, I wanted Shrine to be far less opinionated than Refile, that you can have much more options how to do uploads.
- jrochkind1 11y ago
  Can you explain what you mean by "upfront" vs "on-the-fly"? I'm somehow not getting it. This looks very nice. It'll need lots of docs with examples to understand how to put together the nicely flexible building blocks you provide.
  - janko-m 11y ago
    "Upfront" processing means that you're doing the processing during upload, and on the "on-the-fly" processing means that you first upload the file, and then it's processed dynamically through the URL. For example, in Refile you make an URL to the uploaded file like this: /attachments/sdkd/resize/300/300 That will make Refile trigger the specified processing at the moment the URL is requested. You should then put a CDN or a reverse proxy in front to kind of cache these processed files, so that they're not processed each time. About the documentation, I wrote documentation for each of the plugins, and I linked them all on http://shrinerb.com http://shrinerb.com. The plugin names should hopefully indicate what feature they accomplish, and since they're all individual, it's up to the user to choose what features they want (i.e. combination of plugins).
    - jrochkind1 11y ago
      So "up front" still includes doing the processing in a background job async, as long as it's triggered at point of upload, not at point of request? "upfront" but async background does seem to be the best way to approach it to me, at least if you had to pick one -- that's what Shrine does?
      - janko-m 11y ago
        Yes, you got it right, this is exactly what it means. Yeah, there are some rare use cases for on-the-fly processing (think a WYSIWYG editor where the user can choose the size of the image), but for me upfront definitely works best.
        jrochkind1 11y ago
        Cool. I'd call those two choices "upon upload" (instead of 'upfront'), and "upon first view" (instead of "on the fly"), but I'm sure my terminology would confuse someone else too.

michaelmior 11y ago

I haven't done much Web work in Ruby, but in my head the first thing that comes to mind for file uploads is Paperclip[0]. I'd be curious to hear thoughts on the various alternatives (e.g. CarrierWave, Refile, Shrine). [0] https://github.com/thoughtbot/paperclip https://github.com/thoughtbot/paperclip

gonyea 11y ago
I've used Paperclip in production. It's good and does what it's supposed to. The main issue is if you have a lot of different image sizes. It will basically process all of the images inline + store them to s3, etc. My ideal flow for image processing is to create the thumbnail (which the user sees right away) and to throw the rest of the image processing work on a background queue. This minimizes the time spent processing a request and keeps the front-end snappy.
- hiphopyo 11y ago
  You could do that with https://github.com/jrgifford/delayed_paperclip/ https://github.com/jrgifford/delayed_paperclip/ I guess.
  - janko-m 11y ago
    The part that I mind about both delayed_paperclip and carrierwave_backgrounder is that it doesn't really allow you to write your own job classes. This can be quite limiting for example if you want your jobs to send information to some monitoring services. Another part that I mind is the user experience. Ok, both of these gems allow you to know when the backgrond job is in process, and then you can display some placeholder image until the processing is done. But that's not really nice user experience. In Shrine the user immediately sees the image they uploaded, because it has that image already cached (usually on the filesystem), and then from the user's point of view the uploading is finished, before the background job even started. CarrierWave also has the cached image, but by design it's not possible to use it, while Paperclip doesn't cache images.
    - 11y ago

gonyea 11y ago

Great work! This looks like it addresses my issues with image uploads, specifically the backgrounding. Thanks for open sourcing this. I'll throw it in my stack and give some feedback.