9 ms·
Live Streaming in Rails 4.0
- mtkd 14y agoI really respect the hard work going in to the evolution of Rails, but it does feel like some new features should be considered as Gems. The hardest part of managing mature software is keeping it lean.
- thibaut_barrere 14y agoStreaming looks like a natural evolution to me, to gradually send JSON data back to the client etc for instance. I don't think this is something that could be done easily as a gem (at least not without a serious dose of unstable monkey-patching).
- jeltz 14y agoThe lightweight framework Sinatra has had streaming for a some time now, and if it is not too heavy for Sinatra it should not be too heavy for rails. Rails is way larger than Sinatra.
- masklinn 14y ago> The lightweight framework Sinatra has had streaming for a some time now Rack itself has response streaming support[0]. As a result, even before Sinatra 1.3 and the `stream` helper method you could stream responses in Sinatra. [0] since pretty much forever ago: https://github.com/rack/rack/commit/53e299724b9173efe50e9afc372c1cb7ca02c40f https://github.com/rack/rack/commit/53e299724b9173efe50e9afc...
- jeltz 14y agoYeah, I have implemented streaming myself once for a rack app. It was rather simple to implement and has worked stably in production for over two years now.
- tenderlove 14y agoI didn't go in to this in much detail in my article, but I'd like to follow up with my long term plan for this feature. I think that buffered responses are a special case of streaming responses (they're streaming responses with one chunk), and I'd like to make this API the underpinnings of the response system in Rails. One of the things that I think Rack got wrong and that J2EE, Node.js, etc got right is that the response bodys should be treated as IO objects. Whether the IO object streams, buffers, gzips, etc is up to the IO object. Regardless, the API remains the same. I hope to eventually push the concept of an IO response up to Rack itself and eliminate this code from Rails.
- thibaut_barrere 14y agoI look forward to your follow-up on sending JSON streams back to the client! It would be a nice thing to gradually add the items to a js, client-rendered list (such as with KnockOut.js).
- masklinn 14y ago> One of the things that I think Rack got wrong and that J2EE, Node.js, etc got right is that the response bodys should be treated as IO objects. Whether the IO object streams, buffers, gzips, etc is up to the IO object. I am rather surprised by this declaration, because Rack response bodies do indeed behave like "IO objects": they respond to `each`. That's the only requirement. Guess what responds to `each`? IO. The problem seems entirely self-inflicted by Rails: if it builds an eager, non-streaming Response object, that means you can't stream with it. But it's got nothing whatsoever to do with Rack. By comparison, Rack's own Response helper object does provide easy streaming access: calls to `Response#write` within the `Response#finish` block are streamed directly. Your code would look roughly like this in e.g. Sinatra (note: entirely untested): response.finish do 100.times { response.write "hello, world\n" } end Alternatively, you could implement your own #each: class Stream def each 100.times { yield "hello, world\n" } end end get('/') { Stream.new } this one is straight from the Sinatra docs[0], and aside from the `get` call it can trivially be converted to a raw Rack application: lambda do |env| [200, {'Content-Type' => 'application/octet-stream'}, Stream.new] end [0] Sinatra also provides a `stream` helper method which does most of the wrapping and tries to make non-streaming-aware middleware play nicely, that can be used in stead of these lower-level calls: http://www.sinatrarb.com/contrib/streaming.html http://www.sinatrarb.com/contrib/streaming.html
- heyrhett 14y agoI don't want to be a wet blanket here, but does anyone else find this "tender love making" branding all over Aaron Patterson's technical content a bit... distracting? Personally I do. That said, Aaron does amazing work and he should be able to brand it however he likes. I'm also very grateful for what he gives to the community. So, am I just an old fuddy-duddy, or what? I mean, personally, I actually enjoy it, despite the distraction, but I'm surprised that this seems to be such a non-issue to everyone else.
- why-el 14y agoDo you have an issue with "tender love making" branding or branding in general? I think a healthy dose of personal branding is ok for me. With that said, the developer community is much much tolerating place than other communities. I mean, think about the rude behavior from some developers and how far we go in tolerating it because of our "They built X, so they can do unrelated Y however they want". Compared to that, some love spreading is actually very nice. :) Edit: the sentence "They built X, so they can do unrelated Y however they want" is confusing, even to me. What I meant is, we tolerate some otherwise really inconvenient stuff, like Crockford calling all JSHint users "stupid people". You get the point.
- tenderlove 14y agolol! Do you have suggestions for improvement? I'm not sure what I'm doing to brand the site. My expertise is in tech, not branding. I'm open to suggestions, but I will not compromise on the color pink (or Ruby-San)! :-)
- 14y ago
- jeltz 14y agoI find it curios that he does not mention Thin as a webserver which is good for streaming. Thin was perhaps the first ruby webserver which got good support for streaming responses. While Rainbows! and Puma might be better at least Thin should deserve a mention due to its popularity alone. It is way more downloaded than both Rainbows! and Puma together. Is there some problem with running Thin together with stream in rails 4?
- tenderlove 14y agoAh, sorry. I forgot to mention thin. I will update the article. I just mentioned the web servers I've tested with! :)
- jeltz 14y agoGood to know. I was worried there could be some bad interaction between rails and the eventmachine. :)
- bascule 14y agoI suspect there will be bad interactions with EventMachine as it doesn't have a flow control model for writes and will just accept an unbounded amount of data into its write buffer. This is a big problem if you have slow consumers.
- ZoFreX 14y agoRails + SSE + Thin = a dream come true for me. Thank you for the hard work you're putting in!
- hemancuso 14y agoThe one thing I think Rack really messed up is the hard requirement for rewindable input. Try streaming an upload through to a back-end. Can't do it.
- tbenst 14y agoVery cool - thanks for the early look at this patch! Is this live streaming practical for high bandwidth I/O - say live video streaming?