5 ms·

This is a huge refactoring of an old project of mine -- the Nginx HTTP Push Module. I'm wondering if anyone here has used it. Most importantly, I want feedback

by slact 11y ago

This is a huge refactoring of an old project of mine -- the Nginx HTTP Push Module. I'm wondering if anyone here has used it. Most importantly, I want feedback on the documentation. Did I overcomplicate things? Does it need more examples? Does it need more live code? Is it too long? Too short? Etc.

jkarneges 11y ago
I love the clean client API, e.g. the use of ETags with long-polling. In some ways it reminds me of our LiveResource protocol (http://liveresource.org http://liveresource.org). It's still a work in progress, but maybe we can consolidate ideas. It'd be cool if an LR client could be pointed at an Nchan resource.
- deleted 11y ago
  [deleted]
- slact 11y ago
  Interesting, I'll read through this. Nchan code is quite modular, so it would be pretty easy to add some kind of negotiation like this. Let's talk more. For reference, here's the old document I wrote for the long-polling protocol back in '09: https://pushmodule.slact.net/protocol.html https://pushmodule.slact.net/protocol.html
comboy 11y ago
Would be really cool if you could include some example JS client for this. I'm sure you have some for testing, so even if it's not production ready code it would make testing of your module much easier. Almost everybody will want to write the same code for it anyway, with reconnection and fallbacks, so it may be a good starting point for others to contribute to the project - much more JS guys than C coders intimately familiar with nginx.
- slact 11y ago
  Yep, I'll be adding an example js client, but I don't intend on standardizing this. Mostly because I'm also a js guy and I like mootools, and I don't want to start a best-js-framework flamewar with an official client. But example code will be up for sure.
ende42 11y ago
Thanks for your work! Actually we (laut.fm) are using the HTTP Push Module for our public API for a live stream of tracks which get played on our ~ 1500 icecast stations. We offer 3 formats: http://api.laut.fm/song_change.stream.json http://api.laut.fm/song_change.stream.json for a line separated JSON HTTP stream, ws://api.laut.fm/song_change.ws.json for the same as websocket endpoint and http://api.laut.fm/song_change.chunk.json http://api.laut.fm/song_change.chunk.json for the last x songs. It's not really high volume, just 6 to 7 per second. But it runs basically unattended for years now and I'm pretty happy with it. Is there any reason to update to the new one (other than new features; admittedly I haven't really looked into NCHAN)?
- slact 11y ago
  Well, if it ain't broke, as they say... here's a page on the differences between Nchan and the Push Module: https://nchan.slact.net/upgrade https://nchan.slact.net/upgrade . One important thing I forgot to add is that the Push Module suffered from memory fragmentation under high load, and with a fixed-size shared-memory chunk that could mean running out of usable shared memory for a long-running nginx process. If you're not experiencing that, and you don't need to scale up, or the new features don't appeal to you, don't upgrade -- certainly not yet. Maybe in a month or two when nchan makes its way into the nginx-extras debian package (replacing the push module), then consider upgrading.
  - ende42 11y ago
    Thanks for the upgrade link. I missed that. Great that NCHAN is (almost; besides directive prefix) configuration compatible! The "Subscribers" section of "Push Module" should include websocket, I think.
    - slact 11y ago
      > besides directive prefix The old setting is still recognized. > Subscribers" section of "Push Module" should include websocket Nope, push module didn't do websocket. You may be thinking of the Push _Stream_ module, which is an independent push module fork.
solyaris 11y ago
great project! I retwitted: https://twitter.com/solyarisoftware/status/673798502049280000 https://twitter.com/solyarisoftware/status/67379850204928000... Documentation is VERY well done. publisher: Curl examples are perfect. subscribers: maybe some examples in a language ( Ruby? :-) ) could help dummies like me. I'll study and if I can I'll propose you respect giorgio
- slact 11y ago
  Thanks! I'll be adding a sample JS subscriber client and some sample code with that as well.

by slact 11y ago

jkarneges 11y ago

I love the clean client API, e.g. the use of ETags with long-polling. In some ways it reminds me of our LiveResource protocol (http://liveresource.org http://liveresource.org). It's still a work in progress, but maybe we can consolidate ideas. It'd be cool if an LR client could be pointed at an Nchan resource.

deleted 11y ago
[deleted]
slact 11y ago
Interesting, I'll read through this. Nchan code is quite modular, so it would be pretty easy to add some kind of negotiation like this. Let's talk more. For reference, here's the old document I wrote for the long-polling protocol back in '09: https://pushmodule.slact.net/protocol.html https://pushmodule.slact.net/protocol.html

comboy 11y ago

Would be really cool if you could include some example JS client for this. I'm sure you have some for testing, so even if it's not production ready code it would make testing of your module much easier. Almost everybody will want to write the same code for it anyway, with reconnection and fallbacks, so it may be a good starting point for others to contribute to the project - much more JS guys than C coders intimately familiar with nginx.

slact 11y ago
Yep, I'll be adding an example js client, but I don't intend on standardizing this. Mostly because I'm also a js guy and I like mootools, and I don't want to start a best-js-framework flamewar with an official client. But example code will be up for sure.

ende42 11y ago

Thanks for your work! Actually we (laut.fm) are using the HTTP Push Module for our public API for a live stream of tracks which get played on our ~ 1500 icecast stations. We offer 3 formats: http://api.laut.fm/song_change.stream.json http://api.laut.fm/song_change.stream.json for a line separated JSON HTTP stream, ws://api.laut.fm/song_change.ws.json for the same as websocket endpoint and http://api.laut.fm/song_change.chunk.json http://api.laut.fm/song_change.chunk.json for the last x songs. It's not really high volume, just 6 to 7 per second. But it runs basically unattended for years now and I'm pretty happy with it. Is there any reason to update to the new one (other than new features; admittedly I haven't really looked into NCHAN)?

slact 11y ago
Well, if it ain't broke, as they say... here's a page on the differences between Nchan and the Push Module: https://nchan.slact.net/upgrade https://nchan.slact.net/upgrade . One important thing I forgot to add is that the Push Module suffered from memory fragmentation under high load, and with a fixed-size shared-memory chunk that could mean running out of usable shared memory for a long-running nginx process. If you're not experiencing that, and you don't need to scale up, or the new features don't appeal to you, don't upgrade -- certainly not yet. Maybe in a month or two when nchan makes its way into the nginx-extras debian package (replacing the push module), then consider upgrading.
- ende42 11y ago
  Thanks for the upgrade link. I missed that. Great that NCHAN is (almost; besides directive prefix) configuration compatible! The "Subscribers" section of "Push Module" should include websocket, I think.
  - slact 11y ago
    > besides directive prefix The old setting is still recognized. > Subscribers" section of "Push Module" should include websocket Nope, push module didn't do websocket. You may be thinking of the Push _Stream_ module, which is an independent push module fork.

solyaris 11y ago

great project! I retwitted: https://twitter.com/solyarisoftware/status/673798502049280000 https://twitter.com/solyarisoftware/status/67379850204928000... Documentation is VERY well done. publisher: Curl examples are perfect. subscribers: maybe some examples in a language ( Ruby? :-) ) could help dummies like me. I'll study and if I can I'll propose you respect giorgio

slact 11y ago
Thanks! I'll be adding a sample JS subscriber client and some sample code with that as well.