4 ms·

Show HN: Razorframe – A Node.js module for empowering real-time apps at scale

52 points by edshpark 10y ago

edshpark 10y ago
Razorframe is a Node.js library for enabling the effective backend scaling of your real-time application. Built on top of Socket.io, Razorframe’s simple server-and client-side API allows you to easily employ the task processing of multiple, parallel Node instances in conjunction with an in-memory queue. Our team set out to deliver on pushing the impact of real-time web through maximizing backend resources provided by multi-core server systems. By allowing your Node.js server to run on multiple threads and ensuring data consistency through an evented queueing system, Razorframe makes your application’s backend more resilient and performant under duress. We'd love to get some feedback on Razorframe and ways in which we can make it a better solution for your next Node.js project. Feel free to check us out on GitHub! Thank you!
- songzme 10y ago
  Would this be a firebase competitor? If so, how does it compare?
  - tron1k 10y ago
    Hey, it's Travis, one of the co-creators of Razorframe... Firebase is a paid platform-as-a-service that includes realtime database hosting (what it's primarily known for) as well as numerous other services such as app analytics and hosting. It's a great platform and we definitely drew inspiration from them. Razorframe, on the other hand, is an open-source project that addresses 2 of the main trends we see in modern web applications: real-time client UIs and flexible server-side scaling. We built a Node module that you can bring into your server-side code to quickly get up and running with websockets (via socket.io) and Node clusters in order to accomplish both. Check us out on GitHub and let us know what you think:)
doublerebel 10y ago
What happens when one or many Node processes die with items left in the queue? Related, how are graceful restarts handled? (These are issues solved by existing message queues.)
- tron1k 10y ago
  Nothing happens to items in the queue in such a case. New workers will automatically respawn, and the adapter that links all the workers on the back-end will re-attach to newly spawned workers. Clients formerly connected to workers that are now gone will have to re-attach to new workers by socket.io.
ianstormtaylor 10y ago
Congrats on the launch! Glad to see more people working on these kinds of problems. As someone who hasn't built one of these systems before... What does Razorfish give you that running Socket.io as a separate microservice that talks to your existing API service over an existing message queue doesn't? Is it for people with architectures that don't have an existing queuing or API services already? Or does it handle sharding the Socket.io part easily which usually has to be vertically scaled? (Sorry if these questions don't make sense, I haven't dug into my requirements for these things yet but I know they're coming up!) I think a very clear explanation of that would help me (and maybe others) understand its value and how hard it would be to roll your own here.
- edshpark 10y ago
  Hey Ian - this is Edward, one of the co-creators of Razorframe. Thanks for your thought provoking question! Developers can certainly utilize the architecture you've described to great effect, but our solution aims to deliver a lightweight option that integrates technologies that don't normally play nice (Socket.io and clustered Node instances) but can have awesome scaling potential when they do. By keeping all the features within a familiar environment in Node, we've experienced that development time was vastly reduced in a potential project that expected a high volume of Websocket communications. For example, the incorporation of an in-memory queue reduces the need for developers to incorporate an external piece of tech (and any new language adoption) into their Node.js projects
agentultra 10y ago
> Those interactions are held in the queue only as long as the server needs before dequeuing. How sure are you that this invariant holds? Under what conditions do you expect it to fail? Clearly this is a fairly new project. 148 commits. No tests, benchmarks, or specifications. Keep at it. :)
- tron1k 10y ago
  Thanks for the feedback and support! We have added unit tests this morning and will continue to add more. Also, we have been using Artillery.io for load testing to measure the performance gains from multiple Node workers and the linked-list messaging queue.
- mxvzr 10y ago
  I too think this is a bit premature, especially when I see placeholder code like this [1]. I'd also mention the redis dependency somewhere, IMO I shouldn't have to go all the way here [2] to find about it. Somewhat less important feedback: - I want my libraries to be quiet; currently there is a lot going out to stdout. I see you already depend on debug, why not use it some more? (somewhat related: are the emojis really necessary?) - IMO libraries shouldn't listen for uncaughtException. Libraries don't have enough context, or even the guarantee these errors belong to said library. - require all your deps at the top; I'd like to pay that cost upfront when I start the process rather than after calling a function, potentially in the middle of doing something. - only use backticks if you are actually doing variable interpolation. [1] https://github.com/team-emt/razorframe/blob/master/lib/Razorframe.js#L224-L238 https://github.com/team-emt/razorframe/blob/master/lib/Razor... [2] https://github.com/team-emt/razorframe/blob/master/lib/Razorframe.js#L8-L10 https://github.com/team-emt/razorframe/blob/master/lib/Razor...
finchisko 10y ago
Nice but why not Primus instead socket.io?
ameesdotme 10y ago
Really interesting. I still have some doubts on Socket.io on large scale though!
lossolo 10y ago
"real-time apps at scale" and socket.io ? I don't think so. If you want to compare socket.io with something that works on scale look at uWebSockets[1] which is around 50 times faster and lightweight than socket.io is. https://github.com/uWebSockets/uWebSockets https://github.com/uWebSockets/uWebSockets
- tomcam 10y ago
  Whoa--was unaware of this gem. Thx
sapeien 10y ago
Why should anyone use this, instead of just using a Redis client, the `cluster` module, and a WebSocket server directly? It's only 251 lines of code as of writing, which is partly spent on trivia such as a linked list implementation, a string hashing function, and some console messages. There is a whopping total of 23 lines of test code for the linked list implementation. Also, did you really hardcode credentials to some hosted Redis instance somewhere? https://github.com/team-emt/razorframe/blob/e35004f7f2915275f81d6133455fc95293c88665/lib/Razorframe.js#L10 https://github.com/team-emt/razorframe/blob/e35004f7f2915275... (I'd rewrite git history if I were you)
- sotkrab 10y ago
  Hey this is Michael, one of the co-creators of Razorframe. We're currently hosting a redis server for devs to use during the prototyping phase. Those who are moving into production will need to host their own, but we're happy to share with the community. We're adding information to our documentation to this effect. Great catch and thanks for reviewing our code!
  - mmaunder 10y ago
    Ahem. https://github.com/team-emt/rz_demo/blob/master/server/database.js https://github.com/team-emt/rz_demo/blob/master/server/datab...
    - mmaunder 10y ago
      Yeah. https://github.com/team-emt/rz_demo/commit/59ee149a50f25d13fe3414bb2818b06e286da634 https://github.com/team-emt/rz_demo/commit/59ee149a50f25d13f... Rewrite history, as @sapeien suggested.

52 points by edshpark 10y ago

Razorframe is a Node.js library for enabling the effective backend scaling of your real-time application. Built on top of Socket.io, Razorframe’s simple server-and client-side API allows you to easily employ the task processing of multiple, parallel Node instances in conjunction with an in-memory queue. Our team set out to deliver on pushing the impact of real-time web through maximizing backend resources provided by multi-core server systems. By allowing your Node.js server to run on multiple threads and ensuring data consistency through an evented queueing system, Razorframe makes your application’s backend more resilient and performant under duress. We'd love to get some feedback on Razorframe and ways in which we can make it a better solution for your next Node.js project. Feel free to check us out on GitHub! Thank you!

songzme 10y ago
Would this be a firebase competitor? If so, how does it compare?
- tron1k 10y ago
  Hey, it's Travis, one of the co-creators of Razorframe... Firebase is a paid platform-as-a-service that includes realtime database hosting (what it's primarily known for) as well as numerous other services such as app analytics and hosting. It's a great platform and we definitely drew inspiration from them. Razorframe, on the other hand, is an open-source project that addresses 2 of the main trends we see in modern web applications: real-time client UIs and flexible server-side scaling. We built a Node module that you can bring into your server-side code to quickly get up and running with websockets (via socket.io) and Node clusters in order to accomplish both. Check us out on GitHub and let us know what you think:)

doublerebel 10y ago

What happens when one or many Node processes die with items left in the queue? Related, how are graceful restarts handled? (These are issues solved by existing message queues.)

tron1k 10y ago
Nothing happens to items in the queue in such a case. New workers will automatically respawn, and the adapter that links all the workers on the back-end will re-attach to newly spawned workers. Clients formerly connected to workers that are now gone will have to re-attach to new workers by socket.io.

ianstormtaylor 10y ago

Congrats on the launch! Glad to see more people working on these kinds of problems. As someone who hasn't built one of these systems before... What does Razorfish give you that running Socket.io as a separate microservice that talks to your existing API service over an existing message queue doesn't? Is it for people with architectures that don't have an existing queuing or API services already? Or does it handle sharding the Socket.io part easily which usually has to be vertically scaled? (Sorry if these questions don't make sense, I haven't dug into my requirements for these things yet but I know they're coming up!) I think a very clear explanation of that would help me (and maybe others) understand its value and how hard it would be to roll your own here.

edshpark 10y ago
Hey Ian - this is Edward, one of the co-creators of Razorframe. Thanks for your thought provoking question! Developers can certainly utilize the architecture you've described to great effect, but our solution aims to deliver a lightweight option that integrates technologies that don't normally play nice (Socket.io and clustered Node instances) but can have awesome scaling potential when they do. By keeping all the features within a familiar environment in Node, we've experienced that development time was vastly reduced in a potential project that expected a high volume of Websocket communications. For example, the incorporation of an in-memory queue reduces the need for developers to incorporate an external piece of tech (and any new language adoption) into their Node.js projects

agentultra 10y ago

> Those interactions are held in the queue only as long as the server needs before dequeuing. How sure are you that this invariant holds? Under what conditions do you expect it to fail? Clearly this is a fairly new project. 148 commits. No tests, benchmarks, or specifications. Keep at it. :)

tron1k 10y ago
Thanks for the feedback and support! We have added unit tests this morning and will continue to add more. Also, we have been using Artillery.io for load testing to measure the performance gains from multiple Node workers and the linked-list messaging queue.
mxvzr 10y ago
I too think this is a bit premature, especially when I see placeholder code like this [1]. I'd also mention the redis dependency somewhere, IMO I shouldn't have to go all the way here [2] to find about it. Somewhat less important feedback: - I want my libraries to be quiet; currently there is a lot going out to stdout. I see you already depend on debug, why not use it some more? (somewhat related: are the emojis really necessary?) - IMO libraries shouldn't listen for uncaughtException. Libraries don't have enough context, or even the guarantee these errors belong to said library. - require all your deps at the top; I'd like to pay that cost upfront when I start the process rather than after calling a function, potentially in the middle of doing something. - only use backticks if you are actually doing variable interpolation. [1] https://github.com/team-emt/razorframe/blob/master/lib/Razorframe.js#L224-L238 https://github.com/team-emt/razorframe/blob/master/lib/Razor... [2] https://github.com/team-emt/razorframe/blob/master/lib/Razorframe.js#L8-L10 https://github.com/team-emt/razorframe/blob/master/lib/Razor...

finchisko 10y ago

Nice but why not Primus instead socket.io?

ameesdotme 10y ago

Really interesting. I still have some doubts on Socket.io on large scale though!

lossolo 10y ago

"real-time apps at scale" and socket.io ? I don't think so. If you want to compare socket.io with something that works on scale look at uWebSockets[1] which is around 50 times faster and lightweight than socket.io is. https://github.com/uWebSockets/uWebSockets https://github.com/uWebSockets/uWebSockets

tomcam 10y ago
Whoa--was unaware of this gem. Thx

sapeien 10y ago

Why should anyone use this, instead of just using a Redis client, the `cluster` module, and a WebSocket server directly? It's only 251 lines of code as of writing, which is partly spent on trivia such as a linked list implementation, a string hashing function, and some console messages. There is a whopping total of 23 lines of test code for the linked list implementation. Also, did you really hardcode credentials to some hosted Redis instance somewhere? https://github.com/team-emt/razorframe/blob/e35004f7f2915275f81d6133455fc95293c88665/lib/Razorframe.js#L10 https://github.com/team-emt/razorframe/blob/e35004f7f2915275... (I'd rewrite git history if I were you)

sotkrab 10y ago
Hey this is Michael, one of the co-creators of Razorframe. We're currently hosting a redis server for devs to use during the prototyping phase. Those who are moving into production will need to host their own, but we're happy to share with the community. We're adding information to our documentation to this effect. Great catch and thanks for reviewing our code!
- mmaunder 10y ago
  Ahem. https://github.com/team-emt/rz_demo/blob/master/server/database.js https://github.com/team-emt/rz_demo/blob/master/server/datab...
  - mmaunder 10y ago
    Yeah. https://github.com/team-emt/rz_demo/commit/59ee149a50f25d13fe3414bb2818b06e286da634 https://github.com/team-emt/rz_demo/commit/59ee149a50f25d13f... Rewrite history, as @sapeien suggested.