4 ms·

Keeping node.js servers up forever

84 points by maraksquires 16y ago

deleted 16y ago
[deleted]
moonpolysoft 16y ago
Forever a node.
substack 16y ago
I like how there's a command line interface plus an application interface, so I can tie forever process management into my web interface and backend control interface. Right now the processes are just sitting in a `while true` bash loop.
- indexzero 16y ago
  Thanks. If you dig in a little deeper you can see I use optimist for the CLI parsing :) In the next version I'd like to expose a web service and also have hooks for doing things like sending emails when forever restarts a process.
todd3834 16y ago
I have been using supervisord to solve this problem. It doesn't care what language your app was written in and works very well.
dkasper 16y ago
Just deployed this now. Took all of roughly 10 seconds to setup and working beautifully. I've used supervisor in the past, and this was definitely easier.
gruseom 16y ago
By using the 'http' module we can run a stand-alone web server in node.js without the need for a separate server like Apache or nginx. Last I checked Ryan was saying this was a very bad idea. Has that changed? It's certainly enticing.
- indexzero 16y ago
  When did you last check with him? I've been running w/o a separate server (effectively) for ~3 months.
  - gruseom 16y ago
    Oh, I just meant last I checked on blogs and such (not with him personally). IIRC, his main reason for advising against it was that there were too many security holes. This was in a video talk from several months ago if not longer. I'm curious to know if conditions have changed.
intranation 16y ago
This reminds me of the bad old days of Rails, when every man and his dog had some random hacky and myopic solution to keep Rails and Mongrel running. Just say no to Node/Rails/whatever specific-stuff, kids--use system level tools for running processes. Edit: removed references to "proprietary", as indexzero is correct about meaning of that.
- indexzero 16y ago
  That's a strange usage for "proprietary" since it's all free and open source (MIT). Either way, the nodejs child_process module is really just a light weight wrapper to execvp(): http://linux.die.net/man/3/execvp http://linux.die.net/man/3/execvp I haven't dug through the source for tools like monit, daemontools, but I'm sure near the metal they're using POSIX or similar system APIs. EDIT: There's really no reason Forever has to be node specific, I actually say that in the article: "Honestly, it's a one line fix here (https://github.com/indexzero/forever/blob/master/lib/forever.js#L356 https://github.com/indexzero/forever/blob/master/lib/forever...), but I'm not sure if users want to put 'node' in-front of every command." I'll file your comment as a +1 for that feature ;)
viraptor 16y ago
I just have to wonder - after listing all those projects which already do this job and do it well... Why create / use `forever` which didn't get the same exposure to the production environments yet?
- maraksquires 16y ago
  Well, we can call native node.js code from our process monitor now. We can extend our process monitor with functionality that would be difficult to implement in those other tools.
  - intranation 16y ago
    Any examples of when this would be desirable? Seems to me that monitoring and running tools should just, you know, monitor and run things. Separation of concerns.
    - maraksquires 16y ago
      How would you setup alerts for your monitoring? I'd much rather have access to a well structured library (node.js) where sockets and http are first class citizens, opposed to being forced into a solution or having to write C or bash.
  - indexzero 16y ago
    @intranation Just to toss out a scenario: Lets say under high traffic your application hits an edge case and starts to crash very frequently. Suppose you want to receive an email, SMS, or IM when something like this happens as a devops person. Would you consider that a valid concern of process monitoring? Planned features to Forever include this from the command line, but if you use it from node directly one could implement that feature now: https://github.com/indexzero/forever https://github.com/indexzero/forever
    - tlrobinson 16y ago
      I want my monitoring process to be rock solid. Extensions such as notifications should be done in another process, otherwise you're at the mercy of whatever email/SMS/IM library you include to do the notification.
Skywing 16y ago
bookmarked. thank you.
jacquesm 16y ago
This leads to a very bad practice though, which is to cause bugs to be left laying around in production systems. You should really form the habit of analyzing such crashes, getting to the root cause of the problem and making sure that it never ever happens again. It's good to have a system like this in place, but it should be used as an insurance policy against unknown bugs, not as a way to work around known and reproducible ones.
revoltingx 16y ago
This is something I've come to take for granted using Erlang.
Kilimanjaro 16y ago
Off topic, but I like to read blogs on my ipad while in bed and I hate it when authors fuck up with the meta viewport tag. If I want to double tap to zoom in, because my eyes are not the same as 20 years ago, just let me do that, ok? Sounded snob? maybe. Sounded reasonable? hell yes. Btw, is there a bookmarklet to remove the meta viewport tag available somewhere?
- ionfish 16y ago
  Just for you: https://gist.github.com/725354 https://gist.github.com/725354
  - Kilimanjaro 16y ago
    I was thinking about something like: meta=document.querySelector('meta[name=viewport]') meta.parentNode.removeChild(meta) Thanks anyway for your quick response!
    - ionfish 16y ago
      Obviously that's fine for your use-case where the querySelector function is available, but it wouldn't work e.g. on Internet Explorer. My version is also more robust when dealing with markup errors, i.e. when there's more than one viewport meta element.
- maraksquires 16y ago
  I've opened up a ticket for our blog to look into this. Do you think you could elaborate more on what the exact problem is and what we can do to fix it? Please forgive my ignorance here.
jasonkester 16y ago
When I see a blog post like this, it serves more to raise doubts about Node.js than anything else. Before today, I had been considering Node for an upcoming product. Now I read that people are actually building and releasing software to work around the fact that Node.js servers regularly incapacitate themselves. Really? Like it just goes down and doesn't know how to cycle itself? Ouch. That translates to me as "Don't go anywhere near Node.js until they get it stable." So yeah, I'm sure this is a great way to shore up your system. Buy I'm going to think twice before investing time in a system that needs shoring up.
- jrockway 16y ago
  More of a safety mechanism than anything else. I use daemontools to run qmail, and qmail has never crashed. But if it did, I would still want to receive my email, even though it's "horrible" that qmail could crash. Sometimes the safest way to handle an error is to kill the process and start a new one. Starting recovery from a known-good state is better than starting from a known-bad one.
  - jasonkester 16y ago
    Seems reasonable enough. From the tone of the post, it sounded like crashing was a fairly common thing to happen with Node.js, and that it didn't cycle itself. Coming from the context of IIS/ASP.NET, which hasn't yet crashed on me (at least not without cycling itself harmlessly) in the 10 years I've been running sites on it, the possibility that you'd need to worry about such things seemed a bit novel. So basically what you're saying is that it just follows the Unix philosophy and doesn't run its own daemon to cycle it if it falls down. That doesn't sound anywhere near as unreasonable.
    - silentbicycle 16y ago
      Erlang has heart (http://www.erlang.org/doc/man/heart.html http://www.erlang.org/doc/man/heart.html) for similar reasons. Of course, it also has distribution, for when the whole computer it's running on dies. I use runit (http://smarden.org/runit/ http://smarden.org/runit/) instead of daemontools (same sort of thing, but a bit less opinionated about e.g. where it gets installed). Rather than expecting every daemon to implement its own supervisor, logging system, etc. correctly, just use one of those.
zbanks 16y ago
Oddly enough, the site is down.
jimmyjazz14 16y ago
daemontools is a beautiful thing, personally I think its a much better idea to have a general purpose tool for this kind of thing. I'm not sure why everyone feels the need to reinvent the wheel when it comes to managing background services.
waratuman 16y ago
Use Upstart.
richcollins 16y ago
How is this directly related to node and not servers in general?
- iclelland 16y ago
  The application described in the article is specifically written to interface with Node.js servers. It keeps those processes running, and restarts them as needed. It is not a general process monitor.
mkrecny 16y ago
What's wrong with good ol' inittab?

84 points by maraksquires 16y ago

deleted 16y ago

[deleted]

moonpolysoft 16y ago

Forever a node.

substack 16y ago

I like how there's a command line interface plus an application interface, so I can tie forever process management into my web interface and backend control interface. Right now the processes are just sitting in a `while true` bash loop.

indexzero 16y ago
Thanks. If you dig in a little deeper you can see I use optimist for the CLI parsing :) In the next version I'd like to expose a web service and also have hooks for doing things like sending emails when forever restarts a process.

todd3834 16y ago

I have been using supervisord to solve this problem. It doesn't care what language your app was written in and works very well.

dkasper 16y ago

Just deployed this now. Took all of roughly 10 seconds to setup and working beautifully. I've used supervisor in the past, and this was definitely easier.

gruseom 16y ago

By using the 'http' module we can run a stand-alone web server in node.js without the need for a separate server like Apache or nginx. Last I checked Ryan was saying this was a very bad idea. Has that changed? It's certainly enticing.

indexzero 16y ago
When did you last check with him? I've been running w/o a separate server (effectively) for ~3 months.
- gruseom 16y ago
  Oh, I just meant last I checked on blogs and such (not with him personally). IIRC, his main reason for advising against it was that there were too many security holes. This was in a video talk from several months ago if not longer. I'm curious to know if conditions have changed.

intranation 16y ago

This reminds me of the bad old days of Rails, when every man and his dog had some random hacky and myopic solution to keep Rails and Mongrel running. Just say no to Node/Rails/whatever specific-stuff, kids--use system level tools for running processes. Edit: removed references to "proprietary", as indexzero is correct about meaning of that.

indexzero 16y ago
That's a strange usage for "proprietary" since it's all free and open source (MIT). Either way, the nodejs child_process module is really just a light weight wrapper to execvp(): http://linux.die.net/man/3/execvp http://linux.die.net/man/3/execvp I haven't dug through the source for tools like monit, daemontools, but I'm sure near the metal they're using POSIX or similar system APIs. EDIT: There's really no reason Forever has to be node specific, I actually say that in the article: "Honestly, it's a one line fix here (https://github.com/indexzero/forever/blob/master/lib/forever.js#L356 https://github.com/indexzero/forever/blob/master/lib/forever...), but I'm not sure if users want to put 'node' in-front of every command." I'll file your comment as a +1 for that feature ;)

viraptor 16y ago

I just have to wonder - after listing all those projects which already do this job and do it well... Why create / use `forever` which didn't get the same exposure to the production environments yet?

maraksquires 16y ago
Well, we can call native node.js code from our process monitor now. We can extend our process monitor with functionality that would be difficult to implement in those other tools.
- intranation 16y ago
  Any examples of when this would be desirable? Seems to me that monitoring and running tools should just, you know, monitor and run things. Separation of concerns.
  - maraksquires 16y ago
    How would you setup alerts for your monitoring? I'd much rather have access to a well structured library (node.js) where sockets and http are first class citizens, opposed to being forced into a solution or having to write C or bash.
- indexzero 16y ago
  @intranation Just to toss out a scenario: Lets say under high traffic your application hits an edge case and starts to crash very frequently. Suppose you want to receive an email, SMS, or IM when something like this happens as a devops person. Would you consider that a valid concern of process monitoring? Planned features to Forever include this from the command line, but if you use it from node directly one could implement that feature now: https://github.com/indexzero/forever https://github.com/indexzero/forever
  - tlrobinson 16y ago
    I want my monitoring process to be rock solid. Extensions such as notifications should be done in another process, otherwise you're at the mercy of whatever email/SMS/IM library you include to do the notification.

Skywing 16y ago

bookmarked. thank you.

jacquesm 16y ago

This leads to a very bad practice though, which is to cause bugs to be left laying around in production systems. You should really form the habit of analyzing such crashes, getting to the root cause of the problem and making sure that it never ever happens again. It's good to have a system like this in place, but it should be used as an insurance policy against unknown bugs, not as a way to work around known and reproducible ones.

revoltingx 16y ago

This is something I've come to take for granted using Erlang.

Kilimanjaro 16y ago

Off topic, but I like to read blogs on my ipad while in bed and I hate it when authors fuck up with the meta viewport tag. If I want to double tap to zoom in, because my eyes are not the same as 20 years ago, just let me do that, ok? Sounded snob? maybe. Sounded reasonable? hell yes. Btw, is there a bookmarklet to remove the meta viewport tag available somewhere?

ionfish 16y ago
Just for you: https://gist.github.com/725354 https://gist.github.com/725354
- Kilimanjaro 16y ago
  I was thinking about something like: meta=document.querySelector('meta[name=viewport]') meta.parentNode.removeChild(meta) Thanks anyway for your quick response!
  - ionfish 16y ago
    Obviously that's fine for your use-case where the querySelector function is available, but it wouldn't work e.g. on Internet Explorer. My version is also more robust when dealing with markup errors, i.e. when there's more than one viewport meta element.
maraksquires 16y ago
I've opened up a ticket for our blog to look into this. Do you think you could elaborate more on what the exact problem is and what we can do to fix it? Please forgive my ignorance here.

jasonkester 16y ago

When I see a blog post like this, it serves more to raise doubts about Node.js than anything else. Before today, I had been considering Node for an upcoming product. Now I read that people are actually building and releasing software to work around the fact that Node.js servers regularly incapacitate themselves. Really? Like it just goes down and doesn't know how to cycle itself? Ouch. That translates to me as "Don't go anywhere near Node.js until they get it stable." So yeah, I'm sure this is a great way to shore up your system. Buy I'm going to think twice before investing time in a system that needs shoring up.

jrockway 16y ago
More of a safety mechanism than anything else. I use daemontools to run qmail, and qmail has never crashed. But if it did, I would still want to receive my email, even though it's "horrible" that qmail could crash. Sometimes the safest way to handle an error is to kill the process and start a new one. Starting recovery from a known-good state is better than starting from a known-bad one.
- jasonkester 16y ago
  Seems reasonable enough. From the tone of the post, it sounded like crashing was a fairly common thing to happen with Node.js, and that it didn't cycle itself. Coming from the context of IIS/ASP.NET, which hasn't yet crashed on me (at least not without cycling itself harmlessly) in the 10 years I've been running sites on it, the possibility that you'd need to worry about such things seemed a bit novel. So basically what you're saying is that it just follows the Unix philosophy and doesn't run its own daemon to cycle it if it falls down. That doesn't sound anywhere near as unreasonable.
  - silentbicycle 16y ago
    Erlang has heart (http://www.erlang.org/doc/man/heart.html http://www.erlang.org/doc/man/heart.html) for similar reasons. Of course, it also has distribution, for when the whole computer it's running on dies. I use runit (http://smarden.org/runit/ http://smarden.org/runit/) instead of daemontools (same sort of thing, but a bit less opinionated about e.g. where it gets installed). Rather than expecting every daemon to implement its own supervisor, logging system, etc. correctly, just use one of those.

zbanks 16y ago

Oddly enough, the site is down.

jimmyjazz14 16y ago

daemontools is a beautiful thing, personally I think its a much better idea to have a general purpose tool for this kind of thing. I'm not sure why everyone feels the need to reinvent the wheel when it comes to managing background services.

waratuman 16y ago

Use Upstart.

richcollins 16y ago

How is this directly related to node and not servers in general?

iclelland 16y ago
The application described in the article is specifically written to interface with Node.js servers. It keeps those processes running, and restarts them as needed. It is not a general process monitor.

mkrecny 16y ago

What's wrong with good ol' inittab?