9 ms·

Redbird: A modern reverse proxy for Node

136 points by Immortalin 8y ago

shard972 8y ago
Was recently looking for a reverse proxy for a project at work that utilizes multiple webpack repos that eventually gets built into one big webapp. Without a proxy your forced to only work on one area of the app at once so hopefully something like this will let me run all webpack instances under the same endpoint during dev.
- sbr464 8y ago
  https://github.com/webpack/webpack-dev-server https://github.com/webpack/webpack-dev-server Would this work? Or what’s the issue you are referring to?
  - skrebbel 8y ago
    I'm pretty sure he's aware of Webpack's devserver. A common problem is multiple libraries (with different webpack configs) being built that are required inside a single app. I guess GP's approach is to run multiple devservers, all on a different port, and then put a reverse proxy in front of it. Last I checked, webpack does support building multiple scripts (by turning the `entry` config setting into an array or object), but they'll all have the same settings (so that intermediate results used by multiple entry points only need to be built once). If webpack since added support for multiple entire configurations behind a single webpack dev server then that would be splendid of course, but last I checked they didn't. We have a similar problem at my company that we build our frontend React code for the browser and for node, and the latter has subtly different build settings. Our solution is to run the webpack devserver for the browser, a webpack watch mode for the nodejs version, and then we run the nodejs version using nodemon so that it reloads whenever the webpack watch mode rebuilds. The nodejs version contains an expressjs reverse proxy to forward requests for things in the `assets` directory to the webpack devserver. It's messy, but it works. My biggest beef is that it's rather dissimilar from our production setup. Of course if we'd done node+browser from the start we'd have written our code such that the node version could be run in node without a compilation step, but we were first frontend-only and then added server rendering, and by then the code already made use of a number of webpack-isms (such as weird require("raw!css!stylus!myfile.styl") type of stuff).
    - phpnode 8y ago
      Webpack actually supports exporting an array of configurations. https://github.com/webpack/webpack-dev-server/tree/master/examples/general/config-array https://github.com/webpack/webpack-dev-server/tree/master/ex...
- mbell 8y ago
  I use hoxy[1] for a lot of similar use cases, e.g. integration testing third party code. [1] https://github.com/greim/hoxy https://github.com/greim/hoxy
deleted 8y ago
[deleted]
jaimehrubiks 8y ago
What are some good practices to install this as a service (to replace apache, for example). Do you just setup a systemd script or you have any common utility for this? I read about pm2 being a common choice. Do you recommend it? Is there any alternative for multiple app types apart from node?
- onion2k 8y ago
  I use PM2. It's very good, and it keeps things in the node ecosystem.
  - dzek69 8y ago
    Pm2 isn't the best tool we could have, but it's probably the best one existing. To all future users: don't call `pm2 start ...` too often one by one (like in .sh script) - it won't start all of these, just some of them (at least on raspberry, maybe faster cpu helps here), use "ecosystem" feature for starting multiple services
viraptor 8y ago
I think the readme is missing a "why not" section. For example: why not use nginx instead, which more people have experience with.
- Ayesh 8y ago
  "Because they can" is often the most fitting answer. Even a tiny feature is worthy to start a totally new project with a fancy name.
  - viraptor 8y ago
    To be precise, I meant "why not use ..." comparison. I don't mind why the project was started. Whatever the reason, well done for getting it to this stage! But the other side - why should we be interested - is interesting :-)
    - pvorb 8y ago
      One reason could be that it might be easier for node.js devs to read and understand its source code. I don't use the software nor have read its source code, though.
    - de_watcher 8y ago
      Sometimes I think that there are not enough of real problems to solve in the js world. Too many people pumped into a tight space of the frontend development are creating in there another copy of the whole infrastructure while limited by the tools that weren't designed for that.
      - ricardobeat 8y ago
        Node was designed to build networking applications just like this.
        hamandcheese 8y ago
        No, node was the realization that JS’s concurrency model could be applied to applications like this.
sbmthakur 8y ago
From the roadmap section: Automatic routing via Redis Could anyone explain to me how does this work and what is automatic routing in this context?
- CubicsRube 8y ago
  I assume that instead of config file (or hardcoded values), they will use Redis store for mapping of the domains/target nodes. E.g. traffic from www.hacker.com is forwarded to 192.168.1.1, and traffic from www.news.com is forwarded to 192.168.1.2.
skrebbel 8y ago
This looks excellent. Is it intended to be integrated with other Node libs? I.e. can I use it as an express middleware somehow, or is it explicitly intended as a standalone thing - more like Nginx-but-configured-with-JS-instead-of-a-weird-custom-language? I can see value in the latter too, but I'm curious about the motivation here.
- EtienneK 8y ago
  Good question. I was also looking for an answer to this. Would love to add some middleware to turn this into an OAuth2 proxy.
  - chemicalnovae 8y ago
    I had a similiar thought, although not OAuth2, of integration with something like authelia[1]. [1] https://github.com/clems4ever/authelia https://github.com/clems4ever/authelia
stonewhite 8y ago
This would be more useful for my team if it would also transpile into nginx configuration.
dzek69 8y ago
I used to use Redbird, but configuration is a bit weird and not very well described if you need to use Auto HTTPS feature. Project seems rather dead and bugs are solved very slow (upgrading to newer node version caused it to not install because of outdated dependency, iirc PR with solution were ready, but not merged for weeks (?)). Finally I got rid of it and replaced it with Caddy webserver
KeitIG 8y ago
I am one of those JS guys who like to put JS/Node.js everywhere, but I do not get the problem Redbird is trying to solve: the Express.js doc is quite clear that for serious things, you should use a dedicated http server. [1] If you just want reverse-proxying, you can choose between the simplicity of Caddy or the power of Nginx (or Apache). Why would I want to run a JS app (that I know will be less performant) to do that? [1] https://expressjs.com/en/advanced/best-practice-performance.html
- TomMarius 8y ago
  There are cases when performance is not your main focus but simplicity of use is. Like during development. I'd of course never use this in production, but it's perfect for my development needs.
  - protonfish 8y ago
    I agree with the sentiment, but using nginx as a proxy is reasonably easy (at least after you make one that works and can reuse that configuration.) The advantage of developing on as similar a platform to the projected production platform is that when you do deploy to a real environment, there are fewer nasty surprises.
    - TomMarius 8y ago
      It is not reasonably easy, I can't set it up completely with "git clone" and it does not stay inside the project (that's an issue when your developers use a huge variety of operating systems with varying installed and configured software). That means a lot of time lost during problem solving with the hundreds of developers I work with, most of them have never even heard about nginx, they're developers, not sysadmins. There absolutely should not be any nasty surprises with switching a reverse proxy, and you can cover that with end to end tests. Such issue would be a bug in the reverse proxy. Of course there are things like Docker, but that implies performance issues on Mac.
    - JPKab 8y ago
      How easy is it to use nginx for dynamic proxying? Wondering, because the need I have is to be able to dynamically route requests to ephemeral flask apps that could be on a different host/port at any given moment. Not my design BTW, but it's what I need to do. The thought was to have node manage the flask processes and another tracker service which would dynamically proxy to the apps. Can nginx provide this? Forgive my ignorance, just new to the concept of dynamic proxying.
Aeolun 8y ago
0.2.3 released 4 years ago. I don’t know, a last release that long ago doesn’t inspire faith in the health of the project.
- jeremyjh 8y ago
  The last release was 0.8.0, about a month ago.
  - raprp 8y ago
    The releases on github are outdated. https://github.com/OptimalBits/redbird/releases https://github.com/OptimalBits/redbird/releases
- JosephRedfern 8y ago
  It doesn't look like they're using GitHub releases. npm shows some much more recent versions: https://www.npmjs.com/package/redbird https://www.npmjs.com/package/redbird.
d--b 8y ago
Sorry, but what’s a reverse proxy?
- martin-adams 8y ago
  It's where the server has a proxy that can fetch resources from different places. E.g. ServerA (proxy) get a request for /something, but then fetches it from ServerB (web server) and returns it to the client. It proxies the response to the client. The comparison is a forward proxy, which the client has, and a reverse proxy is where it's on the server side.
  - tyingq 8y ago
    Reverse proxies also often provide optional benefits like... Load balancing, not having to run the webserver as root to bind to 80/443, caching responses, buffering, rate limiting, injecting headers, serving static assets, etc.
  - d--b 8y ago
    Got it. I'm actually doing it using nginx to forward to node... Didn't know that's how it was called...
johnpython 8y ago
Yet another JavaScript project reinventing the wheel and badly at that. It seems the NodeJS community suffers from NIH syndrome more than others. This is a sign of lack of seniors and leadership.
uptime 8y ago
I’m a fan of traefik. https://traefik.io https://traefik.io I’m not ready for full k8s, but traefik gives me ACME and detects my docker containers and routes them for me. If I only need https redir I’ll use that or nginx. I can Redbird being useful where you only have node as an option.
blorenz 8y ago
I like the simplicity of this but am weary of using it in production based on other commenters. I would find something like this very useful if it would generate nginx configs instead of trying to function as the reverse proxy.
davnicwil 8y ago
I mostly agree with the sentiment in other comments of just using nginx, but there is one standout feature here that looks incredibly useful: > We have now support for automatic generation of SSL certificates using LetsEncrypt. Zero config setup for your TLS protected services that just works. I've scripted this kind of letsencrypt certs automation before with certbot and nginx, which is fine but a 'just works' declarative plugin for nginx would be much nicer. Does anything like this exist? Anyone have experience with using it? Asking here as googling brings up a bunch of outdated forum threads etc to wade through.
- bschwindHN 8y ago
  It's not nginx but Caddy had stupid easy SSL automation through LetsEncrypt.
- jmngomes 8y ago
  You can use acme.sh to automate certificate issuing, installation, verification, renewal and some server configuration: https://github.com/Neilpang/acme.sh https://github.com/Neilpang/acme.sh
rdsubhas 8y ago
Quite nice. node-http-proxy is awesome, lacks a few features (but has a few more such as websockets). Redbird has http2 as well. But to be frank, we have used node-http-proxy and other stuff before. It doesn't end with development. People put this to reverse proxy API calls in production, and then the other stuff that this calls into - it also uses proxying to call more stuff. In general, if you find yourself using this as a "convenient, declarative way to route" in production, you _might potentially_ be digging yourself a hole. The lines between "harmless reverse proxying" and "service routing and discovery" can become thin.
novaleaf 8y ago
Regarding proxies, could anyone recommend where to learn about implementing a proxy / proxy chain? a book, or online resource, etc.
api 8y ago
This makes a very nice letsencrypt proxy.
gigel82 8y ago
I'm surprised no one mentioned the threading situation in the comments. The main reason why you need a reverse proxy (like nginx) on top of your express app is because node.js is single-threaded. Yes, you can fork (which means a new process and IPC), but that's even more of a performance hog. If you have to serve a large static file - guess what... no other request gets processed at the same time. It's a neat idea but until we get true multi-threading in Node.js, I'll stick with my nginx, thank you very much.
- erikpukinskis 8y ago
  Wait, what? Node is very happy to asyncronously stream chunks of data over HTTP. You could write a blocking server by collecting the whole response in memory and sending it out all at once, but that’s not required by Node. This is literally the reason Node even exists, nonblocking I/O with a simple threading model. You can have a million open connections all asynchronously nibbling data as the buffers empty.
erikpukinskis 8y ago
What does “modern” mean in this context? ES6?

136 points by Immortalin 8y ago

Was recently looking for a reverse proxy for a project at work that utilizes multiple webpack repos that eventually gets built into one big webapp. Without a proxy your forced to only work on one area of the app at once so hopefully something like this will let me run all webpack instances under the same endpoint during dev.

sbr464 8y ago
https://github.com/webpack/webpack-dev-server https://github.com/webpack/webpack-dev-server Would this work? Or what’s the issue you are referring to?
- skrebbel 8y ago
  I'm pretty sure he's aware of Webpack's devserver. A common problem is multiple libraries (with different webpack configs) being built that are required inside a single app. I guess GP's approach is to run multiple devservers, all on a different port, and then put a reverse proxy in front of it. Last I checked, webpack does support building multiple scripts (by turning the `entry` config setting into an array or object), but they'll all have the same settings (so that intermediate results used by multiple entry points only need to be built once). If webpack since added support for multiple entire configurations behind a single webpack dev server then that would be splendid of course, but last I checked they didn't. We have a similar problem at my company that we build our frontend React code for the browser and for node, and the latter has subtly different build settings. Our solution is to run the webpack devserver for the browser, a webpack watch mode for the nodejs version, and then we run the nodejs version using nodemon so that it reloads whenever the webpack watch mode rebuilds. The nodejs version contains an expressjs reverse proxy to forward requests for things in the `assets` directory to the webpack devserver. It's messy, but it works. My biggest beef is that it's rather dissimilar from our production setup. Of course if we'd done node+browser from the start we'd have written our code such that the node version could be run in node without a compilation step, but we were first frontend-only and then added server rendering, and by then the code already made use of a number of webpack-isms (such as weird require("raw!css!stylus!myfile.styl") type of stuff).
  - phpnode 8y ago
    Webpack actually supports exporting an array of configurations. https://github.com/webpack/webpack-dev-server/tree/master/examples/general/config-array https://github.com/webpack/webpack-dev-server/tree/master/ex...
mbell 8y ago
I use hoxy[1] for a lot of similar use cases, e.g. integration testing third party code. [1] https://github.com/greim/hoxy https://github.com/greim/hoxy

deleted 8y ago

[deleted]

jaimehrubiks 8y ago

What are some good practices to install this as a service (to replace apache, for example). Do you just setup a systemd script or you have any common utility for this? I read about pm2 being a common choice. Do you recommend it? Is there any alternative for multiple app types apart from node?

onion2k 8y ago
I use PM2. It's very good, and it keeps things in the node ecosystem.
- dzek69 8y ago
  Pm2 isn't the best tool we could have, but it's probably the best one existing. To all future users: don't call `pm2 start ...` too often one by one (like in .sh script) - it won't start all of these, just some of them (at least on raspberry, maybe faster cpu helps here), use "ecosystem" feature for starting multiple services

viraptor 8y ago

I think the readme is missing a "why not" section. For example: why not use nginx instead, which more people have experience with.

Ayesh 8y ago
"Because they can" is often the most fitting answer. Even a tiny feature is worthy to start a totally new project with a fancy name.
- viraptor 8y ago
  To be precise, I meant "why not use ..." comparison. I don't mind why the project was started. Whatever the reason, well done for getting it to this stage! But the other side - why should we be interested - is interesting :-)
  - pvorb 8y ago
    One reason could be that it might be easier for node.js devs to read and understand its source code. I don't use the software nor have read its source code, though.
  - de_watcher 8y ago
    Sometimes I think that there are not enough of real problems to solve in the js world. Too many people pumped into a tight space of the frontend development are creating in there another copy of the whole infrastructure while limited by the tools that weren't designed for that.
    - ricardobeat 8y ago
      Node was designed to build networking applications just like this.
      - hamandcheese 8y ago
        No, node was the realization that JS’s concurrency model could be applied to applications like this.

sbmthakur 8y ago

From the roadmap section: Automatic routing via Redis Could anyone explain to me how does this work and what is automatic routing in this context?

CubicsRube 8y ago
I assume that instead of config file (or hardcoded values), they will use Redis store for mapping of the domains/target nodes. E.g. traffic from www.hacker.com is forwarded to 192.168.1.1, and traffic from www.news.com is forwarded to 192.168.1.2.

skrebbel 8y ago

This looks excellent. Is it intended to be integrated with other Node libs? I.e. can I use it as an express middleware somehow, or is it explicitly intended as a standalone thing - more like Nginx-but-configured-with-JS-instead-of-a-weird-custom-language? I can see value in the latter too, but I'm curious about the motivation here.

EtienneK 8y ago
Good question. I was also looking for an answer to this. Would love to add some middleware to turn this into an OAuth2 proxy.
- chemicalnovae 8y ago
  I had a similiar thought, although not OAuth2, of integration with something like authelia[1]. [1] https://github.com/clems4ever/authelia https://github.com/clems4ever/authelia

stonewhite 8y ago

This would be more useful for my team if it would also transpile into nginx configuration.

dzek69 8y ago

I used to use Redbird, but configuration is a bit weird and not very well described if you need to use Auto HTTPS feature. Project seems rather dead and bugs are solved very slow (upgrading to newer node version caused it to not install because of outdated dependency, iirc PR with solution were ready, but not merged for weeks (?)). Finally I got rid of it and replaced it with Caddy webserver

KeitIG 8y ago

I am one of those JS guys who like to put JS/Node.js everywhere, but I do not get the problem Redbird is trying to solve: the Express.js doc is quite clear that for serious things, you should use a dedicated http server. [1] If you just want reverse-proxying, you can choose between the simplicity of Caddy or the power of Nginx (or Apache). Why would I want to run a JS app (that I know will be less performant) to do that? [1] https://expressjs.com/en/advanced/best-practice-performance.html

TomMarius 8y ago
There are cases when performance is not your main focus but simplicity of use is. Like during development. I'd of course never use this in production, but it's perfect for my development needs.
- protonfish 8y ago
  I agree with the sentiment, but using nginx as a proxy is reasonably easy (at least after you make one that works and can reuse that configuration.) The advantage of developing on as similar a platform to the projected production platform is that when you do deploy to a real environment, there are fewer nasty surprises.
  - TomMarius 8y ago
    It is not reasonably easy, I can't set it up completely with "git clone" and it does not stay inside the project (that's an issue when your developers use a huge variety of operating systems with varying installed and configured software). That means a lot of time lost during problem solving with the hundreds of developers I work with, most of them have never even heard about nginx, they're developers, not sysadmins. There absolutely should not be any nasty surprises with switching a reverse proxy, and you can cover that with end to end tests. Such issue would be a bug in the reverse proxy. Of course there are things like Docker, but that implies performance issues on Mac.
  - JPKab 8y ago
    How easy is it to use nginx for dynamic proxying? Wondering, because the need I have is to be able to dynamically route requests to ephemeral flask apps that could be on a different host/port at any given moment. Not my design BTW, but it's what I need to do. The thought was to have node manage the flask processes and another tracker service which would dynamically proxy to the apps. Can nginx provide this? Forgive my ignorance, just new to the concept of dynamic proxying.

Aeolun 8y ago

0.2.3 released 4 years ago. I don’t know, a last release that long ago doesn’t inspire faith in the health of the project.

jeremyjh 8y ago
The last release was 0.8.0, about a month ago.
- raprp 8y ago
  The releases on github are outdated. https://github.com/OptimalBits/redbird/releases https://github.com/OptimalBits/redbird/releases
JosephRedfern 8y ago
It doesn't look like they're using GitHub releases. npm shows some much more recent versions: https://www.npmjs.com/package/redbird https://www.npmjs.com/package/redbird.

d--b 8y ago

Sorry, but what’s a reverse proxy?

martin-adams 8y ago
It's where the server has a proxy that can fetch resources from different places. E.g. ServerA (proxy) get a request for /something, but then fetches it from ServerB (web server) and returns it to the client. It proxies the response to the client. The comparison is a forward proxy, which the client has, and a reverse proxy is where it's on the server side.
- tyingq 8y ago
  Reverse proxies also often provide optional benefits like... Load balancing, not having to run the webserver as root to bind to 80/443, caching responses, buffering, rate limiting, injecting headers, serving static assets, etc.
- d--b 8y ago
  Got it. I'm actually doing it using nginx to forward to node... Didn't know that's how it was called...

johnpython 8y ago

Yet another JavaScript project reinventing the wheel and badly at that. It seems the NodeJS community suffers from NIH syndrome more than others. This is a sign of lack of seniors and leadership.

uptime 8y ago

I’m a fan of traefik. https://traefik.io https://traefik.io I’m not ready for full k8s, but traefik gives me ACME and detects my docker containers and routes them for me. If I only need https redir I’ll use that or nginx. I can Redbird being useful where you only have node as an option.

blorenz 8y ago

I like the simplicity of this but am weary of using it in production based on other commenters. I would find something like this very useful if it would generate nginx configs instead of trying to function as the reverse proxy.

davnicwil 8y ago

I mostly agree with the sentiment in other comments of just using nginx, but there is one standout feature here that looks incredibly useful: > We have now support for automatic generation of SSL certificates using LetsEncrypt. Zero config setup for your TLS protected services that just works. I've scripted this kind of letsencrypt certs automation before with certbot and nginx, which is fine but a 'just works' declarative plugin for nginx would be much nicer. Does anything like this exist? Anyone have experience with using it? Asking here as googling brings up a bunch of outdated forum threads etc to wade through.

bschwindHN 8y ago
It's not nginx but Caddy had stupid easy SSL automation through LetsEncrypt.
jmngomes 8y ago
You can use acme.sh to automate certificate issuing, installation, verification, renewal and some server configuration: https://github.com/Neilpang/acme.sh https://github.com/Neilpang/acme.sh

rdsubhas 8y ago

Quite nice. node-http-proxy is awesome, lacks a few features (but has a few more such as websockets). Redbird has http2 as well. But to be frank, we have used node-http-proxy and other stuff before. It doesn't end with development. People put this to reverse proxy API calls in production, and then the other stuff that this calls into - it also uses proxying to call more stuff. In general, if you find yourself using this as a "convenient, declarative way to route" in production, you _might potentially_ be digging yourself a hole. The lines between "harmless reverse proxying" and "service routing and discovery" can become thin.

novaleaf 8y ago

Regarding proxies, could anyone recommend where to learn about implementing a proxy / proxy chain? a book, or online resource, etc.

api 8y ago

This makes a very nice letsencrypt proxy.

gigel82 8y ago

I'm surprised no one mentioned the threading situation in the comments. The main reason why you need a reverse proxy (like nginx) on top of your express app is because node.js is single-threaded. Yes, you can fork (which means a new process and IPC), but that's even more of a performance hog. If you have to serve a large static file - guess what... no other request gets processed at the same time. It's a neat idea but until we get true multi-threading in Node.js, I'll stick with my nginx, thank you very much.

erikpukinskis 8y ago
Wait, what? Node is very happy to asyncronously stream chunks of data over HTTP. You could write a blocking server by collecting the whole response in memory and sending it out all at once, but that’s not required by Node. This is literally the reason Node even exists, nonblocking I/O with a simple threading model. You can have a million open connections all asynchronously nibbling data as the buffers empty.

erikpukinskis 8y ago

What does “modern” mean in this context? ES6?