6 ms·

Show HN: 100 LOC Ruby forward proxy using only standard libraries

115 points by moriarty_ 6y ago

mfkp 6y ago
I had to Google what a forward proxy was. Apparently it's just a regular proxy server? Never heard it called that before, might be helpful to add that to the readme. https://smartproxy.com/blog/the-difference-between-a-reverse-proxy-and-a-forward-proxy https://smartproxy.com/blog/the-difference-between-a-reverse...
- redis_mlc 6y ago
  A reverse proxy waits for a request from another server and responds to the calling server. You can think of a reverse proxy as the "deep backend" in a web stack. One of the first and most powerful reverse proxies was mod_perl running under httpd. Typically 5 or 6 mod_perl instances can handle requests from 300 normal httpd children. So when somebody says "forward proxy" they usually mean "not a reverse proxy."
  - tyingq 6y ago
    Mod_perl doesn't feel like a reverse proxy to me, it's just executing Perl in the Apache process directly. Fastcgi, on the other hand, is passing requests to persistently running standalone Perl instances, over a pipe, Unix domain socket, or tcp connection. Which feels more like a reverse proxy.
    - mapgrep 6y ago
      Mod_perl would more accurately be called a “commonly reverse proxied app server.” Typical setup was to have another Apache instance in front of it handling static requests and reverse proxying app requests.
    - redis_mlc 6y ago
      Nobody runs mod_perl in production except as a reverse proxy because of the RAM consumption and related issues.
- inopinatus 6y ago
  There are also up, down, charmed, and strange proxies, with obvious applications in quantum computing, nonrepudiation, and dairy farming. Strange proxies were thought to be purely theoretical until an accident involving a rubber band, a liquid lunch, and a particle accelerator collided a transparent squid with varnish at relativistic velocities and the resulting core dump subsequently examined for overflows.
- tmpxgdqrcKFuG 6y ago
  I think preprending “forward” was meant to avoid confusion. Reverse proxy and Forward proxy, but now you’re wondering if there’s 3 types of proxies.
- wg0 6y ago
  I also landed on similar pages and now I am confused about SOCK5 proxy vs HTTP proxy vs SSL proxy as outlines in the links above.
- cortesoft 6y ago
  I think people add 'forward' because reverse proxies are now a lot more common than traditional proxies, so people will assume reverse proxy if you just say 'proxy'.
tutfbhuf 6y ago
I think this is just a tech demonstration, but I wonder how much performance this proxy has in a benchmark against some similar small (in terms of LOC) C/Rust proxy. BTW, Github counts 149 SLOC just for this file: https://github.com/jamesmoriarty/forward-proxy/blob/769b6424b947341c5d6bdfc70ffbdf393296ab0d/lib/forward_proxy/server.rb https://github.com/jamesmoriarty/forward-proxy/blob/769b6424...
- moriarty_ 6y ago
  The thread pool added 35 LOC but can be replaced with 1 LOC `Thread.new`.
hlasdjlfhalwjk 6y ago
A little bit more than 100 LoC (`cloc lib bin` gives me 147 lines of Ruby) but still nice work.
- zaltekk 6y ago
  That should be `cloc exe lib`. `bin` has distribution-related auto generated scripts. The actual entry point script is in `lib`. From the sloc count on GitHub just the server file alone is listed as 149. This also uses WEBrick to do the HTTP parsing. The webrick gem is no longer bundled with Ruby as of 3.0.0.
anonfunction 6y ago
Here's a Go version in 41 lines of source code I wrote in 2016: https://github.com/gophergala2016/goxy/blob/master/goxy.go https://github.com/gophergala2016/goxy/blob/master/goxy.go And a forward proxy that supports middleware like adding CORS headers, logging, etc... https://github.com/montanaflynn/roxy https://github.com/montanaflynn/roxy
phuesler 6y ago
Nice demo. Useful for small projects and teaching. I don't expect the performance to be great, considering it is opening up a new connection to the forwarded host for every request. https://github.com/jamesmoriarty/forward-proxy/blob/bc7d9ec121e52fdbcbc4e70184d4c74581873327/lib/forward_proxy/server.rb#L128 https://github.com/jamesmoriarty/forward-proxy/blob/bc7d9ec1...
alexrsagen 6y ago
I made a similar proxy using Go in 2019: https://github.com/alexrsagen/alexproxy https://github.com/alexrsagen/alexproxy It's got a couple more features than OP, check out the README.
- panpanna 6y ago
  I think yours does a lot more than OPs. Go was made for this kind of problems. It is very easy to write efficient networking code on go that can be read and understood by mere mortals.
studius 6y ago
It looks like it forwards TCP. Forwarding UDP would be useful also.
- yehaaa 6y ago
  Possibly a dumb question, but would it not make sense to only deal with IP for a forward proxy?
- panpanna 6y ago
  Forwarding UDP is a bit more complicated since you will have to keep track of "sessions" unless you are relying data between exactly two points.
studius 6y ago
Touting as 100 LOC using only stdlib: * may show that Ruby can do a lot on its own. * may show that a developer doesn't have to depend on another gem just to get the job done. The Ruby stdlib is powerful, and depending on another gem could bite you if it were to become unmaintained at some point. However, low LOC and use of stdlib isn't always the goal for development in-general. I want to develop solutions quickly that work well and are easy to maintain. If it's easy to learn also, even better. In terms of the size of the code, even in text form, there's a lot I could do. I could reduce lines by replacing EOL with semicolon. I could remove unnecessary spaces. I could use shortest variable names. I could store a compressed version of the code in the file and then do eval of the decompressed version. Those don't make it a better solution, unless the goal is to obfuscate and/or minify it. I know that's not the point, though, and I'm mentioning it only so that people will think about LOC as what it is- just a metric.
- tyingq 6y ago
  It's using something called WEBrick. It's a "default gem", so it sounds like it comes with Ruby. "WEBrick is an HTTP server toolkit that can be configured as an HTTPS server, a proxy server, and a virtual-host server." So I'm reading it as 100 loc of setup to use a proxy toolkit to invoke a proxy.
  - deleted 6y ago
    [deleted]
  - frizkie 6y ago
    Actually, as of Ruby 3, webrick is no longer a "bundled gem". You'll need to add it to your Gemfile and install it in order to use it.
IgorPartola 6y ago
I’m going to be that jerk today: why is it noteworthy that you can do this? It reminds me of the “HTTP server in one line” which essentially executes a pre-made HTTP server that’s part of Python’s standard library. Writing glue code that is small is a sort of expected feature. Writing one without depending on your stdlib is what’s hard. Edit: just checked on one of my projects. Looks like about 600 lines of C for an HTTP server that only uses stdlib’s memory allocation and string manipulation functions as well as socket IO. For an additional 20 or so it works over HTTPS. That was rather difficult to write and I’m sure has at least two bug I am not seeing. See http.c and server.c in https://github.com/ipartola/hawkeye https://github.com/ipartola/hawkeye
- rhines 6y ago
  Eh, who says it needs to be hard? As long as it solves a problem, it seems useful to me. And there may be people out there who aren't super familiar with Ruby or forward proxies who can learn a lot from this implementation. Which isn't to say doing hard stuff is bad either. I've written simple servers in C as well, and those are fun projects that you can learn a lot doing. But I don't think that there's anything wrong with posting stuff like this.
  - IgorPartola 6y ago
    I am not saying it has to be hard. Probably shouldn't have to be. I am saying that why is it noteworthy when it's just the exact glue code you'd expect? There is like 10k LOC that this is using underneath so the 100 LOC is very misleading here.
    - rhines 6y ago
      I suppose some people might be surprised that it can be done so easily in Ruby, or seeing that it's a short example gives a reason to check it out rather than if they'd assumed it was a much larger codebase they wouldn't have time to learn from. And all code is to some extent reliant on all sorts of hidden boilerplate - even a simple listen() call in C triggers a huge chain of function calls behind the scenes. And naturally higher level languages have higher levels of abstraction.
wg0 6y ago
This is really nice boiled down to the very essentials. I am now tempted to write a Python clone and see how much different it turns out to be in terms of code and complexity while only relying on stdlib.
jedisct1 6y ago
You should rewrite it in Rust.
- panpanna 6y ago
  This was downvoted, but I have seen some _extremly_ fast networking software in rust recently and wouldn't mind seeing some simple examples. (I myself would love to code in rust - once the spec and implementation become more stable. Given the current pace, maybe in 2030?)
lukegru 6y ago
I like how you shutdown the threads (with throw), it was a nice touch. This whole project shows the power of IO.copy_stream imo.

115 points by moriarty_ 6y ago

mfkp 6y ago

I had to Google what a forward proxy was. Apparently it's just a regular proxy server? Never heard it called that before, might be helpful to add that to the readme. https://smartproxy.com/blog/the-difference-between-a-reverse-proxy-and-a-forward-proxy https://smartproxy.com/blog/the-difference-between-a-reverse...

redis_mlc 6y ago
A reverse proxy waits for a request from another server and responds to the calling server. You can think of a reverse proxy as the "deep backend" in a web stack. One of the first and most powerful reverse proxies was mod_perl running under httpd. Typically 5 or 6 mod_perl instances can handle requests from 300 normal httpd children. So when somebody says "forward proxy" they usually mean "not a reverse proxy."
- tyingq 6y ago
  Mod_perl doesn't feel like a reverse proxy to me, it's just executing Perl in the Apache process directly. Fastcgi, on the other hand, is passing requests to persistently running standalone Perl instances, over a pipe, Unix domain socket, or tcp connection. Which feels more like a reverse proxy.
  - mapgrep 6y ago
    Mod_perl would more accurately be called a “commonly reverse proxied app server.” Typical setup was to have another Apache instance in front of it handling static requests and reverse proxying app requests.
  - redis_mlc 6y ago
    Nobody runs mod_perl in production except as a reverse proxy because of the RAM consumption and related issues.
inopinatus 6y ago
There are also up, down, charmed, and strange proxies, with obvious applications in quantum computing, nonrepudiation, and dairy farming. Strange proxies were thought to be purely theoretical until an accident involving a rubber band, a liquid lunch, and a particle accelerator collided a transparent squid with varnish at relativistic velocities and the resulting core dump subsequently examined for overflows.
tmpxgdqrcKFuG 6y ago
I think preprending “forward” was meant to avoid confusion. Reverse proxy and Forward proxy, but now you’re wondering if there’s 3 types of proxies.
wg0 6y ago
I also landed on similar pages and now I am confused about SOCK5 proxy vs HTTP proxy vs SSL proxy as outlines in the links above.
cortesoft 6y ago
I think people add 'forward' because reverse proxies are now a lot more common than traditional proxies, so people will assume reverse proxy if you just say 'proxy'.

tutfbhuf 6y ago

I think this is just a tech demonstration, but I wonder how much performance this proxy has in a benchmark against some similar small (in terms of LOC) C/Rust proxy. BTW, Github counts 149 SLOC just for this file: https://github.com/jamesmoriarty/forward-proxy/blob/769b6424b947341c5d6bdfc70ffbdf393296ab0d/lib/forward_proxy/server.rb https://github.com/jamesmoriarty/forward-proxy/blob/769b6424...

moriarty_ 6y ago
The thread pool added 35 LOC but can be replaced with 1 LOC `Thread.new`.

hlasdjlfhalwjk 6y ago

A little bit more than 100 LoC (`cloc lib bin` gives me 147 lines of Ruby) but still nice work.

zaltekk 6y ago
That should be `cloc exe lib`. `bin` has distribution-related auto generated scripts. The actual entry point script is in `lib`. From the sloc count on GitHub just the server file alone is listed as 149. This also uses WEBrick to do the HTTP parsing. The webrick gem is no longer bundled with Ruby as of 3.0.0.

anonfunction 6y ago

Here's a Go version in 41 lines of source code I wrote in 2016: https://github.com/gophergala2016/goxy/blob/master/goxy.go https://github.com/gophergala2016/goxy/blob/master/goxy.go And a forward proxy that supports middleware like adding CORS headers, logging, etc... https://github.com/montanaflynn/roxy https://github.com/montanaflynn/roxy

phuesler 6y ago

Nice demo. Useful for small projects and teaching. I don't expect the performance to be great, considering it is opening up a new connection to the forwarded host for every request. https://github.com/jamesmoriarty/forward-proxy/blob/bc7d9ec121e52fdbcbc4e70184d4c74581873327/lib/forward_proxy/server.rb#L128 https://github.com/jamesmoriarty/forward-proxy/blob/bc7d9ec1...

alexrsagen 6y ago

I made a similar proxy using Go in 2019: https://github.com/alexrsagen/alexproxy https://github.com/alexrsagen/alexproxy It's got a couple more features than OP, check out the README.

panpanna 6y ago
I think yours does a lot more than OPs. Go was made for this kind of problems. It is very easy to write efficient networking code on go that can be read and understood by mere mortals.

studius 6y ago

It looks like it forwards TCP. Forwarding UDP would be useful also.

yehaaa 6y ago
Possibly a dumb question, but would it not make sense to only deal with IP for a forward proxy?
panpanna 6y ago
Forwarding UDP is a bit more complicated since you will have to keep track of "sessions" unless you are relying data between exactly two points.

studius 6y ago

Touting as 100 LOC using only stdlib: * may show that Ruby can do a lot on its own. * may show that a developer doesn't have to depend on another gem just to get the job done. The Ruby stdlib is powerful, and depending on another gem could bite you if it were to become unmaintained at some point. However, low LOC and use of stdlib isn't always the goal for development in-general. I want to develop solutions quickly that work well and are easy to maintain. If it's easy to learn also, even better. In terms of the size of the code, even in text form, there's a lot I could do. I could reduce lines by replacing EOL with semicolon. I could remove unnecessary spaces. I could use shortest variable names. I could store a compressed version of the code in the file and then do eval of the decompressed version. Those don't make it a better solution, unless the goal is to obfuscate and/or minify it. I know that's not the point, though, and I'm mentioning it only so that people will think about LOC as what it is- just a metric.

tyingq 6y ago
It's using something called WEBrick. It's a "default gem", so it sounds like it comes with Ruby. "WEBrick is an HTTP server toolkit that can be configured as an HTTPS server, a proxy server, and a virtual-host server." So I'm reading it as 100 loc of setup to use a proxy toolkit to invoke a proxy.
- deleted 6y ago
  [deleted]
- frizkie 6y ago
  Actually, as of Ruby 3, webrick is no longer a "bundled gem". You'll need to add it to your Gemfile and install it in order to use it.

IgorPartola 6y ago

I’m going to be that jerk today: why is it noteworthy that you can do this? It reminds me of the “HTTP server in one line” which essentially executes a pre-made HTTP server that’s part of Python’s standard library. Writing glue code that is small is a sort of expected feature. Writing one without depending on your stdlib is what’s hard. Edit: just checked on one of my projects. Looks like about 600 lines of C for an HTTP server that only uses stdlib’s memory allocation and string manipulation functions as well as socket IO. For an additional 20 or so it works over HTTPS. That was rather difficult to write and I’m sure has at least two bug I am not seeing. See http.c and server.c in https://github.com/ipartola/hawkeye https://github.com/ipartola/hawkeye

rhines 6y ago
Eh, who says it needs to be hard? As long as it solves a problem, it seems useful to me. And there may be people out there who aren't super familiar with Ruby or forward proxies who can learn a lot from this implementation. Which isn't to say doing hard stuff is bad either. I've written simple servers in C as well, and those are fun projects that you can learn a lot doing. But I don't think that there's anything wrong with posting stuff like this.
- IgorPartola 6y ago
  I am not saying it has to be hard. Probably shouldn't have to be. I am saying that why is it noteworthy when it's just the exact glue code you'd expect? There is like 10k LOC that this is using underneath so the 100 LOC is very misleading here.
  - rhines 6y ago
    I suppose some people might be surprised that it can be done so easily in Ruby, or seeing that it's a short example gives a reason to check it out rather than if they'd assumed it was a much larger codebase they wouldn't have time to learn from. And all code is to some extent reliant on all sorts of hidden boilerplate - even a simple listen() call in C triggers a huge chain of function calls behind the scenes. And naturally higher level languages have higher levels of abstraction.

wg0 6y ago

This is really nice boiled down to the very essentials. I am now tempted to write a Python clone and see how much different it turns out to be in terms of code and complexity while only relying on stdlib.

jedisct1 6y ago

You should rewrite it in Rust.

panpanna 6y ago
This was downvoted, but I have seen some _extremly_ fast networking software in rust recently and wouldn't mind seeing some simple examples. (I myself would love to code in rust - once the spec and implementation become more stable. Given the current pace, maybe in 2030?)

lukegru 6y ago

I like how you shutdown the threads (with throw), it was a nice touch. This whole project shows the power of IO.copy_stream imo.