8 ms·

Show HN: Easy-to-configure Web Server in Go

106 points by oakaz 13y ago

Uchikoma 13y ago
I always wonder with this kind of posts, is it minimalistic or is it an alternative?
- RyanZAG 13y ago
  It's technically an alternative, but since it performs worse and has less features you would probably never actually use it as an alternative. It should probably be described as 'an easily embeddable and extendable(?) option for hosting your golang web app'.
  - oakaz 13y ago
    just updated the title now
- jacques_chester 13y ago
  The point is that it's written in language-du-jour and therefore fascinating and novel. In fairness though, json will probably wind up being a common configuration format anyhow. And the idea of programmatically configurable system software is nice. I liked the Mongrel2 approach of having it in SQLite.
  - harrytuttle 13y ago
    json is fine until your config system needs accurate numeric values.
    - shuzchen 13y ago
      Nothing in the json spec requires that numbers are limited to those internally representable in a javascript runtime. With the right tooling, you can encode/decode very accurate numeric values. I'm not familiar with the go libraries, but python's simplejson has a use_decimal flag that interprets floats using python's Decimal module. That said, I can't think of any reason a web server needs super accurate numbers. All the config options I can think of require nice round ints. Maybe specifying weights to individual backends to proxy to (so all the weights sum up to 1.0)?
      - harrytuttle 13y ago
        99% of sites are CRUD apps, but we're math heavy financial software. We store a lot of reference data in configuration for performance reasons. This needs to be decimal format. JSON is a subset of the ECMA JavaScript standard which declares only floating point. Do "100!" on wolfram alpha and find me a parser that will load that as an example.
        shuzchen 13y ago
        works for me: https://gist.github.com/sirpengi/609711 https://gist.github.com/sirpengi/609711
        harrytuttle 13y ago
        I get a 404. ?
        shuzchen 13y ago
        Bad copy pasta (missing 3 at the end): https://gist.github.com/sirpengi/6097113 https://gist.github.com/sirpengi/6097113
    - stock_toaster 13y ago
      or until someone wants to put in a comment that documents a change they made at 3am to a production system. I think I would prefer toml take off as a config language.
      - harrytuttle 13y ago
        Good point. We use XML (still after 10 years). For all its warts, at least parsers and deserializers work properly.
alexchamberlain 13y ago
Nice to see an alternative on the cards. Of course, we need some independent security reviews before this can be used seriously.
deleted 13y ago
[deleted]
laumars 13y ago
A word of warning guys, I've had a look through the code and unless I've missed the obvious, there's nothing in there to change user ID; which means this would either need to be run as root, or would need to listen on a port > 1024. In the case of the former, that's a huge step backwards in terms of security. In the case of the latter, that would mean you'd need another reverse proxy hooked up - which would negate the need for this web server to begin with. This can easily be fixed within Go though: import ( "log" "syscall" ) const ( user_id int = 1000 group_id int = 1000 ) func secureDaemon() { // set group id first as you need to be root to change group err := syscall.Setgid(group_id) if err != nil { log.Fatalln(err) } err = syscall.Setuid(user_id) if err != nil { log.Fatalln(err) } } You can also add chroot to your code if you want to be ultra paranoid: import ( "os" ) const ( chroot_dir string = "/opt/go-webserver" ) func chrootDaemon() { err := os.Chdir(chroot_dir) if err != nil { log.Fatalln(err) } err = syscall.Chroot(chroot_dir) if err != nil { log.Fatalln(err) } } This will need to be done before you change your user ID (as you need root permissions to chroot) and you may need to compile the Go without CGO because some of the standard Go libraries will have SO dependencies (I found this to be the case with domain name lookups). (the above code is adapted from my own Go web framework that I'm in the processes of building)
- 1331 13y ago
  A program run as a non-privileged user can access privileged ports using authbind: http://en.wikipedia.org/wiki/Authbind http://en.wikipedia.org/wiki/Authbind
  - laumars 13y ago
    Indeed, but it would make more sense (in my opinion) to have the web server natively 'de-elevate' it's permissions than to expect the sysadmin to be competent enough to set up authbind manually. Particularly when setuid/setgid is so easy in Go and that every other webserver out there natively drops down to a designated webuser after the daemons been launched and ports binded.
  - 13y ago
pandeiro 13y ago
Code organization question: is this a common convention in Go, to nest the main package in a subdirectory and put all of the 'modules' in different files but with the same package name? (I'm more used to the Clojure namespace/filesystem symmetry, so this is somewhat new to me. I like the more shallow project tree but it's not immediately apparent where things are defined.)
- redbad 13y ago
  In a package, main or otherwise, files are purely for programmer convenience. They define import scope but are otherwise transparent to the toolchain.
- sarnowski 13y ago
  You have to have your main() in package "main" and all your files in one directory have to be in the same package. So when you want a main() and your logic in a "real" package, your have to seperate them.
justinsb 13y ago
I remain not-entirely-convinced (1) by Go, but things like this are turning the tide. If it is indeed slower than nginx, it is not dramatically so. Considering the amount of time that went in to writing each, the Go version clearly "wins" from the point of view of anyone thinking of writing new code. (1) I find error handling just too tedious to get right (Edit: "I find error handling _in Go_ just too tedious...")
- harrytuttle 13y ago
  From experience of running big production web apps, you need to get error handling right regardless of how tedious it might be. A recent issue where one of our developers ate an exception cost us 12 days of developer time. Another historical issue where the error handling and return codes weren't understood and handled correctly threw 500mb of stack dumps a minute and took our logging system out. Spent the time to get it right :)
  - justinsb 13y ago
    Totally agree, and if Go used exceptions I would probably think it perfect. I added an edit to my comment to avoid the implication you picked up on!
    - gngeal 13y ago
      How do you propose to make exceptions work in the presence of large number of cooperating goroutines? As in, there's no implicit control flow the way you have it in "less-threaded" languages.
      - justinsb 13y ago
        I don't believe that's a problem; goroutines still have a stack, they just aren't always assigned to an OS thread. At least that's my understanding. Go even has exceptions, it just doesn't usually use them. Can you explain what you think the problem is with supporting exceptions? You raise a good point though, in that there is a valid question of "how would you introduce exceptions into the language / runtime". I think the answer is that the err return value can become implicit. Throwing an exception is the equivalent of "return _, err". Invoking a function that can throw an exception is done by not checking the err return; if an exception is thrown/returned and not checked then it is immediately rethrown (return _,err) in an implicit method. try/catch should be easy to introduce, although the semantics of e.g. "defer" could be tricky.
tszming 13y ago
json format is too limited for a web server configuration if you want to be the nginx alternative. Think about how you would represent conditional statements, how to include and reuse external configurations and how to have comments in json.
deleted 13y ago
[deleted]
stevekemp 13y ago
So it's a reverse HTTP proxy, which can also serve files locally? That's an interesting thing, no doubt, but it doesn't feel like a web-server. I wrote a flexible reverse proxy[0] using node.js, a year or so ago, and haven't missed the ability to serve static files directly - so I'm wondering what the use-case for that is? I guess proxying to rails, or similar? 0- http://steve.org.uk/Software/node-reverse-proxy/ http://steve.org.uk/Software/node-reverse-proxy/
- justinsb 13y ago
  Did you benchmark your node.js version? I would love to know how node stacks up here. I agree that static serving is not as interesting, but it is useful to be able to serve up a fail-whale page!
  - stevekemp 13y ago
    Honestly I never have. Because in practise it is fast enough that it just didn't come up. I'm sure that if I were to pimp the project properly then that would be a good thing to do, but half the fun of my proxy is to allow "interesting" rewrites, dynamically. To benchmark I'd be too tempted try to game the results by doing a straight pass-through comparison to mod_proxy, nginx, etc. That would lose half of the appeal of the project in the first place.
    - oakaz 13y ago
      In my experience, NodeJS was slow. I first write a boxcars-like server in NodeJS; http://github.com/azer/door http://github.com/azer/door and here is the benchmarks of it; https://gist.github.com/azer/5946227 https://gist.github.com/azer/5946227
JoeAcchino 13y ago
I'm very interested in this because this is the kind of web server I was planning to write for my work, but from a quick glance I didn't find HTTPS support. Is HTTPS supported or planned?
- oakaz 13y ago
  have you tried it?
simonw 13y ago
The custom 404 page example in the readme looked incomplete - how do you ensure those pages are served with the correct HTTP status code?

106 points by oakaz 13y ago

Uchikoma 13y ago

I always wonder with this kind of posts, is it minimalistic or is it an alternative?

RyanZAG 13y ago
It's technically an alternative, but since it performs worse and has less features you would probably never actually use it as an alternative. It should probably be described as 'an easily embeddable and extendable(?) option for hosting your golang web app'.
- oakaz 13y ago
  just updated the title now
jacques_chester 13y ago
The point is that it's written in language-du-jour and therefore fascinating and novel. In fairness though, json will probably wind up being a common configuration format anyhow. And the idea of programmatically configurable system software is nice. I liked the Mongrel2 approach of having it in SQLite.
- harrytuttle 13y ago
  json is fine until your config system needs accurate numeric values.
  - shuzchen 13y ago
    Nothing in the json spec requires that numbers are limited to those internally representable in a javascript runtime. With the right tooling, you can encode/decode very accurate numeric values. I'm not familiar with the go libraries, but python's simplejson has a use_decimal flag that interprets floats using python's Decimal module. That said, I can't think of any reason a web server needs super accurate numbers. All the config options I can think of require nice round ints. Maybe specifying weights to individual backends to proxy to (so all the weights sum up to 1.0)?
    - harrytuttle 13y ago
      99% of sites are CRUD apps, but we're math heavy financial software. We store a lot of reference data in configuration for performance reasons. This needs to be decimal format. JSON is a subset of the ECMA JavaScript standard which declares only floating point. Do "100!" on wolfram alpha and find me a parser that will load that as an example.
      - shuzchen 13y ago
        works for me: https://gist.github.com/sirpengi/609711 https://gist.github.com/sirpengi/609711
        harrytuttle 13y ago
        I get a 404. ?
        shuzchen 13y ago
        Bad copy pasta (missing 3 at the end): https://gist.github.com/sirpengi/6097113 https://gist.github.com/sirpengi/6097113
  - stock_toaster 13y ago
    or until someone wants to put in a comment that documents a change they made at 3am to a production system. I think I would prefer toml take off as a config language.
    - harrytuttle 13y ago
      Good point. We use XML (still after 10 years). For all its warts, at least parsers and deserializers work properly.

alexchamberlain 13y ago

Nice to see an alternative on the cards. Of course, we need some independent security reviews before this can be used seriously.

deleted 13y ago

[deleted]

laumars 13y ago

A word of warning guys, I've had a look through the code and unless I've missed the obvious, there's nothing in there to change user ID; which means this would either need to be run as root, or would need to listen on a port > 1024. In the case of the former, that's a huge step backwards in terms of security. In the case of the latter, that would mean you'd need another reverse proxy hooked up - which would negate the need for this web server to begin with. This can easily be fixed within Go though: import ( "log" "syscall" ) const ( user_id int = 1000 group_id int = 1000 ) func secureDaemon() { // set group id first as you need to be root to change group err := syscall.Setgid(group_id) if err != nil { log.Fatalln(err) } err = syscall.Setuid(user_id) if err != nil { log.Fatalln(err) } } You can also add chroot to your code if you want to be ultra paranoid: import ( "os" ) const ( chroot_dir string = "/opt/go-webserver" ) func chrootDaemon() { err := os.Chdir(chroot_dir) if err != nil { log.Fatalln(err) } err = syscall.Chroot(chroot_dir) if err != nil { log.Fatalln(err) } } This will need to be done before you change your user ID (as you need root permissions to chroot) and you may need to compile the Go without CGO because some of the standard Go libraries will have SO dependencies (I found this to be the case with domain name lookups). (the above code is adapted from my own Go web framework that I'm in the processes of building)

1331 13y ago
A program run as a non-privileged user can access privileged ports using authbind: http://en.wikipedia.org/wiki/Authbind http://en.wikipedia.org/wiki/Authbind
- laumars 13y ago
  Indeed, but it would make more sense (in my opinion) to have the web server natively 'de-elevate' it's permissions than to expect the sysadmin to be competent enough to set up authbind manually. Particularly when setuid/setgid is so easy in Go and that every other webserver out there natively drops down to a designated webuser after the daemons been launched and ports binded.
- 13y ago

pandeiro 13y ago

Code organization question: is this a common convention in Go, to nest the main package in a subdirectory and put all of the 'modules' in different files but with the same package name? (I'm more used to the Clojure namespace/filesystem symmetry, so this is somewhat new to me. I like the more shallow project tree but it's not immediately apparent where things are defined.)

redbad 13y ago
In a package, main or otherwise, files are purely for programmer convenience. They define import scope but are otherwise transparent to the toolchain.
sarnowski 13y ago
You have to have your main() in package "main" and all your files in one directory have to be in the same package. So when you want a main() and your logic in a "real" package, your have to seperate them.

justinsb 13y ago

I remain not-entirely-convinced (1) by Go, but things like this are turning the tide. If it is indeed slower than nginx, it is not dramatically so. Considering the amount of time that went in to writing each, the Go version clearly "wins" from the point of view of anyone thinking of writing new code. (1) I find error handling just too tedious to get right (Edit: "I find error handling _in Go_ just too tedious...")

harrytuttle 13y ago
From experience of running big production web apps, you need to get error handling right regardless of how tedious it might be. A recent issue where one of our developers ate an exception cost us 12 days of developer time. Another historical issue where the error handling and return codes weren't understood and handled correctly threw 500mb of stack dumps a minute and took our logging system out. Spent the time to get it right :)
- justinsb 13y ago
  Totally agree, and if Go used exceptions I would probably think it perfect. I added an edit to my comment to avoid the implication you picked up on!
  - gngeal 13y ago
    How do you propose to make exceptions work in the presence of large number of cooperating goroutines? As in, there's no implicit control flow the way you have it in "less-threaded" languages.
    - justinsb 13y ago
      I don't believe that's a problem; goroutines still have a stack, they just aren't always assigned to an OS thread. At least that's my understanding. Go even has exceptions, it just doesn't usually use them. Can you explain what you think the problem is with supporting exceptions? You raise a good point though, in that there is a valid question of "how would you introduce exceptions into the language / runtime". I think the answer is that the err return value can become implicit. Throwing an exception is the equivalent of "return _, err". Invoking a function that can throw an exception is done by not checking the err return; if an exception is thrown/returned and not checked then it is immediately rethrown (return _,err) in an implicit method. try/catch should be easy to introduce, although the semantics of e.g. "defer" could be tricky.

tszming 13y ago

json format is too limited for a web server configuration if you want to be the nginx alternative. Think about how you would represent conditional statements, how to include and reuse external configurations and how to have comments in json.

deleted 13y ago

[deleted]

stevekemp 13y ago

So it's a reverse HTTP proxy, which can also serve files locally? That's an interesting thing, no doubt, but it doesn't feel like a web-server. I wrote a flexible reverse proxy[0] using node.js, a year or so ago, and haven't missed the ability to serve static files directly - so I'm wondering what the use-case for that is? I guess proxying to rails, or similar? 0- http://steve.org.uk/Software/node-reverse-proxy/ http://steve.org.uk/Software/node-reverse-proxy/

justinsb 13y ago
Did you benchmark your node.js version? I would love to know how node stacks up here. I agree that static serving is not as interesting, but it is useful to be able to serve up a fail-whale page!
- stevekemp 13y ago
  Honestly I never have. Because in practise it is fast enough that it just didn't come up. I'm sure that if I were to pimp the project properly then that would be a good thing to do, but half the fun of my proxy is to allow "interesting" rewrites, dynamically. To benchmark I'd be too tempted try to game the results by doing a straight pass-through comparison to mod_proxy, nginx, etc. That would lose half of the appeal of the project in the first place.
  - oakaz 13y ago
    In my experience, NodeJS was slow. I first write a boxcars-like server in NodeJS; http://github.com/azer/door http://github.com/azer/door and here is the benchmarks of it; https://gist.github.com/azer/5946227 https://gist.github.com/azer/5946227

JoeAcchino 13y ago

I'm very interested in this because this is the kind of web server I was planning to write for my work, but from a quick glance I didn't find HTTPS support. Is HTTPS supported or planned?

oakaz 13y ago
have you tried it?

simonw 13y ago

The custom 404 page example in the readme looked incomplete - how do you ensure those pages are served with the correct HTTP status code?