13 ms·

Go vs. Python for a simple web server

93 points by amarsahinovic 14y ago

asimjalis 14y ago
Do you have a link to your source code? It would be interesting to see the difference in the actual code.
- jlarocco 14y ago
  Yeah, an article like this is almost useless without being able to see the author's code and see why he came to the conclusions he came to.
  - dsl 14y ago
    The author didn't post this to HN. It's over 2 years old and reads like a "brain dump" type blog post.
- chimeracoder 14y ago
  Here's a really basic Go web server I wrote that doesn't do much except log someone in via OAuth: https://github.com/ChimeraCoder/go-server-bootstrap https://github.com/ChimeraCoder/go-server-bootstrap The only messy part is the OAuth callback, which I did manually instead of using one of the OAuth2 libraries, because at the time I was trying to identify a possible bug in an OAuth implementation. Keep in mind that it's not the cleanest or most idiomatic Go code, but even that aside, it's relatively concise - and more importantly, straightforward to understand, even if you've never written Go before.
- jff 14y ago
  Unless he's updated it, his 2 year old Go code won't compile any more. 2 years ago, Go was a rapidly-churning target. Now, it's settled off to Go 1, but you'll have to go through and fix some changes--and there may now be better ways to do things.
  - Benferhat 14y ago
    I'm sure you already know this, but for future readers: check out gofix [0] when you find yourself in this situation. [0] http://golang.org/cmd/fix/ http://golang.org/cmd/fix/ [1] http://blog.golang.org/2011/04/introducing-gofix.html http://blog.golang.org/2011/04/introducing-gofix.html
mseepgood 14y ago
"Written on Jun 13 2010" Why are people posting ancient and outdated stuff to Hacker NEWS?
- manojlds 14y ago
  Hate articles which have the date at the bottom. Had to read the entire stuff to realize it was more than 2 years old.
  - saraid216 14y ago
    When I get suspicious about dated information, I usually make a point of looking for a date, even if it means scrolling down and checking.
- scott_s 14y ago
  We regularly post old articles. Not everything we post is "news." However, common practice is to put the year in the title if it's not recent.
  - S4M 14y ago
    That is true if the content of the article contains a truth that is not time dependent, or if the article is posted for historical purpose. But this article is comparing one new language against a more established one, so in two years you should expect the new language to evolve significantly. Especially, the problems mentioned in the article about Go may be now fixed. Talking about the Go from 2010 is just not relevant in 2013!
riquito 14y ago
Where is the math? Really, what's the point of making assumptions on the whole subject and invent a conclusion? You may be right, but you need facts to prove a point (which here is "go is as easy or easier than Python, faster, use less memory and has better concurrency"). At least the source code of both implementations..
- jack9 14y ago
  The linecount is of particular interest, as it ensures that Go has a decent amount of abstraction (almost as much as a scripting language?!) I think a performance comparison of memory/req/load ... anything, would be the next thing I would want to see. I liked this writeup better than the last one.
noobface 14y ago
Hardware/Network guy here. I realize this thread is probably only tangentially appropriate for this discussion, but I digress, I haven't seen anything too close in my daily visits here. There's a ton to be said for per-thread connections. This author certainly recognizes the benefit of truly parallel communications, however, there are some networking considerations that might be of interest. Especially regarding the tendency to simply "open more sockets" to increase overall speed. Most software guys I know tend to ignore or just be ignorant of how their choices affect the underlying hardware. I realize this is a brash generalization; there are some full-spectrum people out there, and I appreciate you. That said, opening new sockets is not without diminishing returns. The sheer volume of port numbers for those sockets can lead to memory exhaustion in consumer network devices, especially when something else is running. In this respect, bit torrent is atypically aggressive. Basic consumer routers have hard time keeping up with their internal processes when flooded with new TCP sessions for hundreds/thousands of ports. NAT tends to run slowly as memory exhaustion takes hold and many times the only option is to shorten the timeout window on connections. (Admittedly, the default settings for NAT table timeouts is ridiculous; sometimes up to 300 seconds). With long default timeouts, it's no wonder these simple MIPS systems are overwhelmed. Count how many tabs you have open now, and consider that they all may be using the same number of ports the services you're building do. Not to mention the ad networks...etc All I'm really trying to get across is, more ports, especially on consumer devices, does have diminishing returns despite its apparent immediate benefit. Yes, newer routers do have better hardware. Yes, your AWS machine can handle tons of connections. But your end users may be like you, they may not know how much memory their local router has or particularly care. I'm not advocating anything other than moderation the number of sockets you employ. I just wanted to get this out there.
- jlouis 14y ago
  NAT (Or more correctly PAT) was never a good idea and this is the main reason why the cheap network device suffer. It should not need any kind of intelligence. When some of us "Software" guys are up in arms over the slow IPv6 adoption it is because the current IPv4 state translates to thousands of extra code lines for little to no benefit. We also need those cheap consumer devices to understand Active Queue Management, but that is another problem :)
jstanley 14y ago
"Speed and memory efficiency I haven’t measured." This is the part I was most interested in. It's my understanding that Goroutines are not currently implemented very efficiently and I wanted to see how the current implementation compares to a typical Python version.
- chimeracoder 14y ago
  Well, multiple goroutines can run in parallel, which isn't possible in CPython. That's a huge difference right off the bat, even if gc has some work left in optimizing the implementation. (As far as non-concurrent code goes, gccgo is very well-optimized, because it's basically piggy-backing on the extensive optimizations of gcc over the decades.)
- jff 14y ago
  Goroutines are "green threads", small threads managed by the Go runtime and scheduled within multiple OS processes. I've never seen any indication that they're inefficient; I regularly create and destroy thousands of goroutines in less than a second. Oh, and Go doesn't have a global interpreter lock, so I'd expect the threading to work better than in python.
Goopplesoft 14y ago
> I haven’t measured. I'm extremely interested in the measurements if someone could do them. I want to know how much the web concurrency/gorutines adds in terms of load testing and response times.
- voidlogic 14y ago
  This is an extremely simple comparison, but unless you are really pushing to garbage collector in go 1.0.3 these results are pretty typical across the board: https://docs.google.com/open?id=0ByTAIYFOarh0cm9aSXlzc0NWaXc https://docs.google.com/open?id=0ByTAIYFOarh0cm9aSXlzc0NWaXc Go tip (aka 1.1 alpha/in progress) is even faster!
  - voidlogic 14y ago
    My previous link is a more general performance comparison across many languages. I realized upon re-reading that you were more interested in concurrency/scaling, this might be more what you are looking for: http://eric.themoritzfamily.com/websocket-demo-results-v2.html http://eric.themoritzfamily.com/websocket-demo-results-v2.ht...
    - Goopplesoft 14y ago
      Thanks!
SeanDav 14y ago
I would like to see the equivalent done with node.js and something like express for comparison purposes.
- voidlogic 14y ago
  If you do this comparison and are doing performance testing yourself, make sure to test the scaling. At low concurrent connection counts I find their request handling latency to be pretty similar, but I have found Go scales much better. Also, another factor (and different type of scaling), is how big the test app is. Go is for the most part written in Go (stdlibs etc). But in node.js,which is mostly C under the hood, as the app gets larger and larger and the ratio time spent in C vs JS changes to favor JS, things can take a downhill turn. Enough said: https://twitter.com/felixge/status/275320642059001857 https://twitter.com/felixge/status/275320642059001857
jeremyjh 14y ago
> The downside of that is that at any given time Tornado can process only one HTTP request. If that processing takes a long time, it blocks all other requests. What? The whole point of an async framework is to service multiple HTTP requests concurrently.
- smilliken 14y ago
  Yes, concurrently, but not necessarily in parallel [1]. Here's an example of what the author meant: while (True); Executing this within a request will block all other requests in a single-threaded web server. [1] http://stackoverflow.com/questions/1897993/difference-between-concurrent-programming-and-parallel-programming http://stackoverflow.com/questions/1897993/difference-betwee...
  - jeremyjh 14y ago
    The author is talking about concurrency, and so am I. Why the downvote? That quote comes from his paragraph headed "Concurrency". It also refers directly to concurrent HTTP requests. Well, a single Tornado or Node.js process can generally handle many thousands of concurrent HTTP requests if they are implemented correctly without blocking on I/O.
    - spankalee 14y ago
      But even if the request handlers don't perform blocking I/O, the interpreter is still single threaded, so any computation the handler does blocks all other handlers - only one runs at a time. "Concurrency" in the context of node.js usually is referring to simultaneous connections, not parallel handling of the connections.
bishop_mandible 14y ago
Note: this blog posting is 2 years old. In the meantime Go 1 was released with full Windows support and the ecosystem has matured a lot and it's used in production.
st3fan 14y ago
These comparisons always forget about packaging and deploying.
- prewett 14y ago
  I would imagine that the Python deployment process is a few `pip install package-name` and a `hg clone ...`. For Go, `go build -o name` creates a complete, statically linked binary, so deploying is `cp name dest/`, but I'd guess to deploy to your server you usually just do `hg clone ...` and call `go run name`.
  - voidlogic 14y ago
    Most shops that use a compiled language like Go take advantage of the fact that they do not need to take the risk of storing source code on production hosts. The deploy process is most likely an scp or git pull, etc from a binary/asset repo. Go binaries are statically linked to you have no dynamically linked libraries (.so/.dll files) to worry about either. If you have the binary you don't need to use "go run" which is a compile and run, just run it (nohup ./myProgram &) or run it with which whichever supervisor script/daemon/etc you use for other services.
ternaryoperator 14y ago
>Go is used internally at Google, so it won’t be abandoned. I don't see how A implies B. Given the number of APIs Google used and abandoned, I think the best that can be said is: Because Google uses Go internally, it's less likely to be abandoned.
laurent123456 14y ago
He wrote that "In Go, writing concurrent code is more natural and efficiency of goroutines is much better than threads." I would have assumed goroutines were threads too. If not, how are they implemented?
- wmf 14y ago
  Goroutines are M:N green threads but they're not 1:1 native threads.
jebblue 14y ago
>> Go is used internally at Google, so it won’t be abandoned. That's what I used to think about GWT.

93 points by amarsahinovic 14y ago

asimjalis 14y ago

Do you have a link to your source code? It would be interesting to see the difference in the actual code.

jlarocco 14y ago
Yeah, an article like this is almost useless without being able to see the author's code and see why he came to the conclusions he came to.
- dsl 14y ago
  The author didn't post this to HN. It's over 2 years old and reads like a "brain dump" type blog post.
chimeracoder 14y ago
Here's a really basic Go web server I wrote that doesn't do much except log someone in via OAuth: https://github.com/ChimeraCoder/go-server-bootstrap https://github.com/ChimeraCoder/go-server-bootstrap The only messy part is the OAuth callback, which I did manually instead of using one of the OAuth2 libraries, because at the time I was trying to identify a possible bug in an OAuth implementation. Keep in mind that it's not the cleanest or most idiomatic Go code, but even that aside, it's relatively concise - and more importantly, straightforward to understand, even if you've never written Go before.
jff 14y ago
Unless he's updated it, his 2 year old Go code won't compile any more. 2 years ago, Go was a rapidly-churning target. Now, it's settled off to Go 1, but you'll have to go through and fix some changes--and there may now be better ways to do things.
- Benferhat 14y ago
  I'm sure you already know this, but for future readers: check out gofix [0] when you find yourself in this situation. [0] http://golang.org/cmd/fix/ http://golang.org/cmd/fix/ [1] http://blog.golang.org/2011/04/introducing-gofix.html http://blog.golang.org/2011/04/introducing-gofix.html

mseepgood 14y ago

"Written on Jun 13 2010" Why are people posting ancient and outdated stuff to Hacker NEWS?

manojlds 14y ago
Hate articles which have the date at the bottom. Had to read the entire stuff to realize it was more than 2 years old.
- saraid216 14y ago
  When I get suspicious about dated information, I usually make a point of looking for a date, even if it means scrolling down and checking.
scott_s 14y ago
We regularly post old articles. Not everything we post is "news." However, common practice is to put the year in the title if it's not recent.
- S4M 14y ago
  That is true if the content of the article contains a truth that is not time dependent, or if the article is posted for historical purpose. But this article is comparing one new language against a more established one, so in two years you should expect the new language to evolve significantly. Especially, the problems mentioned in the article about Go may be now fixed. Talking about the Go from 2010 is just not relevant in 2013!

riquito 14y ago

Where is the math? Really, what's the point of making assumptions on the whole subject and invent a conclusion? You may be right, but you need facts to prove a point (which here is "go is as easy or easier than Python, faster, use less memory and has better concurrency"). At least the source code of both implementations..

jack9 14y ago
The linecount is of particular interest, as it ensures that Go has a decent amount of abstraction (almost as much as a scripting language?!) I think a performance comparison of memory/req/load ... anything, would be the next thing I would want to see. I liked this writeup better than the last one.

noobface 14y ago

Hardware/Network guy here. I realize this thread is probably only tangentially appropriate for this discussion, but I digress, I haven't seen anything too close in my daily visits here. There's a ton to be said for per-thread connections. This author certainly recognizes the benefit of truly parallel communications, however, there are some networking considerations that might be of interest. Especially regarding the tendency to simply "open more sockets" to increase overall speed. Most software guys I know tend to ignore or just be ignorant of how their choices affect the underlying hardware. I realize this is a brash generalization; there are some full-spectrum people out there, and I appreciate you. That said, opening new sockets is not without diminishing returns. The sheer volume of port numbers for those sockets can lead to memory exhaustion in consumer network devices, especially when something else is running. In this respect, bit torrent is atypically aggressive. Basic consumer routers have hard time keeping up with their internal processes when flooded with new TCP sessions for hundreds/thousands of ports. NAT tends to run slowly as memory exhaustion takes hold and many times the only option is to shorten the timeout window on connections. (Admittedly, the default settings for NAT table timeouts is ridiculous; sometimes up to 300 seconds). With long default timeouts, it's no wonder these simple MIPS systems are overwhelmed. Count how many tabs you have open now, and consider that they all may be using the same number of ports the services you're building do. Not to mention the ad networks...etc All I'm really trying to get across is, more ports, especially on consumer devices, does have diminishing returns despite its apparent immediate benefit. Yes, newer routers do have better hardware. Yes, your AWS machine can handle tons of connections. But your end users may be like you, they may not know how much memory their local router has or particularly care. I'm not advocating anything other than moderation the number of sockets you employ. I just wanted to get this out there.

jlouis 14y ago
NAT (Or more correctly PAT) was never a good idea and this is the main reason why the cheap network device suffer. It should not need any kind of intelligence. When some of us "Software" guys are up in arms over the slow IPv6 adoption it is because the current IPv4 state translates to thousands of extra code lines for little to no benefit. We also need those cheap consumer devices to understand Active Queue Management, but that is another problem :)

jstanley 14y ago

"Speed and memory efficiency I haven’t measured." This is the part I was most interested in. It's my understanding that Goroutines are not currently implemented very efficiently and I wanted to see how the current implementation compares to a typical Python version.

chimeracoder 14y ago
Well, multiple goroutines can run in parallel, which isn't possible in CPython. That's a huge difference right off the bat, even if gc has some work left in optimizing the implementation. (As far as non-concurrent code goes, gccgo is very well-optimized, because it's basically piggy-backing on the extensive optimizations of gcc over the decades.)
jff 14y ago
Goroutines are "green threads", small threads managed by the Go runtime and scheduled within multiple OS processes. I've never seen any indication that they're inefficient; I regularly create and destroy thousands of goroutines in less than a second. Oh, and Go doesn't have a global interpreter lock, so I'd expect the threading to work better than in python.

Goopplesoft 14y ago

> I haven’t measured. I'm extremely interested in the measurements if someone could do them. I want to know how much the web concurrency/gorutines adds in terms of load testing and response times.

voidlogic 14y ago
This is an extremely simple comparison, but unless you are really pushing to garbage collector in go 1.0.3 these results are pretty typical across the board: https://docs.google.com/open?id=0ByTAIYFOarh0cm9aSXlzc0NWaXc https://docs.google.com/open?id=0ByTAIYFOarh0cm9aSXlzc0NWaXc Go tip (aka 1.1 alpha/in progress) is even faster!
- voidlogic 14y ago
  My previous link is a more general performance comparison across many languages. I realized upon re-reading that you were more interested in concurrency/scaling, this might be more what you are looking for: http://eric.themoritzfamily.com/websocket-demo-results-v2.html http://eric.themoritzfamily.com/websocket-demo-results-v2.ht...
  - Goopplesoft 14y ago
    Thanks!

SeanDav 14y ago

I would like to see the equivalent done with node.js and something like express for comparison purposes.

voidlogic 14y ago
If you do this comparison and are doing performance testing yourself, make sure to test the scaling. At low concurrent connection counts I find their request handling latency to be pretty similar, but I have found Go scales much better. Also, another factor (and different type of scaling), is how big the test app is. Go is for the most part written in Go (stdlibs etc). But in node.js,which is mostly C under the hood, as the app gets larger and larger and the ratio time spent in C vs JS changes to favor JS, things can take a downhill turn. Enough said: https://twitter.com/felixge/status/275320642059001857 https://twitter.com/felixge/status/275320642059001857

jeremyjh 14y ago

> The downside of that is that at any given time Tornado can process only one HTTP request. If that processing takes a long time, it blocks all other requests. What? The whole point of an async framework is to service multiple HTTP requests concurrently.

smilliken 14y ago
Yes, concurrently, but not necessarily in parallel [1]. Here's an example of what the author meant: while (True); Executing this within a request will block all other requests in a single-threaded web server. [1] http://stackoverflow.com/questions/1897993/difference-between-concurrent-programming-and-parallel-programming http://stackoverflow.com/questions/1897993/difference-betwee...
- jeremyjh 14y ago
  The author is talking about concurrency, and so am I. Why the downvote? That quote comes from his paragraph headed "Concurrency". It also refers directly to concurrent HTTP requests. Well, a single Tornado or Node.js process can generally handle many thousands of concurrent HTTP requests if they are implemented correctly without blocking on I/O.
  - spankalee 14y ago
    But even if the request handlers don't perform blocking I/O, the interpreter is still single threaded, so any computation the handler does blocks all other handlers - only one runs at a time. "Concurrency" in the context of node.js usually is referring to simultaneous connections, not parallel handling of the connections.

bishop_mandible 14y ago

Note: this blog posting is 2 years old. In the meantime Go 1 was released with full Windows support and the ecosystem has matured a lot and it's used in production.

st3fan 14y ago

These comparisons always forget about packaging and deploying.

prewett 14y ago
I would imagine that the Python deployment process is a few `pip install package-name` and a `hg clone ...`. For Go, `go build -o name` creates a complete, statically linked binary, so deploying is `cp name dest/`, but I'd guess to deploy to your server you usually just do `hg clone ...` and call `go run name`.
- voidlogic 14y ago
  Most shops that use a compiled language like Go take advantage of the fact that they do not need to take the risk of storing source code on production hosts. The deploy process is most likely an scp or git pull, etc from a binary/asset repo. Go binaries are statically linked to you have no dynamically linked libraries (.so/.dll files) to worry about either. If you have the binary you don't need to use "go run" which is a compile and run, just run it (nohup ./myProgram &) or run it with which whichever supervisor script/daemon/etc you use for other services.

ternaryoperator 14y ago

>Go is used internally at Google, so it won’t be abandoned. I don't see how A implies B. Given the number of APIs Google used and abandoned, I think the best that can be said is: Because Google uses Go internally, it's less likely to be abandoned.

laurent123456 14y ago

He wrote that "In Go, writing concurrent code is more natural and efficiency of goroutines is much better than threads." I would have assumed goroutines were threads too. If not, how are they implemented?

wmf 14y ago
Goroutines are M:N green threads but they're not 1:1 native threads.

jebblue 14y ago

>> Go is used internally at Google, so it won’t be abandoned. That's what I used to think about GWT.