7 ms·

I think this is a much needed and excellent point to make. Just take a look at how Go dips down when using Webgo.

• goodwink 14y ago
  I think Go is almost the ideal example here, you're right. Go provides a pretty rich "standard library" for writing web serving stuff so it's a good place where you could really imagine writing your performance critical stuff just on the base platform even if you use something like Webgo for the rest of your app. Some of the other platforms are much less amenable to that since the standard primitives the platform exposes are very primitive indeed (servlets api, rack api, etc.). Perhaps there's some value in looking at how your favorite framework stacks up against its raw platform and trying to contribute some optimizations to close the gap a bit.
• voidlogic 14y ago
  Webgo... I'll stick with "net/http" and Gorilla thanks. Also, They used Go 1.0.3... I hope they update to 1.1 next month. Most everyone using Go for intensive production uses is using Go tip (which is the branch due to become 1.1 RC next month)
  • pfalls 14y ago
    This is great to know, we were hesitant to use non-stable versions (although we were forced to in certain cases), but knowing that it's what is common practice for production environments would change our minds.
    • voidlogic 14y ago
      We switched to using tip after several Go core devs recommended that move to us, the folks on go-nuts IRC agreed and we tested it and found it to be more stable than 1.0.3
  • goodwink 14y ago
    Wow, directly on tip? That seems to speak very highly of the day-to-day development stability of Go.
    • voidlogic 14y ago
      A good tip build tends to be more stable then 1.0.3 and has hugely improved performance (most importantly for large application in garbage collection and generation). To select a suitable tip build we use http://build.golang.org/ http://build.golang.org/ and https://groups.google.com/forum/?fromgroups#!forum/golang-dev https://groups.google.com/forum/?fromgroups#!forum/golang-de... . My recommendation would be to find a one or two week old build that passed all checks, do a quick skim of the mailing list to make sure there weren't any other issues and use that revision. Also, you will see some the the builders are broken- Of course if your application has automated unit tests and load tests, run those too before a full deployment.
      • goodwink 14y ago
        Thanks, this comment really helped me in my evaluation of Go today. I had been playing around with 1.0.3 for a couple days, but tip is definitely where it's at.
        • voidlogic 14y ago
          I'm glad I could help. Go 1.1 RC should be out early next month. So if you want you could wait for that (for production use).
    • papsosouid 14y ago
      Or it could speak poorly of their release process, which is more accurate. The stable release is simply so bad compared to tip that everyone uses tip. There should have been multiple releases since the last stable release so that people could get those improvements without having to run tip.
      • pekk 14y ago
• grey-area 14y ago
  I'm curious about that - because there's so little to webgo I suspect the answer is something really trivial. I haven't really looked at it before, but the framework itself is just 500 lines or so unless I'm looking at the wrong one... Given that the json marshalling and server components would be exactly the same between go and webgo, I'm curious as to whether changing the url recognised to be just /json in the goweb tests would make any difference, any reason it was different?
  • grey-area 14y ago
    Just had a look at the tests and the urls responded to differ: http://localhost:8080/json http://localhost:8080/json http://localhost:8080/ http://localhost:8080/(.*) Shouldn't all these examples at least be trying to do the same sort of work? For such a trivial test differences like that could make a huge difference to the outcome. It's great to see replicable tests like this which show their working, but they do need to be testing the same thing. I also think they should have something more substantial to test as well as json marshalling on all the platforms, like serving an HTML page made with a template and with the message, as that'd give a far better indication of speed for tasks web servers typically perform. Still, it's a great idea and really interesting to see this sort of comparison, even if it could be improved.
    • pfalls 14y ago
      One of the next steps we'd like to take is to have a test that does cover a more typical web request, and is less database heavy than our 20 query test, just like you describe. Ultimately, we felt that these tests were a sufficient starting point.
      • grey-area 14y ago
        I was a little confused by the different urls used in the tests, as for this sort of light test, particularly in Go, where all the serving stuff is common between frameworks, you're mostly going to be testing routing. Any reason you chose a different route here? (/json versus /(.★) )? I can't think of much else that this little web.go framework does (assuming the fcgi bits etc are unused now and it has moved over to net/http). I don't think many people use web.go, gorilla and its mux router seems to be more popular as a bare bones option on Go, so it'd possibly be interesting to use that instead. It'd be great to see a follow up post with a few changes to the tests to take in the criticisms or answer questions. While you may come in for a lot of criticism and nitpicking here for flaws (real or imagined) in the methodology, I do think this is a valuable exercise if you try to make it as consistent and repeatable as possible - if nothing else it'd be a good reference for other framework authors to test against.
• bilbo0s 14y ago
  I'm not sure many people would use webgo in real life. I don't know... maybe some people... certainly not pros. Also, the 1.0.3 thing is probably dragging on the numbers a bit. 1.1 would boost it a little. Not enough to get it into the top tier... but a little. Also, for Vert.x, they seem to be only running one verticle. Which would never happen in real life. Play could be optimized a bit... but not much. What they have is, to my mind, a fair ranking for it. Small issues with a few of the others but nothing major. I think Go and Vert.x are the ones that would get the biggest jumps if experts looked at them. And let's be frank... does Vert.x really need a jump? So what they have here is pretty accurate... I mean... just based on looking through the code. But Go might fare better if it used tip. And Vert.x would DEFINITELY fair better with proper worker-nonworker verticles running.
  • pfalls 14y ago
    for Vert.x, we specified the number of verticals in App.groovy, rather than on the command line, which we think is a valid way to specify it.
    • bilbo0s 14y ago
      OK... I ran the Vert.x test... runs a bit faster here with 4 workers instead of 8. I suspect what is happening there is that at times all 8 cores can be pinned by workers, while responses wait to be sent back on the 8 non workers. But not that big a change in speed actually. One thing more, when you swap in a couchbase persister for the mongo persister it's faster yet. The difference is actually much larger than the difference you get balancing the number of worker vs non worker verticles. Also thinking that swapping gson in for jackson would improve things... but I don't think that those are fair changes. (well... the couchbase may be a fair change) Also tested Cake just because it had been a while since I have used it... and I couldn't believe it was that much worse than PHP. Your numbers there seem valid though given my test results. That's pretty sad. Finally, tried to get in a test of your Go stuff. I'm making what I think are some fair changes ... but it did not speed up as much as I thought. In fact, initially it was slower than your test with 1.1. So after further review... well done sir.
  • jholla14 14y ago
• bradrydzewski 14y ago
  I used Go's benchmarking tool to compare raw routing performance of various frameworks. The handlers all return a simple "Hello World" string. Here are the results: PASS Benchmark_Routes 100000 13945 ns/op Benchmark_Pat 500000 6068 ns/op Benchmark_GorillaHandler 200000 11042 ns/op Benchmark_Webgo 100000 26350 ns/op ok github.com/bradrydzewski/routes/bench 12.605s I then ran the same benchmark, but this time I modified the handler to serialize a struct to JSON and write to the response. Here are the results: Benchmark_Routes 100000 21446 ns/op Benchmark_Pat 100000 14130 ns/op Benchmark_GorillaHandler 100000 17735 ns/op Benchmark_Webgo 50000 33726 ns/op ok github.com/bradrydzewski/routes/bench 9.805s In the first test, Pat is almost twice as a fast as the Gorilla framework. In the second test, when we added a bit more logic to the handler (marshaling a struct to JSON), Pat was only about 18% faster than Gorilla. In fact, it turns out it takes longer to serialize to JSON (8000ns) than it does for Pat to route and serve the request (6000ns). Now, imagine I created a third benchmark that did something more complex, like executing a database query and serving the results using the html/template package. There would be a negligible difference in performance across frameworks because routing is not going to be your bottleneck. I would personally choose my framework not just based on performance, but also based on productivity. One that can help me write code that is easier to test and easier to maintain in the long run.
  • rorrr 14y ago
    > Now, imagine I created a third benchmark that did something more complex, like executing a database query and serving the results using the html/template package. There would be a negligible difference in performance across frameworks because routing is not going to be your bottleneck. If you're performing a DB query on every request, you're doing something wrong. In the real world your app will check Memcached, and if there's a cached response, it will return it. Thus making the framework performance quite important.
  • pifflesnort 14y ago
    rorr, you appear to be hellbanned. Here's your comment, since it seemed like a reasonable one: > Now, imagine I created a third benchmark that did something more complex, like executing a database query and serving the results using the html/template package. There would be a negligible difference in performance across frameworks because routing is not going to be your bottleneck. If you're performing a DB query on every request, you're doing something wrong. In the real world your app will check Memcached, and if there's a cached response, it will return it. Thus making the framework performance quite important.