6 ms·

Thoughts on Go after writing 3 websites

153 points by kkowalczyk 14y ago

chewxy 14y ago
And for those of you who are not cofortable running with net/http (it is a rather simple server), there is always uWSGI support for Go, which I found to be a rather brilliant idea
- jinushaun 14y ago
  Running directly off net/http? Isn't that just Node.js? People are comfortable with that.
  - realrocker 14y ago
    In the sense of your statement, people are uncomfortable with compulsory concurrency for everything in Node.js off http not http itself. Though you have to look at the package api yourself to decide whether it's running directly or not.
  - chewxy 14y ago
    Uh... no. Ruby on Rails runs off a ruby webserver. Flask has its own built in webserver, web.py has its own built in webserver. Tho these are not languages, rather frameworks, (and in the case of web.py, a library), they have built in webservers My point? These things came way before node.js, so no, node.js isn't the only one with a built in webserver. The difference is node.js has a rather performant built in webserver (still, exeperience shows that a properly tuned gevent or netty beats a properly tuned node.js). The idea is that instead of spending time tuning these things, you can just wrap it in a uWSGI container, and let better tuned-out-of-the-box servers like nginx handle the webserving
    - gary4gar 14y ago
      even php has built-in webserver too! http://php.net/manual/en/features.commandline.webserver.php http://php.net/manual/en/features.commandline.webserver.php
- kkowalczyk 14y ago
  (I wrote the article and 3 web apps on Go). I would be much more uncomfortable running on uWSGI than on net/http. As far as I can tell (https://github.com/unbit/uwsgi/blob/master/plugins/go/src/uwsgi/uwsgi.go https://github.com/unbit/uwsgi/blob/master/plugins/go/src/uw...) Go's uWSGI implementation is just a small wrapper over C code inside a dll. It's not portable (support for cgo on Windows is extremely sketchy). It also goes against common sense. Any code you can write in C, you can write in Go, except it'll be easier (Go is garbage collected, has built-in arrays, strings, hash tables etc.) and there's drastically smaller chance Go will have a security issues due to memory over-write. It duplicates lots of net/http functionality. net/http is well documented, well tested, reviewed by extremely knowledgeable people. Most importantly, it is being used in production by me and many other people. Finally, net/http gave me everything I needed to write my web applications. Before I would even considering using uWSGI (or anything else) someone would have to explain what exactly does it provide that is so indispensable for writing web apps.
  - mitchi 14y ago
    You have basically just described the advantages of the D language too. Do you plan on trying D?
    - kaib 14y ago
      I've written some hundred thousand lines of production code in Go and D and feel they are very different languages. To sum it up: Go is a better C while D is a better C++.
    - kkowalczyk 14y ago
      I did try D when it was in its early days (i.e. D 1) but at the time I was put off by lack of documentation, the standard library split (phobos vs. D) and general immaturity of the system. I'm sure things have improved since then but I really like Go's simplicity.
  - unbit 14y ago
    If net/http (or whatever http wrapper in whatever language) gives you enough batteries for a web application i am really happy for you. Sadly me (and a lot of other people) need a lot more. That is why the uWSGI project exists. Things like logging, clustering, statistics, multi protocol support, alarms, time events, request routing and blah blah are added to your app for free (and the funny thing is that you can tune lot of internals without rebuilding your app). Obviously if not having windows support is a too much high price for you the discussion ends here :)
- zerohp 14y ago
  Run nginx or lighttpd as a reverse proxy in front of net/http.
chimeracoder 14y ago
> If you know the shape of [JSON] data, you can serialize (marshal, in Go’s parlance) data structures to JSON or XML and de-serialize (unmarshal) from JSON or XML into a struct. I wrote a short script to automate generating the struct definitions, since I didn't want to generate the structs by hand. Go's reflection made this really easy. https://github.com/chimeracoder/gojson https://github.com/chimeracoder/gojson
- realrocker 14y ago
  Nice, I was about to write one tomorrow.
  - chimeracoder 14y ago
    If you're interested, there's still some low-hanging fruit left, namely in two edge cases: 1. It assumes all slices to be of type []interface{} by default, though oftentimes a more precise type is possible. (This is just a matter of looping through and determining the most specific type) 2. It only accepts valid JSON. Not a bug, but for example, the sample responses on the foursquare API documentation omit the (required) quotation marks around identifiers: https://developer.foursquare.com/docs/venues/venues https://developer.foursquare.com/docs/venues/venues (click "Try it Out"). That's a common problem which I'd like to fix eventually.
- lucian303 14y ago
  PHP has json_decode() and json_encode() which do all of this correctly ... GO is interesting, but production ready? Not for awhile.
  - munificent 14y ago
    PHP is dynamically typed, Go isn't.
    - lucian303 14y ago
      And this really has no bearing on a discussion of JSON, a data exchange format.
hvs 14y ago
I've just worked my way through the Go Tour and have begun working on my first real project in it (a basic wiki, for learning purposes). At first, I was expecting C with some syntax changes, but the changes made feel like they were almost entirely made for legitimate reasons. Types definitions after variable names really does make sense when you start to use it. Even with my limited experience in it, I can say that it is really a joy to develop in.
lazyjones 14y ago
Thanks for posting this, the code is very readable and good for learning. How much time did you spend on Go before you finished these projects?
- kkowalczyk 14y ago
  I can't really quantify it as it was in drips but in general, since I already knew C and Python, learning Go was easy. I could have started to write a web app right away, following tutorials on Go website and elsewhere. At the beginning it's frustratingly slow, as it always is when you learn something new, but if you know other C-like languages, picking up Go will be fast. You just have to put the time to learn standard library and Go way of doing some things.
  - mitchi 14y ago
    I have some questions. Are you on Linux or Windows? Did you have to use a debugger or no need?
    - kkowalczyk 14y ago
      I develop on Mac, deploy to Linux (Ubuntu). Didn't have to use a debugger so far.
    - rartichoke 14y ago
      Go has full support for gdb if you want a debugger. You can even hook into go routines.
graue 14y ago
The links for securecookie and mux links are down (redirecting to https://www.gorillatoolkit.org/ https://www.gorillatoolkit.org/, which doesn't respond).
- BarkMore 14y ago
  The documentation is also available on GoDoc: http://godoc.org/github.com/gorilla/securecookie http://godoc.org/github.com/gorilla/securecookie http://godoc.org/github.com/gorilla/mux http://godoc.org/github.com/gorilla/mux
- Benferhat 14y ago
  Try without SSL: http://www.gorillatoolkit.org/ http://www.gorillatoolkit.org/ Or on GitHub: https://github.com/gorilla https://github.com/gorilla
eikenberry 14y ago
I don't understand the deployment section. Go has the simplest deployment I've ever had the pleasure to use. You compile the binary and copy it up. Static binaries make it so simple as to be trivial.
- skrebbel 14y ago
  What about dependencies? Client-side resources such as .js and .css files, configuration files maybe, migrating the data even? A web app is (virtually) never just its server side code.
  - alecthomas 14y ago
    I wrote gobundle[1] as one solution to this. [1] https://github.com/alecthomas/gobundle https://github.com/alecthomas/gobundle
    - StavrosK 14y ago
      That's very interesting! Do you just append the bundles to the executable and then have it read itself? I didn't look at the code, admittedly.
      - alecthomas 14y ago
        It generates Go code which is compiled in to the package.
  - eikenberry 14y ago
    The original article didn't discuss deploying assets, only the app/code. I suppose that might have been implicit, but I typically handle them separately.
- kkowalczyk 14y ago
  First, if you use C libraries, you can't just compile on OS A and copy to OS B. I'm not 100% sure but while Go has much better cross-compiling story than others, I believe net/http does uses OS's C library and therefore cannot be cross-compiled that easily. Also, you have to copy the assets (templates and such) and if you're deploying a new version of a web app, you don't just overwrite the binary. My script makes sure that if something goes wrong (i.e. a new version doesn't work or has a bug), it's easy for me to revert to previous version. That being said, none of that implies that Go makes deployment harder (or easier, for that matter) than deploying code in other languages would be. It's pain in the ass but nothing that I couldn't solve in a day with a script written with Fabric.
Benferhat 14y ago
Is 7776000 pageviews/month the kind of traffic that you'd expect to be able to generate $80/month? $0.00001/pageview/month? If the average user viewed 1,000 pages a month, that would be $0.01/user/month. Obviously it depends on the average pageviews per user, but what's your gut feeling? Should $80 have been too expensive?
- charliesome 14y ago
  $80 is FAR too expensive for 3 requests per second. If you're paying that much, you've been taken for a ride. That's ridiculous.
  - Benferhat 14y ago
    Thanks, turns out he manually set it to 4 always-on instances, so we don't know what the GAE scheduler would have done.
- kkowalczyk 14y ago
  Your math, while precise, is meaningless. I probably could have gotten away with paying $0 if I didn't care about the best performance. App Engine is distributed. You get one front-end instance for free. The more front-end instances you spin up, the lower the latency for your requests (consider a fron-end instance a process, the more processes you have, the more requests you can handle in parallel, which lowers latency for each request). All the money was spent on additional (3, I believe, front-end instances). Was the price high? Yes, which is one reason I switched to a $60/month kimsufi server which hosts the blog and still has 99% of capacity free for other things. It's possible I was over-paying. I'm not an App Engine expert. I picked the number of instances more on a gut-basis than a comprehensive analysis basis. I could afford it so it wasn't a big deal.
  - Benferhat 14y ago
    Ah, I thought GAE was spinning those instances up for you automatically. That's the default, it spins up extra instances as-needed to keep latency down. With Go apps, that's generally down to i/o from API calls.
jack9 14y ago
What a ringing anti-endorsement. The author seems...retarded. - Go is a good language for building websites/web servers. Uh those are completely different goals, but lets see if we can see how he ties it together. 1. Databases: I wrote my own implementation in Go. (to file) Just wow. 2. RSS Feeds: I wrote my own implementation in Go. Of course you did, because Go's libraries have gaps. That's what you should have said. 3. S3/Zip: I use files, so this was trivial. How is this a common problem (it isn't) and why did you mention it at all? 4. Deployment: Always difficult. What is wrong with this guy. 5. Server Config: Standard. Thanks for telling us nothing except that you split 3 webservices over different ports and tied them together on the same ngix webserver. That's not awkward at all. Wait, didn't we start with the statement that Go was suited for building webservers...nevermind. Nowhere in here did I gleam why I would choose to use Go or even RISK it.

153 points by kkowalczyk 14y ago

And for those of you who are not cofortable running with net/http (it is a rather simple server), there is always uWSGI support for Go, which I found to be a rather brilliant idea

jinushaun 14y ago
Running directly off net/http? Isn't that just Node.js? People are comfortable with that.
- realrocker 14y ago
  In the sense of your statement, people are uncomfortable with compulsory concurrency for everything in Node.js off http not http itself. Though you have to look at the package api yourself to decide whether it's running directly or not.
- chewxy 14y ago
  Uh... no. Ruby on Rails runs off a ruby webserver. Flask has its own built in webserver, web.py has its own built in webserver. Tho these are not languages, rather frameworks, (and in the case of web.py, a library), they have built in webservers My point? These things came way before node.js, so no, node.js isn't the only one with a built in webserver. The difference is node.js has a rather performant built in webserver (still, exeperience shows that a properly tuned gevent or netty beats a properly tuned node.js). The idea is that instead of spending time tuning these things, you can just wrap it in a uWSGI container, and let better tuned-out-of-the-box servers like nginx handle the webserving
  - gary4gar 14y ago
    even php has built-in webserver too! http://php.net/manual/en/features.commandline.webserver.php http://php.net/manual/en/features.commandline.webserver.php
kkowalczyk 14y ago
(I wrote the article and 3 web apps on Go). I would be much more uncomfortable running on uWSGI than on net/http. As far as I can tell (https://github.com/unbit/uwsgi/blob/master/plugins/go/src/uwsgi/uwsgi.go https://github.com/unbit/uwsgi/blob/master/plugins/go/src/uw...) Go's uWSGI implementation is just a small wrapper over C code inside a dll. It's not portable (support for cgo on Windows is extremely sketchy). It also goes against common sense. Any code you can write in C, you can write in Go, except it'll be easier (Go is garbage collected, has built-in arrays, strings, hash tables etc.) and there's drastically smaller chance Go will have a security issues due to memory over-write. It duplicates lots of net/http functionality. net/http is well documented, well tested, reviewed by extremely knowledgeable people. Most importantly, it is being used in production by me and many other people. Finally, net/http gave me everything I needed to write my web applications. Before I would even considering using uWSGI (or anything else) someone would have to explain what exactly does it provide that is so indispensable for writing web apps.
- mitchi 14y ago
  You have basically just described the advantages of the D language too. Do you plan on trying D?
  - kaib 14y ago
    I've written some hundred thousand lines of production code in Go and D and feel they are very different languages. To sum it up: Go is a better C while D is a better C++.
  - kkowalczyk 14y ago
    I did try D when it was in its early days (i.e. D 1) but at the time I was put off by lack of documentation, the standard library split (phobos vs. D) and general immaturity of the system. I'm sure things have improved since then but I really like Go's simplicity.
- unbit 14y ago
  If net/http (or whatever http wrapper in whatever language) gives you enough batteries for a web application i am really happy for you. Sadly me (and a lot of other people) need a lot more. That is why the uWSGI project exists. Things like logging, clustering, statistics, multi protocol support, alarms, time events, request routing and blah blah are added to your app for free (and the funny thing is that you can tune lot of internals without rebuilding your app). Obviously if not having windows support is a too much high price for you the discussion ends here :)
zerohp 14y ago
Run nginx or lighttpd as a reverse proxy in front of net/http.

chimeracoder 14y ago

> If you know the shape of [JSON] data, you can serialize (marshal, in Go’s parlance) data structures to JSON or XML and de-serialize (unmarshal) from JSON or XML into a struct. I wrote a short script to automate generating the struct definitions, since I didn't want to generate the structs by hand. Go's reflection made this really easy. https://github.com/chimeracoder/gojson https://github.com/chimeracoder/gojson

realrocker 14y ago
Nice, I was about to write one tomorrow.
- chimeracoder 14y ago
  If you're interested, there's still some low-hanging fruit left, namely in two edge cases: 1. It assumes all slices to be of type []interface{} by default, though oftentimes a more precise type is possible. (This is just a matter of looping through and determining the most specific type) 2. It only accepts valid JSON. Not a bug, but for example, the sample responses on the foursquare API documentation omit the (required) quotation marks around identifiers: https://developer.foursquare.com/docs/venues/venues https://developer.foursquare.com/docs/venues/venues (click "Try it Out"). That's a common problem which I'd like to fix eventually.
lucian303 14y ago
PHP has json_decode() and json_encode() which do all of this correctly ... GO is interesting, but production ready? Not for awhile.
- munificent 14y ago
  PHP is dynamically typed, Go isn't.
  - lucian303 14y ago
    And this really has no bearing on a discussion of JSON, a data exchange format.

hvs 14y ago

I've just worked my way through the Go Tour and have begun working on my first real project in it (a basic wiki, for learning purposes). At first, I was expecting C with some syntax changes, but the changes made feel like they were almost entirely made for legitimate reasons. Types definitions after variable names really does make sense when you start to use it. Even with my limited experience in it, I can say that it is really a joy to develop in.

lazyjones 14y ago

Thanks for posting this, the code is very readable and good for learning. How much time did you spend on Go before you finished these projects?

kkowalczyk 14y ago
I can't really quantify it as it was in drips but in general, since I already knew C and Python, learning Go was easy. I could have started to write a web app right away, following tutorials on Go website and elsewhere. At the beginning it's frustratingly slow, as it always is when you learn something new, but if you know other C-like languages, picking up Go will be fast. You just have to put the time to learn standard library and Go way of doing some things.
- mitchi 14y ago
  I have some questions. Are you on Linux or Windows? Did you have to use a debugger or no need?
  - kkowalczyk 14y ago
    I develop on Mac, deploy to Linux (Ubuntu). Didn't have to use a debugger so far.
  - rartichoke 14y ago
    Go has full support for gdb if you want a debugger. You can even hook into go routines.

graue 14y ago

The links for securecookie and mux links are down (redirecting to https://www.gorillatoolkit.org/ https://www.gorillatoolkit.org/, which doesn't respond).

BarkMore 14y ago
The documentation is also available on GoDoc: http://godoc.org/github.com/gorilla/securecookie http://godoc.org/github.com/gorilla/securecookie http://godoc.org/github.com/gorilla/mux http://godoc.org/github.com/gorilla/mux
Benferhat 14y ago
Try without SSL: http://www.gorillatoolkit.org/ http://www.gorillatoolkit.org/ Or on GitHub: https://github.com/gorilla https://github.com/gorilla

eikenberry 14y ago

I don't understand the deployment section. Go has the simplest deployment I've ever had the pleasure to use. You compile the binary and copy it up. Static binaries make it so simple as to be trivial.

skrebbel 14y ago
What about dependencies? Client-side resources such as .js and .css files, configuration files maybe, migrating the data even? A web app is (virtually) never just its server side code.
- alecthomas 14y ago
  I wrote gobundle[1] as one solution to this. [1] https://github.com/alecthomas/gobundle https://github.com/alecthomas/gobundle
  - StavrosK 14y ago
    That's very interesting! Do you just append the bundles to the executable and then have it read itself? I didn't look at the code, admittedly.
    - alecthomas 14y ago
      It generates Go code which is compiled in to the package.
- eikenberry 14y ago
  The original article didn't discuss deploying assets, only the app/code. I suppose that might have been implicit, but I typically handle them separately.
kkowalczyk 14y ago
First, if you use C libraries, you can't just compile on OS A and copy to OS B. I'm not 100% sure but while Go has much better cross-compiling story than others, I believe net/http does uses OS's C library and therefore cannot be cross-compiled that easily. Also, you have to copy the assets (templates and such) and if you're deploying a new version of a web app, you don't just overwrite the binary. My script makes sure that if something goes wrong (i.e. a new version doesn't work or has a bug), it's easy for me to revert to previous version. That being said, none of that implies that Go makes deployment harder (or easier, for that matter) than deploying code in other languages would be. It's pain in the ass but nothing that I couldn't solve in a day with a script written with Fabric.

Benferhat 14y ago

Is 7776000 pageviews/month the kind of traffic that you'd expect to be able to generate $80/month? $0.00001/pageview/month? If the average user viewed 1,000 pages a month, that would be $0.01/user/month. Obviously it depends on the average pageviews per user, but what's your gut feeling? Should $80 have been too expensive?

charliesome 14y ago
$80 is FAR too expensive for 3 requests per second. If you're paying that much, you've been taken for a ride. That's ridiculous.
- Benferhat 14y ago
  Thanks, turns out he manually set it to 4 always-on instances, so we don't know what the GAE scheduler would have done.
kkowalczyk 14y ago
Your math, while precise, is meaningless. I probably could have gotten away with paying $0 if I didn't care about the best performance. App Engine is distributed. You get one front-end instance for free. The more front-end instances you spin up, the lower the latency for your requests (consider a fron-end instance a process, the more processes you have, the more requests you can handle in parallel, which lowers latency for each request). All the money was spent on additional (3, I believe, front-end instances). Was the price high? Yes, which is one reason I switched to a $60/month kimsufi server which hosts the blog and still has 99% of capacity free for other things. It's possible I was over-paying. I'm not an App Engine expert. I picked the number of instances more on a gut-basis than a comprehensive analysis basis. I could afford it so it wasn't a big deal.
- Benferhat 14y ago
  Ah, I thought GAE was spinning those instances up for you automatically. That's the default, it spins up extra instances as-needed to keep latency down. With Go apps, that's generally down to i/o from API calls.

jack9 14y ago

What a ringing anti-endorsement. The author seems...retarded. - Go is a good language for building websites/web servers. Uh those are completely different goals, but lets see if we can see how he ties it together. 1. Databases: I wrote my own implementation in Go. (to file) Just wow. 2. RSS Feeds: I wrote my own implementation in Go. Of course you did, because Go's libraries have gaps. That's what you should have said. 3. S3/Zip: I use files, so this was trivial. How is this a common problem (it isn't) and why did you mention it at all? 4. Deployment: Always difficult. What is wrong with this guy. 5. Server Config: Standard. Thanks for telling us nothing except that you split 3 webservices over different ports and tied them together on the same ngix webserver. That's not awkward at all. Wait, didn't we start with the statement that Go was suited for building webservers...nevermind. Nowhere in here did I gleam why I would choose to use Go or even RISK it.