18 ms·
Gotchas, Irritants and Warts in Go Web Development
- chimeracoder 14y ago> Go does not allow many basic types, such as strings or booleans, to be nil. Instead, when a type is initialized without a value, it defaults to the “zero value” for that type. This is frequently useful, but complicates database interactions, where null values are common. This sounds like they weren't defining their types properly. If your value can potentially be null, it should be a pointer to the type, not the type itself. A string can't be null, but a pointer to a string can. (In fact, there's no magic going on here - 'nil' is simply the zero value of a pointer. So you always get the zero value - you just need to choose the type that has the zero value you want... which is, in this case, a pointer, not a value). As explained well in this thread[0], this is the most accurate representation of the data itself. You could create your own type that automatically decodes all null values to whatever the zero value is (empty string, etc)., but then you lose that information. Yes, this forces you to do a check for the null value before using the data for the first time (or to invent your own monad for abstracting this), but at a high level, that's what you have to do in every language. [0] https://groups.google.com/forum/?fromgroups#!topic/golang-nuts/JOFWAqrTbUs https://groups.google.com/forum/?fromgroups#!topic/golang-nu...
- lbarrow 14y agoI think that's fair, and in retrospect we should have started with pointers to strings. If this is the best practice, though, we should change the standard library to reflect that, rather than providing NullStrings.
- chimeracoder 14y agoWell, it seems that they've just chosen the second option (no pun intended) that I've mentioned above - that is, to define a monad to abstract this. There may be some benefit to doing this instead of pointers, but as you've found yourself, using a monad requires you to pass that monadic type up the calling chain (OR to unwrap that and deal with both cases safely). I haven't used SQL databases with Go much, so I'm just looking at NullFoo types now for the first time, but every time I've thought I could improve on the design of Go in some way, the folks on #go-nuts have (kindly) shown me that I wasn't really thinking about things the right way. So, there's probably some reason that it's been designed this way, but I'm saying that purely based on prior experience with other Go matters, not with package "database/sql".
- abraininavat 14y agoIt's extremely generous... hell it's just wrong.. to call Go's NullString/NullBool/etc monads. "Monad" is not a synonym for "Wrapper type"
- Tyr42 14y agoWell, they do correspond to the "Maybe a" for some fixed type a, and Maybe is a monad, however you are right that it would be clearer to to say Maybe or Nullable or Option type, since that's what is meant, not some general monad.
- icey 14y agoThis is the approach I've taken for dealing with database nulls in go. I tried using NullString / NullTime / etc, but doing that required writing a custom JSON marshaller for every class with nullable data, which was a huge pain in the ass.
- chimeracoder 14y agoI don't know if you're using SQL or NoSQL databases, but if the latter (and your data is stored in JSON), you may find this package I wrote useful: https://www.github.com/ChimeraCoder/gojson https://www.github.com/ChimeraCoder/gojson Instead of unmarshalling to map[string]interface{} and then doing awkward type assertions and checks, etc., this will generate structs that you can unmarshal into. That way, if the unmarshalling succeeds, you're guaranteed to know the types of all your values instead of checking them each time. As for null values, it automatically uses a pointer in the struct definition if it finds a null value - writing that package was how I first learned about this entire topic, actually, which is why I mention it!
- pcwalton 14y agoUsing a pointer adds an extra heap allocation, though. The right thing would be to use a Nullable<T> type and define a custom marshaller once and for all... but Go doesn't have generics, so you can't do that.
- chimeracoder 14y ago> Using a pointer adds an extra heap allocation, though. If you want to dereference it immediately (as they seem to want to in the post), this isn't really going to affect you. You probably want to pass around a pointer, anyway, so that you're not copying values over each time. > The right thing would be to use a Nullable<T> type and define a custom marshaller once and for all... but Go doesn't have generics, so you can't do that. Sure you can - that's literally what the NullBool, etc. types do A pointer to a string IS a 'Nullable<String>' - what they need to do is define a way to unwrap this cleanly, which is easy.
- pcwalton 14y ago> If you want to dereference it immediately (as they seem to want to in the post), this isn't really going to affect you. You probably want to pass around a pointer, anyway, so that you're not copying values over each time. Sure, I'm not saying that the extra allocation will always matter. In most cases it won't. My point is just that this type system workaround does cost some performance. For example, a heap-allocated struct that contains two nullable ints using the pointer trick has 3 heap allocations, not one. > Sure you can - that's literally what the NullBool, etc. types do But that has to be done for each type. If you define a custom type Foo and want a nullable version, you have to write the NullFoo boilerplate yourself. This is what generics are for.
- chimeracoder 14y ago> If you define a custom type Foo and want a nullable version, you have to write the NullFoo boilerplate yourself. This is what generics are for. I don't want to descend down the rabbit hole of 'Go doesn't have generics', but as I said above, in this case, pointers do exactly what is needed.
- drbawb 14y agoTo avoid the nil pointer dereference panic, could you use an idiom borrowed from Go's map implementation? A library could take advantage of multiple assignment, a la maps: `i, ok := m["route"]` If the map has no entry for key "route", i will be the zero type [e.g: an empty string], and OK will be a bool indicating that the key wasn't found in the map. So in a SQL library it might look something like `func (s Statement) GetString(columnNum int) string,bool` Called as: `str,ok := s.GetString(i)` At least this way your program won't panic() if you forget to check that string != nil and dereference *string, but you still retain the information that the string is nil.
- chimeracoder 14y agoYes, that's actually more or less what NullString does. It's a struct that contains two fields: String and Valid (a boolean). It also implements a couple of other interfaces, but at its core, it's what you describe. Functionally, they're basically the same thing - it's only a syntactic difference whether you check whether a pointer is nil vs. checking whether a boolean is false. At some point, you either have to do a check or coerce a null value into a reasonable value. (Or, I guess, just risk it!). Functional programming purists may cringe at the analogy, but this is is all more or less the same functionality as Scala's 'Option' monad - it just depends on how you want to express it. The only difference is that the Go compiler doesn't force you to check against nil before dereferencing, but if you want to ensure the safety before converting the database value to a String and forgetting about it, then the struct will do that.
- pkulak 14y agoSeems like most issues (except the nil problem, which is explained very well by chimeracoder) are not really with the language itself, but just with its ecosystem, which naturally is nowhere near something like Java or Ruby at this point. I agree, and while I love Go, I would probably develop a new web app in Sinatra. Who knows if that will be the case next year, though.
- rartichoke 14y agoYeah and this is why I don't think it will ever become semi-main stream. I personally use Express with Node and it seems a good amount of people use Sinatra/Flask as well. It's much more than just that though. If you scanned Express feature by feature you could implement most of them in Go. I would imagine someone experienced in JS and Go could port the entire thing over in a day. It's just then what? Now you have: 1. Similar / worse performance. 2. Way less useful libs to leverage. 3. Have to write 2 languages instead of 1 (if you happen to already use Node). 4. Dealing with way less mature libs for crucial components that you definitely don't want to be writing yourself. There's basically no gains. Deploying single binaries is great but a "write once, use nearly forever" build script makes deployment a snap with any non-compiled language. Testing support is amazing in JS too and debugging is leap years ahead of Go. Using gdb is just archaic compared to using node-inspector. I'm sure Ruby and Python have equally as amazing testing/debugging support too.
- pkulak 14y agoInteresting that you are using Node.js as your example. Just a couple years ago Node was the new hotness, but was pretty terrible for creating an entire website.
- rartichoke 14y agoGo has been out for a number of years. It shipped v1.0 almost a year ago and Node hasn't shipped v1.0 but who cares about version numbers. Both languages have been available for quite some time. It's not about hotness at all. No one seems to WANT to use Go for serious web application development and this is proven time again by the lack of good libs available.
- Jabbles 14y agoI'm not an expert in databases, but a boolean that can take 3 values smells wrong to me. Is this regarded as best-practice?
- thibaut_barrere 14y agoIt really depends on what type of app you design. For data-entry apps, having nullable allowed for booleans is fairly common, it would mean "not provided" for instance.
- jcoby 14y agoNULL is one of the most powerful features of SQL. It simply means a lack of data. This means it is neither equal nor not equal to anything else (including NULL). NULL is not a value; it's a state of non-existence. So, for example, if you were providing a survey with an optional question with a yes/no answer. NULL would mean "no answer", false would mean "no", and true would mean "yes". Storing the "no answer" as a false would be incorrect since they did not answer the question. It could also happen if you were adding a new column. Existing rows do not have data and would deserve a NULL unless you had a deterministic way to fill in a true or false value.
- dragonwriter 14y ago> NULL is one of the most powerful features of SQL. I'd argue that NULL is, from a logical perspective, the single most broken feature of SQL. > It simply means a lack of data. The semantics of NULLs are less straightforward than that, and have a poor relationship to how SQL actually treats them. Every table with one or more nullable columns really should be a table with all the non nullable columns, plus additional table with each combination of columns that would never be missing together, each of which has a foreign key relationship back to the first table. That's for the simple case, where the semantics of missing data are always consistent for any set of columns; in real-world databases there are often more than one reason data that can be missing might be missing, and those different reasons (because they are different classes of fact), for any given column or set of columns to which they apply, each call for another table with a foreign key reference to the table containing only the mandatory columns. > So, for example, if you were providing a survey with an optional question with a yes/no answer. NULL would mean "no answer", false would mean "no", and true would mean "yes". Storing the "no answer" as a false would be incorrect since they did not answer the question. Sure, storing it as one table with all the questions as columns and storing the "no" answer when the answer was missing would be an error. If all the questions aren't required for the survey to be valid, then -- from a logical perspective -- the problem is presenting the whole thing as a single relation in the first place. Its a set of relations, that share a key (but not necessarily all values of the key.)
- grey-area 14y agoGreat post. I've done some experimentation with Go for web dev, and encountered similar problems; it's such a delight to write code in that I'd love to use it in production, but can't justify spending a lot of time debugging the immature libraries and writing new ones, particularly not on the clock for clients. I too ran into a problem with null types in another library interfacing with postgresql. The weak points I found were: * Lack of an ORM (there are many, almost all incomplete, and lacking in some way) * Immature or incomplete db interface libraries * No db migrations (would love to see a simple sql-only solution here) * Package management is simple and elegant but without explicit versioning, so forking is the end result to ensure stability * No process pool management for running Go behind something like Apache or nginx. This goagain tool looks interesting for the last point though: https://github.com/rcrowley/goagain https://github.com/rcrowley/goagain and the routing I found pretty straightforward with something like github.com/gorilla/mux so there are solutions, they're just not always as fully baked or as mature as you'd find in other ecosystems. One place Go shines though is the great built in web server which really simplifies getting up and running and testing, and which I see they've built on here. Given how easy and pleasurable it is to work with, and the focus on practicality rather than language features, I'm quite confident that Go will reach it's stated aim of becoming a popular server-side language quite soon. For those new to the philosophy behind the language, I found this informative - http://talks.golang.org/2012/splash.article http://talks.golang.org/2012/splash.article
- jcoby 14y agoI went on a framework binge over the past two weeks to see what the state of web development was. I looked at a dozen or so frameworks including Go frameworks. I came to the same conclusion you did. Revel was the most promising Go web framework that I saw though very immature. Go ORMs aren't ORMs. They are object persistence frameworks (at best). None of them really provide the "R" in ORM. Most of them basically do "SELECT * FROM table WHERE id=12" and dump that into a struct. There goes the "M" as well. The Go language itself is very nice to work with though. It's a great language. Give it 2-3 more years and a truly usable ORM and web framework will crop up.
- NateDad 14y ago
- bfrog 14y agoI basically encountered the same annoyances, primarily with the whole NullBool, NullString business. Go is pretty good at most things, dealing with SQL so far was a bit annoying however. The Goroutine scheduler also needs some love still. The Garbage Collector and global heap I feel like is always a bad idea but Go did it anyways. Shouldn't they have learned from Java that global heaps and concurrency don't mix well?
- JulienSchmidt 14y agoThere will be a new scheduler and an improved GC in Go 1.1, which will be released soon.
- JulienSchmidt 14y agoThe timeout issue was fixed in Go 1.0.3. The problem wasn't that the connections timed out, but that the database/sql package wasn't able to handle this correctly. http://code.google.com/p/go/source/detail?r=b397807815a6 http://code.google.com/p/go/source/detail?r=b397807815a6 Please keep in mind, that the database/sql package is - like Go itself - still very young. There is still a lot of work to do compared to mature libraries like JDBC, but it will improve with every release.
- pico303 14y agoYeah, like the fact two queries can't run simultaneously in a single transaction. All queries in an single transaction must be run sequentially. What worries me though is that this is "by design", and seen as completely proper by the Go devs. That's the reason I stopped using Go: they have guys who don't understand databases writing their database code.
- TallboyOne 14y agoI'm 100% a ruby fan, but it seems crazy they went from Go to ruby if ultimate performance isn't their goal. Wouldnt a static type language be worlds faster?
- pkulak 14y agoGo is pretty slow right now. I wrote a prime number finder in Go and Ruby 1.9 and Ruby smoked it. Granted that's probably mostly due to all the math in Ruby being C, but still, you can't just say that Go is faster because it's compiled before hand.
- NateDad 14y agoHonestly, my guess is that you were doing something wrong in the Go code. Go is definitely faster than Ruby, even on a single core machine. http://benchmarksgame.alioth.debian.org/u64/benchmark.php?test=all&lang=go&lang2=yarv http://benchmarksgame.alioth.debian.org/u64/benchmark.php?te...
- pkulak 14y agoEDIT: haha, thanks so much, guys! Stupid mistake. Go is _significantly_ faster if written properly. I'd love to know what I screwed up. The results surprised me: https://gist.github.com/pkulak/99d2d5a5968b5fc03754 https://gist.github.com/pkulak/99d2d5a5968b5fc03754 https://gist.github.com/pkulak/909d882614e0781e4525 https://gist.github.com/pkulak/909d882614e0781e4525 bender:Desktop phil$ time go run primes.go Found them! 78702 real 0m21.349s user 0m21.304s sys 0m0.033s bender:Desktop phil$ ruby -v ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-darwin11.4.0] bender:Desktop phil$ time ruby primes.rb Found them! 78702 real 0m7.656s user 0m7.651s sys 0m0.005s bender:Desktop phil$ time go build primes.go real 0m0.269s user 0m0.231s sys 0m0.033s
- hndc 14y agoYou're using integer arithmetic in Ruby and floating-point arithmetic in Go. Try replacing math.Mod(float64(i), float64(j)) with i%j My results: $ time go run primes.go Found them! 78702 real 0m0.835s user 0m0.787s sys 0m0.039s $ time ruby primes.rb Found them! 78702 real 0m21.013s user 0m21.005s sys 0m0.016s
- deleted 14y ago[deleted]
- trungonnews 14y agoYou should try out Play2 on Scala if Go doesn't work for you.