9 ms·

Golang Diaries II

96 points by espadrine 13y ago

espadrine 13y ago
Of course, anyone can make a Map function for any particular type combination, but Tim Bray has a point. The generic, functional map function cannot be written, and never will. There is an issue of elegance. If generics were to be added, lists and maps would need to have similar syntax, because they feel and behave like generics. However, they cannot. Their syntax is… not generic enough. Oddly enough, elegance through consistency was never reached when designing built-in functions like make(): those can be applied on any type, but user-defined functions don't have this freedom. Essentially, there is a Java-like divide between built-in types (which, for instance, you cannot create methods on) and user types. Their rules of elegance don't apply to the built-ins, which is quite odd. I doubt that this impedes on programming efficiency compared to C. However, some of the duplicated code out there cannot be factored into a library, and map is an example of that.
- deleted 13y ago
  [deleted]
- pdeuchler 13y ago
  I took a couple minutes to whip up a quick map function, to be honest I didn't quite believe you. This may be my naivete, but is there any way it's not sufficiently generic for types? I realize it's not suitable for maps, but it satisfies the OP's request for a map that can convert arrays of type Foo to type Bar http://play.golang.org/p/jxMFq5UYs1 http://play.golang.org/p/jxMFq5UYs1
  - burntsushi 13y ago
    Here is something that is less onerous on the caller, but probably slower. [1] Here's the source. [2] [1] - https://github.com/BurntSushi/ty#examples https://github.com/BurntSushi/ty#examples [2] - https://github.com/BurntSushi/ty/blob/master/fun/list.go#L17 https://github.com/BurntSushi/ty/blob/master/fun/list.go#L17
  - drewolson 13y ago
    I think the point is, if you write map like this you might as well be using a dynamically typed language in the first place because you're deriving no benefit from the type checker. In fact, you're just using casting to completely circumvent it.
    - pdeuchler 13y ago
      Of course. If the main cornerstone of your project is map it's probably not the greatest idea to use Go, but if you're using Go for other reasons and happen to need a quick map function there's no reason to change languages just because Go doesn't have a standard library implementation
      - bad_user 13y ago
        But then it's pretty odd to use Go because you like the language, while not being able to design a proper map(). And map() btw is only the tip of the iceberg in functional programming. But then again, Go is not a functional programming language.
  - pcwalton 13y ago
    "I doubt that this impedes on programming efficiency compared to C." will not hold true if you're using reflection.
- dragonwriter 13y ago
  > The generic, functional map function cannot be written, and never will. https://github.com/droundy/gotgo/blob/master/example/slice.got https://github.com/droundy/gotgo/blob/master/example/slice.g...
comex 13y ago
> Isn’t it pathetic that so many allegedly-mainstream languages don’t have HTTP libraries that are this natural and idiomatic? That's after 13 SLoC to make a single HTTP request - I like Go, but that seems almost like a parody. It's not that different from other languages, and the error handling makes it look longer, but for something described as "natural and idiomatic" I would expect it to be one function call plus one failure test, not six and three...
- cmccabe 13y ago
  Experiencing a networking error is different than receiving Status 200 (or any other error status), which is different than getting a media-type different than "application/json". Personally, I don't see what would be gained by combining them all together in a library, except confusion. A lot of libraries do get this wrong and make it difficult to disentangle what is going on. If you feel like golang needs one method to do all the above, feel free to write one and post it to the mailing list... I doubt that it does, but feel free.
  - comex 13y ago
    Depends on the use case. I'd say that for many, many cases, all errors should lead to the same result in the application, but with an error-specific informative message. Even if you want to have special handling for errors such as 4xx that might mean something is permanently wrong, getting a 500 is the same as experiencing a network error (except in a client side app where the latter might mean you're not connected to the Internet and should try again, but then you should use yet another API to determine when to connect), and getting a 200 with the wrong content-type has an excellent chance of meaning the same thing as a network error, in the form of a captive portal. To this extent, APIs with many separate error paths that encourage the user to write their own nonstandard error messages are harmful, because they will probably do worse explaining things than with a common method of triage. Of course there are other cases where the difference matters. It just... doesn't seem very "natural and idiomatic" to me, that's all.
- throwit1979 13y ago
  When people complain about comprehensive error-checking, it makes my blood boil. Checking and handling all possible error conditions is a GOOD thing.
cmccabe 13y ago
With regard to the functional-style "map" function, is it really that much less typing than a simple for loop? output := make(output_type, len(input)) for idx, e := range(input) { output[idx] = convert(e) } compared to: output := map(input, func(in input_type) output_type { return convert(in) } ) There are cases where functional programming can be a lot terser (for better and worse), but I don't think this is one of them. My guess is that the original poster is just adjusting to different ways of doing things between Golang and Ruby-- that's natural. My biggest complaint about Go so far is that you have to use an external library to get an ordered tree structure. Maps suffice most of the time, but sometimes you need ordering. [edit: fixed map example]
- comex 13y ago
  The second example should be output := map(convert, input) After all, map generally handles fetching the element from the array.
- pcwalton 13y ago
  > There are cases where functional programming can be a lot terser (for better and worse), but I don't think this is one of them. That's only true in Go because Go doesn't have type inference for closures. In ML for example this would be: let output = map (fn i -> convert i) input or, more succinctly: let output = map convert input
  - cmccabe 13y ago
    Yes, that's true. I just wanted to address the specific suggestion that the original author made. Languages make a whole set of related design choices and typically none of them makes sense without considering the others. (Sometimes none of them makes sense AFTER considering the others, but that's another story.)
- jacques_chester 13y ago
  Loops impose, and guarantee, ordered traversal of a collection. Maps do not. This means that maps can sped up by dividing up the collection into smaller parts and mapping each of them in parallel. For a language that has concurrency as a central design motivation, it's an unfortunate omission. As we say in the database world: think in sets, not in arrays.
jlujan 13y ago
Screw map, let me pass a comparator function into sort for interfaces.
- drivebyacct2 13y ago
  Not sure you understand how the sort interface works...
- cmccabe 13y ago
  That's not necessary. You can just create a type that embeds the type you're sorting, and define the Less function however you like. It's very efficient since you're just wrapping the entire slice object, not every element in the slice. package main import "sort" type RevIntSlice struct { sort.IntSlice } func (arr RevIntSlice) Less(i, j int) bool { return arr.IntSlice.Less(j, i) } func main() { myList := RevIntSlice { sort.IntSlice {1, 2 , 3 } } sort.Sort(myList) for _, elem := range(myList.IntSlice) { println(elem) } } Note that if all you want to do is reverse the sort order, sort.Reverse is a better way than what I wrote here. This is just an example of the kinds of things you can do.
  - chongli 13y ago
    >That's not necessary. You can just create a type that embeds the type you're sorting But that's more work (and more boilerplate) than simply passing an anonymous comparator to the sort function.
    - drivebyacct2 13y ago
      Which isn't possible at the language level, so why are we even discussing impl with jlujan's shitty attitude.
    - cmccabe 13y ago
      Let me guess. You are coming from Java. In Java this would be: myArray.sort(new Comparator<MyClass> ({ @Override CompareTo(MyClass lhs, MyClass rhs) { return rhs.compare(lhs); } })); Is that really shorter? I don't think so. In any case, there is a more elegant solution in Go, which is simply: sort.Sort(sort.Reverse(array)) You need to open your mind a little bit, to learn a new language.
buro9 13y ago
> Isn’t it pathetic that so many allegedly-mainstream languages don’t have HTTP libraries that are this natural and idiomatic? But at every point where Tim then returns a failure we lose insight as to why something failed. My preferred idiom for handling all errors when it comes to web requests or responses is to return both the error (as it has the detailed message of what happened) and the http status code that matches the error (as it has translated the message into a code that describes the type of error). This bit would change: if !strings.HasPrefix(mediaType, "application/json") { // bogus media type, deal return failure } to: if !strings.HasPrefix(mediaType, "application/json") { // bogus media type, deal return HttpErr{Err: errors.New(""), Status: http.StatusUnsupportedMediaType} } Whatever receives that knows there has been an error, and has the code available to do tell the developer or end user something meaningful without relying on the error message. I'm sure others are also creating their own http error as well, so whilst Go does http really well it seems in practise that there are still a few places where it could benefit from a little extra to help standardise the code we write. All of those Http codes are here: http://golang.org/pkg/net/http/#pkg-constants http://golang.org/pkg/net/http/#pkg-constants Regarding the map stuff, the only other place the lack of generics has stood out has been within slices of a type of thing. The scenario is: I have a slice of identifiers in an order important to me, for each of these concurrently fetch the record being identified and create a slice of those records ordered as per the initial slice. We've ended up using this type of thing: // Envelope for making sorted concurrent requests for Comment type CommentRequest struct { Comment Comment Err error Seq int } type CommentRequestBySeq []CommentRequest func (v CommentRequestBySeq) Len() int { return len(v) } func (v CommentRequestBySeq) Swap(i, j int) { v[i], v[j] = v[j], v[i] } func (v CommentRequestBySeq) Less(i, j int) bool { return v[i].Seq < v[j].Seq } // Much later in the code... // Make a request for each id req := make(chan CommentRequest) defer close(req) for seq, id := range ids { go HandleCommentRequest(id, seq, req) } resps := []CommentRequest{} for i := 0; i < len(ids); i++ { resp := <-req if resp.Err != nil { return resp.Err } resps = append(resps, resp) } // Sort them sort.Sort(CommentRequestBySeq(resps)) // Extract the values ems := []Comment{} for _, resp := range resps { ems = append(ems, resp.Item) } In that example a Comment is a comment made on a forum web page and we're fetching lots concurrently, but needing to return them in the order that they will be displayed. The problem stems from that envelope, and the need to implement that sort function for every type that we ever want to concurrently fetch and sort. It's the only place where I think, if we just had generics this could be a lot better.
dscrd 13y ago
Just recently, I needed a concurrent map for my project. I looked for a library and started using it. Fast forward a bit and I realized the map was completely untyped. Fast forward a slightly longer bit and I realized (perhaps wrongly? Enlighten me) that there was no way to implement a concurrent map library that wasn't untyped. We need generics, or perhaps it's more goy to have the same features by some more clever way. But still. We need those features and bad. I'll be looking at Rust with some more interest meanwhile.
ExpiredLink 13y ago
if ... { return failure } three(!) times in a row and then he writes: > Isn’t it pathetic that so many allegedly-mainstream languages don’t have HTTP libraries that are this natural and idiomatic? Isn’t it pathetic that a wannabe mainstream language doesn’t provide exception handling?
- jameskilton 13y ago
  Exceptions are horrible if you're trying to make a fast, dependable, reliable, low-level, concurrent modern day programming language, as Go is. Exceptions are also horrible in that they never improve the readability and understandability of any code they exist in. The Go team, some of whom actually designed C back in the day, are well familiar with Exceptions, why they are bad, and explicitly chose to not implement them. So no, using the word "pathetic" implies a gross oversight. This was a very deliberate decision, and a good one at that.
  - Fistandantilus 13y ago
    > Exceptions are horrible if you're trying to make a fast, dependable, reliable, low-level, concurrent modern day programming language, as Go is. Go is in no way "low-level", and if Go doesn't force you to check the returned error (hopefully it does), then it's not particularly reliable or dependable, either. It doesn't have much in the way of modern constructs such as pattern matching, generics, or compile-time guarantees, so I wouldn't say that it's really that modern, either. At least it did concurrency pretty well.
- Laremere 13y ago
  You're excluding the comments which would be actual code to handle the very different events which all result in failure. Also, arguable only the first case would be something you'd want to throw an exception in a language which has them anyways. (Not getting a json response from a general http library is far from exceptional)
illumen 13y ago
tbray works for Google as "Developer Advocate". Which means he markets to developers. This is a marketing fluff piece from Google. "I still haven't written 1000 lines of Go." Why do we care what this guy says then? Because lots of Googlers vote it up. I suggest you flag this spam :)

96 points by espadrine 13y ago

Of course, anyone can make a Map function for any particular type combination, but Tim Bray has a point. The generic, functional map function cannot be written, and never will. There is an issue of elegance. If generics were to be added, lists and maps would need to have similar syntax, because they feel and behave like generics. However, they cannot. Their syntax is… not generic enough. Oddly enough, elegance through consistency was never reached when designing built-in functions like make(): those can be applied on any type, but user-defined functions don't have this freedom. Essentially, there is a Java-like divide between built-in types (which, for instance, you cannot create methods on) and user types. Their rules of elegance don't apply to the built-ins, which is quite odd. I doubt that this impedes on programming efficiency compared to C. However, some of the duplicated code out there cannot be factored into a library, and map is an example of that.

deleted 13y ago
[deleted]
pdeuchler 13y ago
I took a couple minutes to whip up a quick map function, to be honest I didn't quite believe you. This may be my naivete, but is there any way it's not sufficiently generic for types? I realize it's not suitable for maps, but it satisfies the OP's request for a map that can convert arrays of type Foo to type Bar http://play.golang.org/p/jxMFq5UYs1 http://play.golang.org/p/jxMFq5UYs1
- burntsushi 13y ago
  Here is something that is less onerous on the caller, but probably slower. [1] Here's the source. [2] [1] - https://github.com/BurntSushi/ty#examples https://github.com/BurntSushi/ty#examples [2] - https://github.com/BurntSushi/ty/blob/master/fun/list.go#L17 https://github.com/BurntSushi/ty/blob/master/fun/list.go#L17
- drewolson 13y ago
  I think the point is, if you write map like this you might as well be using a dynamically typed language in the first place because you're deriving no benefit from the type checker. In fact, you're just using casting to completely circumvent it.
  - pdeuchler 13y ago
    Of course. If the main cornerstone of your project is map it's probably not the greatest idea to use Go, but if you're using Go for other reasons and happen to need a quick map function there's no reason to change languages just because Go doesn't have a standard library implementation
    - bad_user 13y ago
      But then it's pretty odd to use Go because you like the language, while not being able to design a proper map(). And map() btw is only the tip of the iceberg in functional programming. But then again, Go is not a functional programming language.
- pcwalton 13y ago
  "I doubt that this impedes on programming efficiency compared to C." will not hold true if you're using reflection.
dragonwriter 13y ago
> The generic, functional map function cannot be written, and never will. https://github.com/droundy/gotgo/blob/master/example/slice.got https://github.com/droundy/gotgo/blob/master/example/slice.g...

comex 13y ago

> Isn’t it pathetic that so many allegedly-mainstream languages don’t have HTTP libraries that are this natural and idiomatic? That's after 13 SLoC to make a single HTTP request - I like Go, but that seems almost like a parody. It's not that different from other languages, and the error handling makes it look longer, but for something described as "natural and idiomatic" I would expect it to be one function call plus one failure test, not six and three...

cmccabe 13y ago
Experiencing a networking error is different than receiving Status 200 (or any other error status), which is different than getting a media-type different than "application/json". Personally, I don't see what would be gained by combining them all together in a library, except confusion. A lot of libraries do get this wrong and make it difficult to disentangle what is going on. If you feel like golang needs one method to do all the above, feel free to write one and post it to the mailing list... I doubt that it does, but feel free.
- comex 13y ago
  Depends on the use case. I'd say that for many, many cases, all errors should lead to the same result in the application, but with an error-specific informative message. Even if you want to have special handling for errors such as 4xx that might mean something is permanently wrong, getting a 500 is the same as experiencing a network error (except in a client side app where the latter might mean you're not connected to the Internet and should try again, but then you should use yet another API to determine when to connect), and getting a 200 with the wrong content-type has an excellent chance of meaning the same thing as a network error, in the form of a captive portal. To this extent, APIs with many separate error paths that encourage the user to write their own nonstandard error messages are harmful, because they will probably do worse explaining things than with a common method of triage. Of course there are other cases where the difference matters. It just... doesn't seem very "natural and idiomatic" to me, that's all.
throwit1979 13y ago
When people complain about comprehensive error-checking, it makes my blood boil. Checking and handling all possible error conditions is a GOOD thing.

cmccabe 13y ago

With regard to the functional-style "map" function, is it really that much less typing than a simple for loop? output := make(output_type, len(input)) for idx, e := range(input) { output[idx] = convert(e) } compared to: output := map(input, func(in input_type) output_type { return convert(in) } ) There are cases where functional programming can be a lot terser (for better and worse), but I don't think this is one of them. My guess is that the original poster is just adjusting to different ways of doing things between Golang and Ruby-- that's natural. My biggest complaint about Go so far is that you have to use an external library to get an ordered tree structure. Maps suffice most of the time, but sometimes you need ordering. [edit: fixed map example]

comex 13y ago
The second example should be output := map(convert, input) After all, map generally handles fetching the element from the array.
pcwalton 13y ago
> There are cases where functional programming can be a lot terser (for better and worse), but I don't think this is one of them. That's only true in Go because Go doesn't have type inference for closures. In ML for example this would be: let output = map (fn i -> convert i) input or, more succinctly: let output = map convert input
- cmccabe 13y ago
  Yes, that's true. I just wanted to address the specific suggestion that the original author made. Languages make a whole set of related design choices and typically none of them makes sense without considering the others. (Sometimes none of them makes sense AFTER considering the others, but that's another story.)
jacques_chester 13y ago
Loops impose, and guarantee, ordered traversal of a collection. Maps do not. This means that maps can sped up by dividing up the collection into smaller parts and mapping each of them in parallel. For a language that has concurrency as a central design motivation, it's an unfortunate omission. As we say in the database world: think in sets, not in arrays.

jlujan 13y ago

Screw map, let me pass a comparator function into sort for interfaces.

drivebyacct2 13y ago
Not sure you understand how the sort interface works...
cmccabe 13y ago
That's not necessary. You can just create a type that embeds the type you're sorting, and define the Less function however you like. It's very efficient since you're just wrapping the entire slice object, not every element in the slice. package main import "sort" type RevIntSlice struct { sort.IntSlice } func (arr RevIntSlice) Less(i, j int) bool { return arr.IntSlice.Less(j, i) } func main() { myList := RevIntSlice { sort.IntSlice {1, 2 , 3 } } sort.Sort(myList) for _, elem := range(myList.IntSlice) { println(elem) } } Note that if all you want to do is reverse the sort order, sort.Reverse is a better way than what I wrote here. This is just an example of the kinds of things you can do.
- chongli 13y ago
  >That's not necessary. You can just create a type that embeds the type you're sorting But that's more work (and more boilerplate) than simply passing an anonymous comparator to the sort function.
  - drivebyacct2 13y ago
    Which isn't possible at the language level, so why are we even discussing impl with jlujan's shitty attitude.
  - cmccabe 13y ago
    Let me guess. You are coming from Java. In Java this would be: myArray.sort(new Comparator<MyClass> ({ @Override CompareTo(MyClass lhs, MyClass rhs) { return rhs.compare(lhs); } })); Is that really shorter? I don't think so. In any case, there is a more elegant solution in Go, which is simply: sort.Sort(sort.Reverse(array)) You need to open your mind a little bit, to learn a new language.

buro9 13y ago

> Isn’t it pathetic that so many allegedly-mainstream languages don’t have HTTP libraries that are this natural and idiomatic? But at every point where Tim then returns a failure we lose insight as to why something failed. My preferred idiom for handling all errors when it comes to web requests or responses is to return both the error (as it has the detailed message of what happened) and the http status code that matches the error (as it has translated the message into a code that describes the type of error). This bit would change: if !strings.HasPrefix(mediaType, "application/json") { // bogus media type, deal return failure } to: if !strings.HasPrefix(mediaType, "application/json") { // bogus media type, deal return HttpErr{Err: errors.New(""), Status: http.StatusUnsupportedMediaType} } Whatever receives that knows there has been an error, and has the code available to do tell the developer or end user something meaningful without relying on the error message. I'm sure others are also creating their own http error as well, so whilst Go does http really well it seems in practise that there are still a few places where it could benefit from a little extra to help standardise the code we write. All of those Http codes are here: http://golang.org/pkg/net/http/#pkg-constants http://golang.org/pkg/net/http/#pkg-constants Regarding the map stuff, the only other place the lack of generics has stood out has been within slices of a type of thing. The scenario is: I have a slice of identifiers in an order important to me, for each of these concurrently fetch the record being identified and create a slice of those records ordered as per the initial slice. We've ended up using this type of thing: // Envelope for making sorted concurrent requests for Comment type CommentRequest struct { Comment Comment Err error Seq int } type CommentRequestBySeq []CommentRequest func (v CommentRequestBySeq) Len() int { return len(v) } func (v CommentRequestBySeq) Swap(i, j int) { v[i], v[j] = v[j], v[i] } func (v CommentRequestBySeq) Less(i, j int) bool { return v[i].Seq < v[j].Seq } // Much later in the code... // Make a request for each id req := make(chan CommentRequest) defer close(req) for seq, id := range ids { go HandleCommentRequest(id, seq, req) } resps := []CommentRequest{} for i := 0; i < len(ids); i++ { resp := <-req if resp.Err != nil { return resp.Err } resps = append(resps, resp) } // Sort them sort.Sort(CommentRequestBySeq(resps)) // Extract the values ems := []Comment{} for _, resp := range resps { ems = append(ems, resp.Item) } In that example a Comment is a comment made on a forum web page and we're fetching lots concurrently, but needing to return them in the order that they will be displayed. The problem stems from that envelope, and the need to implement that sort function for every type that we ever want to concurrently fetch and sort. It's the only place where I think, if we just had generics this could be a lot better.

dscrd 13y ago

Just recently, I needed a concurrent map for my project. I looked for a library and started using it. Fast forward a bit and I realized the map was completely untyped. Fast forward a slightly longer bit and I realized (perhaps wrongly? Enlighten me) that there was no way to implement a concurrent map library that wasn't untyped. We need generics, or perhaps it's more goy to have the same features by some more clever way. But still. We need those features and bad. I'll be looking at Rust with some more interest meanwhile.

ExpiredLink 13y ago

if ... { return failure } three(!) times in a row and then he writes: > Isn’t it pathetic that so many allegedly-mainstream languages don’t have HTTP libraries that are this natural and idiomatic? Isn’t it pathetic that a wannabe mainstream language doesn’t provide exception handling?

jameskilton 13y ago
Exceptions are horrible if you're trying to make a fast, dependable, reliable, low-level, concurrent modern day programming language, as Go is. Exceptions are also horrible in that they never improve the readability and understandability of any code they exist in. The Go team, some of whom actually designed C back in the day, are well familiar with Exceptions, why they are bad, and explicitly chose to not implement them. So no, using the word "pathetic" implies a gross oversight. This was a very deliberate decision, and a good one at that.
- Fistandantilus 13y ago
  > Exceptions are horrible if you're trying to make a fast, dependable, reliable, low-level, concurrent modern day programming language, as Go is. Go is in no way "low-level", and if Go doesn't force you to check the returned error (hopefully it does), then it's not particularly reliable or dependable, either. It doesn't have much in the way of modern constructs such as pattern matching, generics, or compile-time guarantees, so I wouldn't say that it's really that modern, either. At least it did concurrency pretty well.
Laremere 13y ago
You're excluding the comments which would be actual code to handle the very different events which all result in failure. Also, arguable only the first case would be something you'd want to throw an exception in a language which has them anyways. (Not getting a json response from a general http library is far from exceptional)

illumen 13y ago

tbray works for Google as "Developer Advocate". Which means he markets to developers. This is a marketing fluff piece from Google. "I still haven't written 1000 lines of Go." Why do we care what this guy says then? Because lots of Googlers vote it up. I suggest you flag this spam :)