5 ms·

I need to update this for the latest generics syntax, but I wrote this a while back: http://www.jerf.org/iri/post/2955 http://www.jerf.org/iri/post/2955 "Why Go

by jerf 6y ago

I need to update this for the latest generics syntax, but I wrote this a while back: http://www.jerf.org/iri/post/2955 http://www.jerf.org/iri/post/2955 "Why Go Getting Generics Will Not Change Idiomatic Go" I do not understand the people who think that functional-esque programming is the only and best way to do things who are apparently just sitting and waiting for Go to be able to do it. Are you forced to use Go or something? This is an honest question, btw, and I am legitimately interested in answers. If the only reason or the biggest reason you're not using Go is that you can't import map-centric programming into it, why not use any of the many languages that support that rather than griping about a language where it's still ultimately going to be bodged on? Again, honest question, because there's no shortage of such languages. My perception is that Go isn't that popular that there's a lot of people forced to use it against their will because it's just The Language. Anyhow, even after generics drop, it still isn't going to be any fun. I expect to use a map here or there often enough that it'll be nice to have it in the standard library but there's no realistic way that it's going to become "the best way to program in Go". If you're waiting to try it out after the generics drop, I would suggest trying something else instead; Elixir, Nim, Crystal, Pony.

mohanmcgeek 6y ago
Many problems for which you might end up using something like Kafka streams just lend themselves nicely into the streams/collections -like style I am happy using Go right now when they don't have map.. but including map and not including reduce/flatten is just odd
- jerf 6y ago
  "Many problems for which you might end up using something like Kafka streams just lend themselves nicely into the streams/collections -like style" Have you actually taken the time to write out what it looks like, as I have in that blog post above, though? It's not pretty. Any sane Go programmer is not going to want to use this for very much. (I write that carefully. I anticipate using it a non-zero amount myself. But it's not going to be a lot.) I understand liking map/filter/reduce, even if it is the beginning of functional programming rather than the end. I don't understand absolutely demanding it in languages where it doesn't fit well and insisting on using it no matter the cost.
  - MAGZine 6y ago
    it's pretty clear to me. people coming from other languages are used to typing less characters and expressing their ideas like that, and they want to type less characters in go. for go, a language that idiomizes variable names like `f`, you think that people would be more on board with fewer characters. There are many other areas where go has chosen to avoid tidying up the syntax/making lives easier in the name of simplicity, though I digress. As for your earlier question of "why don't you pick another language," I think this is a silly question to ask. The vast majority of people working for hire are not picking their language. If you have a few hundred kloc written in go, I doubt very much people are in a position to casually introduce new languages, especially if they're trying to improve the current system.
    - jerf 6y ago
      "it's pretty clear to me. people coming from other languages are used to typing less characters and expressing their ideas like that, and they want to type less characters in go." It's not. It's more. A lot more sometimes, plus the indentation for trying to chain maps together is ugly. I think more people who think this is going to be great need to clock some time on the Go Generics playground [1], implement some Map and Filter and MapError functions, and convert some real code to this style in real Go. It's awful. I'm probably spoiled by Haskell and how nice it makes all this, but it's awful even against normal Go code. Try it. The problem is, the thing that makes "functional-style" programming in Go difficult wasn't "missing generics". It was, missing generics, somewhat heavy-weight anonymous function syntax, indentation rules not optimized for deeply-nested functions, a compiler that almost certainly won't optimize through very many layers of these constructs, the inability to put methods on core types and the fact the core types are missing the useful methods/interfaces like "Traversable", no higher kinded types so no monads or anything else like that, a variety of little syntax issues that make this all slick, and probably some things I'm missing. Just getting some generics (and not having generic methods that can add new generic types hurts this style too) just isn't enough to make it slick. Go would need a suite of changes to make this slick and efficient, not just these generics. [1]: https://go2goplay.golang.org/ https://go2goplay.golang.org/
hughrr 6y ago
Your blog post was a great read. Completely agree. I’ve seen so much stuff turn to excrement the moment FP is applied. I’m writing a complex aggregator in go and the abstraction is buffered channels. There is no map or filter in sight. It is not needed.
Serow225 6y ago
I dunno, for some domains it feels like 80% of the job openings are using Go.
morelisp 6y ago
> Are you forced to use Go or something? I mean, yes, most companies have a relatively small list of supported languages, whether explicitly or practically. If someone at our company wants to write a program that does some binary data munging that's even more frustrating in Java, and needs to do it faster than Python allows, they have to use Go. We will not let them use C or C++, because most of the team will not understand it and it won't be able to integrate with any of our standardized libraries or CI infrastructure.
1_person 6y ago
Your statements in the blog about performance of that pattern are not accurate to the best of my knowledge. A program structured as a sequence of short, tight loops over vectors/slices as described is more or less hitting the performance sweet spot of modern microarchitectures. The general form of a short loop over a vector plays to the strengths of branch prediction and the instruction cache while the temporal and spatial locality of the memory accesses are conducive to caching and pipelining. Moreover, there's usually no need to allocate or copy the array in that sort of data flow. I mean, unless you're chasing worst case performance to make a point or something. Slice the buffers out of a pool and allow ownership of the data to follow the execution context, then you're free to modify it in place. The above points cover 3-4 orders of magnitude real world performance. It is true that there are some issues with the optimizer working in this pattern but I have only seen performance severely degraded in this pattern by compile time type ambiguity -- some but not all interface{} parameters, anonymous functions / closures in scopes with dynamic type. Interface types with a pointer receiver do not typically encounter these specific issues, and I believe the empty interface type has much better performance since ~1.12 or so as well.
- jerf 6y ago
  "A program structured as a sequence of short, tight loops over vectors/slices as described is more or less hitting the performance sweet spot of modern microarchitectures." Sure, but even faster is not to loop over intermediate arrays at all, by virtue of never constructing them in the first place when they aren't necessary. "Moreover, there's usually no need to allocate or copy the array in that sort of data flow. I mean, unless you're chasing worst case performance to make a point or something. Slice the buffers out of a pool and allow ownership of the data to follow the execution context, then you're free to modify it in place." That starts getting into "I'm sure someone can come up with some solution that meets some of these goals". Whatever map you're talking about here isn't one that is defined as creating a new slice based on mapping a function over the old slice. I mean, it kinda sounds like you're saying "well, if you just write a conventional for loop you can do all this in one pass" to me? Which is my point? I'm not the one pitching for lots of array creation, it's people who insist on using maps and filters in a language that, of all the major languages, just isn't going to put in the optimization time to convert them back into loops under the hood.

by jerf 6y ago

mohanmcgeek 6y ago

Many problems for which you might end up using something like Kafka streams just lend themselves nicely into the streams/collections -like style I am happy using Go right now when they don't have map.. but including map and not including reduce/flatten is just odd

jerf 6y ago
"Many problems for which you might end up using something like Kafka streams just lend themselves nicely into the streams/collections -like style" Have you actually taken the time to write out what it looks like, as I have in that blog post above, though? It's not pretty. Any sane Go programmer is not going to want to use this for very much. (I write that carefully. I anticipate using it a non-zero amount myself. But it's not going to be a lot.) I understand liking map/filter/reduce, even if it is the beginning of functional programming rather than the end. I don't understand absolutely demanding it in languages where it doesn't fit well and insisting on using it no matter the cost.
- MAGZine 6y ago
  it's pretty clear to me. people coming from other languages are used to typing less characters and expressing their ideas like that, and they want to type less characters in go. for go, a language that idiomizes variable names like `f`, you think that people would be more on board with fewer characters. There are many other areas where go has chosen to avoid tidying up the syntax/making lives easier in the name of simplicity, though I digress. As for your earlier question of "why don't you pick another language," I think this is a silly question to ask. The vast majority of people working for hire are not picking their language. If you have a few hundred kloc written in go, I doubt very much people are in a position to casually introduce new languages, especially if they're trying to improve the current system.
  - jerf 6y ago
    "it's pretty clear to me. people coming from other languages are used to typing less characters and expressing their ideas like that, and they want to type less characters in go." It's not. It's more. A lot more sometimes, plus the indentation for trying to chain maps together is ugly. I think more people who think this is going to be great need to clock some time on the Go Generics playground [1], implement some Map and Filter and MapError functions, and convert some real code to this style in real Go. It's awful. I'm probably spoiled by Haskell and how nice it makes all this, but it's awful even against normal Go code. Try it. The problem is, the thing that makes "functional-style" programming in Go difficult wasn't "missing generics". It was, missing generics, somewhat heavy-weight anonymous function syntax, indentation rules not optimized for deeply-nested functions, a compiler that almost certainly won't optimize through very many layers of these constructs, the inability to put methods on core types and the fact the core types are missing the useful methods/interfaces like "Traversable", no higher kinded types so no monads or anything else like that, a variety of little syntax issues that make this all slick, and probably some things I'm missing. Just getting some generics (and not having generic methods that can add new generic types hurts this style too) just isn't enough to make it slick. Go would need a suite of changes to make this slick and efficient, not just these generics. [1]: https://go2goplay.golang.org/ https://go2goplay.golang.org/

hughrr 6y ago

Your blog post was a great read. Completely agree. I’ve seen so much stuff turn to excrement the moment FP is applied. I’m writing a complex aggregator in go and the abstraction is buffered channels. There is no map or filter in sight. It is not needed.

Serow225 6y ago

I dunno, for some domains it feels like 80% of the job openings are using Go.

morelisp 6y ago

> Are you forced to use Go or something? I mean, yes, most companies have a relatively small list of supported languages, whether explicitly or practically. If someone at our company wants to write a program that does some binary data munging that's even more frustrating in Java, and needs to do it faster than Python allows, they have to use Go. We will not let them use C or C++, because most of the team will not understand it and it won't be able to integrate with any of our standardized libraries or CI infrastructure.

1_person 6y ago

Your statements in the blog about performance of that pattern are not accurate to the best of my knowledge. A program structured as a sequence of short, tight loops over vectors/slices as described is more or less hitting the performance sweet spot of modern microarchitectures. The general form of a short loop over a vector plays to the strengths of branch prediction and the instruction cache while the temporal and spatial locality of the memory accesses are conducive to caching and pipelining. Moreover, there's usually no need to allocate or copy the array in that sort of data flow. I mean, unless you're chasing worst case performance to make a point or something. Slice the buffers out of a pool and allow ownership of the data to follow the execution context, then you're free to modify it in place. The above points cover 3-4 orders of magnitude real world performance. It is true that there are some issues with the optimizer working in this pattern but I have only seen performance severely degraded in this pattern by compile time type ambiguity -- some but not all interface{} parameters, anonymous functions / closures in scopes with dynamic type. Interface types with a pointer receiver do not typically encounter these specific issues, and I believe the empty interface type has much better performance since ~1.12 or so as well.

jerf 6y ago
"A program structured as a sequence of short, tight loops over vectors/slices as described is more or less hitting the performance sweet spot of modern microarchitectures." Sure, but even faster is not to loop over intermediate arrays at all, by virtue of never constructing them in the first place when they aren't necessary. "Moreover, there's usually no need to allocate or copy the array in that sort of data flow. I mean, unless you're chasing worst case performance to make a point or something. Slice the buffers out of a pool and allow ownership of the data to follow the execution context, then you're free to modify it in place." That starts getting into "I'm sure someone can come up with some solution that meets some of these goals". Whatever map you're talking about here isn't one that is defined as creating a new slice based on mapping a function over the old slice. I mean, it kinda sounds like you're saying "well, if you just write a conventional for loop you can do all this in one pass" to me? Which is my point? I'm not the one pitching for lots of array creation, it's people who insist on using maps and filters in a language that, of all the major languages, just isn't going to put in the optimization time to convert them back into loops under the hood.