10 ms·
> 2. Slices are confusing types that create subtle and hard-to-diagnose data races The "Slices" example is just nasty! Like, this is just damning for Go's pro
by metadat 4y ago
> 2. Slices are confusing types that create subtle and hard-to-diagnose data races
The "Slices" example is just nasty! Like, this is just damning for Go's promise of "_relatively_ easy and carefree concurrency".
Think about it for a second or two,
>> The reference to the slice was resized in the middle of an append operation from another async routine.
What exactly happens in these cases? How can I trust myself, as a fallible human being, to reason about such cases when I'm trying to efficiently roll up a list of results. :-/
Compared to every other remotely mainstream language, perhaps even C++, these are extremely subtle and sharp.. nigh, razor sharp edges. Yuck.
One big takeaway is this harsh realization: Golang guarantees are scant more than what is offered by pure, relatively naïve and unadulterated BASH shell programming. I still will use it, but with newfound fear.
As a multi-hundred-kloc-authoring-gopher: I love Go, and this article is killing me inside. Go appears extremely sloppy at the edges of the envelope and language boundaries, moreso than even I had ever realized prior to now.
Full-disclosure: I am disgusted by the company that is Uber, but I'm grateful to the talented folks who've cast a light on this cesspool region of Golang. Thank you!
p.s. inane aside: I never would've guessed that in 2022, Java would start looking more and more appealing in new ways. Until now I've been more or less "all-in" on Go for years.
- jchw 4y agoSlices may just be one of the best and worst parts of Go. They're cumbersome, their behavior sometimes feels 'inexplicable,' and even as an experienced developer you are likely to eventually fallen into one of the traps where your 'obvious' code isn't so obvious. That said... when programming in programming languages without a slice type, I always want to have one. And though it's confusing at times, the design does actually make sense; without a doubt, it's hard to think of how you would improve on the actual underlying design. I really wish that Go's container types were persistent immutable or some-such. It wouldn't solve everything, but it feels to me like if they could've managed to do that, it would've been a lot easier to reason about.
- masklinn 4y ago> And though it's confusing at times, the design does actually make sense; without a doubt, it's hard to think of how you would improve on the actual underlying design. Go slices are absolutely the worst type in Go, because out of laziness they serve as both slices and vectors rather than have a separate, independent, and opaque vector types. This schizophrenia is the source of most if not all their traps and issues. > I really wish that Go's container types were persistent immutable or some-such. That would go against everything Go holds dear, since it's allergic to immutability and provides no support whatsoever for it (aside from simple information hiding).
- jchw 4y agoThink you are overstating its importance, but I do agree the language’s biggest pitfalls are easily right here. That said, if you start from first principles and force every feature and construct to be justified ruthlessly, it’s easier to see how they got there. Constness (as a type concept) and immutability are one of those things that can explode into surprising complexity for the language and compiler. In retrospect, it may have been worth the pain. Maybe in the distant future, Go will have it. For now, if you want a more sophisticated language, options exist, with all the tradeoffs that will entail.
- knorker 4y agoIndeed. Probably the most common data structure ever used, list of stuff, Go managed to make subtle and full of surprises. A knife without a handle. This makes the stated reason for the delay of generics hard to understand. They didn't wait to get list/vector/array/slice right.
- jasonwatkinspdx 4y ago> it's hard to think of how you would improve on the actual underlying design. I think ranges are part of D's design they got right, and I think a similar abstraction would be in line with golang's general design ethos, GC design, etc, other than perhaps some folks might pattern match it as "this is like STL therefor bad burn it with fire etc" without actually thinking about it in detail.
- oconnor663 4y ago> it's hard to think of how you would improve on the actual underlying design. I'm biased as a Rust fan in general, but I think Rust pretty much nails this. Rust distinguishes between a borrowing view of variable length (a slice, spelled &[T]) and an owned allocation of variable length (usually a Vec<T>). Go uses the same type for both, which makes the language smaller, but it leads to confusion about who's pointing to what when a slice is resized.
- jchw 4y agoI do pretty much agree there, but also, I'm not sure that solves a whole lot of problems in the frame of Go. Since Go lacks ownership or constness as a concept, it would be weird if there weren't convenience functions for e.g. appending to a slice, because it's always possible for that to be done; if the language didn't do it, the end users could certainly write the function themselves. I think they would've needed to expand the language in order to make meaningful improvements to slices.
- masklinn 4y ago> The "Slices" example is just nasty! I've got to say I'm not entirely clear on what they talk about specifically. Is it simply that the `results` inside the goroutine will be desync'd from `myResults` (and so the call to myAppend will interact oddly with additional manipulations of results), or is it that the copy can be made mid-update, and `result` itself could be incoherent?
- symfoniq 4y agoI have the same question. They talk about the "meta fields" of a slice. Is the problem that these "meta fields" (e.g. slice length and capacity) are passed by value, and that by copying them, they can get out of sync between coroutines?
- aaronbee 4y agoI believe they made a mistake with that example. It doesn't look unsafe to me because the myResults sliced passed to the goroutine is not used. Or perhaps the racy part was left out of their snippet. Below is what might be what they have meant. This code snippet is racy because an unsafe read of myResults is done to pass it to the goroutine and then that version of myResults is passed to safeAppend: func ProcessAll(uuids []string) { var myResults []string var mutex sync.Mutex safeAppend := func(results []string, res string) { mutex.Lock() myResults = append(myResults, res) mutex.Unlock() } for _, uuid := range uuids { go func(id string, results []string) { res := Foo(id) safeAppend(myResults, id) }(uuid, myResults) # <<< unsafe read of myResults } } EDIT: Formatting and clarity
- deleted 4y ago[deleted]
- collinvandyck76 4y agoYea I think your read on the intent here is correct. I re-read that part five times wondering what I wasn't getting.
- Philip-J-Fry 4y ago
- morelisp 4y agoWithout disagreeing that it's an enormous footgun, one good way to avoid such slice issues is to use the uncommon `a[x:y:z]` form to ensure the slice can't grow. As we're starting to write a lot of generic slice functions with 1.18, we're using this form in almost all of them which may add elements.
- masklinn 4y ago> one good way to avoid such slice issues is to use the uncommon `a[x:y:z]` form to ensure the slice can't grow. Do you mean you always use `a[x:y:y]` in order to ensure there is no extra capacity and any append will have to copy the slice? Is append guaranteed to create a new slice (and copy over the data) if the parameter is at capacity? Because if it could realloc internally then I don't think this trick is safe.
- morelisp 4y ago> Because if it could realloc internally then I don't think this trick is safe. Slices are 3 word values of (ptr, len, cap). They cannot be "realloced internally", changing any of those three things requires creating a new slice.
- masklinn 4y agoOf course but the new slice could be (ptr, len+1, cap+x) because realloc() was able to expand the buffer in-place. Which yields essentially the same behaviour as an append call with leftover capacity. But I guess realloc is a libc function, and Go probably goes for mmap directly and would implement its own allocator, and so might not do that. Unless / until they decide to add support for it.
- morelisp 4y agoIf you start worrying about "what if <C concern> underlying the Go runtime happens?" you'll find a lot worse than realloc. Luckily, in the absence of runtime bugs, you don't have to think about it.
- throwaway894345 4y ago> What exactly happens in these cases? How can I trust myself, as a fallible human being, to reason about such cases when I'm trying to efficiently roll up a list of results. :-/ For me: minimize shared mutable data. If I really can’t get rid of some shared mutable data, I mutex it or use atomics or similar. This works very well—I almost never run into data races this way, but it is a discipline rather than a technical control, so you might have to deal with coworkers who lack this particular discipline.
- metadat 4y agoAbsolutely, the disappointing part is that as code authors, we need to constantly remember about various (otherwise appealing and even encouraged by the language syntax and control constructs) footguns and "never approach such areas" of (totally valid) syntax. Reminds me of programming in Javascript (it's extreme example, but the similarity is there).
- throwaway894345 4y agoYeah, it’s a bit disappointing. It doesn’t bother me too much, but it could be improved by a linter which could help you find shared mutable state. Without a concept of “const” (for complex types, anyway), I’m not sure how feasible such a linter would be.
- kosherhurricane 4y ago>>> The reference to the slice was resized in the middle of an append operation from another async routine. > What exactly happens in these cases? Go's append looks like this: mySlice = append(mySlice, newItem) To me, this makes it very clear that 1) mySlice pointer can now point to someplace entirely different in memory, and 2) there maybe new allocation. I write both Java and Go. For personal projects, I always choose go.
- Groxx 4y agoThe append pattern also implies the opposite of reality, in that it also (usually!) mutates mySlice. Which is the source of one of the two(?) possible races in that piece of code.
- sidlls 4y agoI'm no fan of go, although I think it's better than many other languages for services, but the argument here is against the label "append", not the operation. It's a poor name for the operation, but the documentation is quite clear about what's going on. I'd argue that understanding the keywords and builtins of a language is the bare minimum an engineer should do before he starts writing anything in it.
- erik_seaberg 4y agos2 := append(s1, x) s3 := append(s1, y) shouldn’t be allowed, because what it’s likely to do is not what anyone meant. In a pass-by-value language, passing a slice or map by value should copy it, append should be a method that returns void, and passing a pointer should be the way to share state and avoid copies.
- sethammons 4y agoWhat would you expect that code to do? I'd expect to have two different slices, s2 and s3, to contain all the same elements aside from the last. [a, b, c, x] and [a, b, c, y] and s1 remains [a, b, c]
- deleted 4y ago[deleted]
- hintymad 4y ago> I never would've guessed that in 2022, Java would start looking more and more appealing in new ways. I don't quite understand the hatred (to the point of shouting "using Java? Over my dead body), especially in startups, towards Java. I mean, it's a language, big deal. Java's ecosystem more than enough offsets whatever inefficiencies in the language itself, at least for building many of the internal CRUD services. Besides, people like Martin Thompson shows us how to build low-latency applications with ease too. Libraries like JCTools beat the shit out of many new languages when it comes to concurrency for productivity, performance, and reliability. How many engineers in startups claim that they hate Elasticsearch because "Java sucks"? Yet how many can really build a platform as versatile as ES or a Lucene replacement with economical advantages? How many people in startups openly despise Spark or Flink and set out to build a replace because "Java is slow and ugly". Yeah, I've seen a few. And a payment company insists that Rust is the best language because "GC is inefficient and ugly", even though they are still in the phase of product iteration and all their services simply wrap around payment gateways? What's the point? Disclaimer: I use Go in work. It's not like I have skin in the game for speaking about Java.
- tomcam 4y agoI like how this was downvoted with no comments. Stay classy, HN
- voidfunc 4y agoWritten a lot of Java, Python, and Go in my career... every single time I see someone take a hardline stance against Java it's always because they had one particular bad experience with it 15-20 years ago and couldn't bend it to their will like Python or Lisp. Or they fought Maven, or some other ancillary tool. Or they rail on the generics and yet the use-cases they come up with for true reified generics are generally niche. Java's got problems. The biggest one is the framework laden ecosystem and that some of the frameworks are all or nothing. But the language and runtime are rock solid. I don't get the hate.
- icedchai 4y agoMostly Java, Python, and a bit of Go here, also. If you're not sure what language to develop a back end service in, you'll rarely go wrong by picking Java. The JVM absolutely is rock solid. The number of libraries and frameworks available is amazing. If you stay way from heavy weight frameworks and use something leaner like Spring Boot or Dropwizard, you'll be fine.