8 ms·

This comes up now and again, somewhat akin to the Rust soundness hole issue. To be fair, it is a legitimate issue, and you could definitely cause it by accident

by jchw 1y ago

This comes up now and again, somewhat akin to the Rust soundness hole issue. To be fair, it is a legitimate issue, and you could definitely cause it by accident, which is more than I can say about the Rust soundness hole(s?), which as far as I know are basically incomprehensible and about as likely to come across naturally as guessing someone's private key. That said in many years of using Go in production I don't think I've ever come across a situation where the exact requirements to cause this bug have occurred. Uber has talked a lot about bugs in Go code. This article is useful to understand some of the practical problems facing Go developers actually wind up being, particularly the table at the bottom summarizing how common each issue is. https://www.uber.com/en-US/blog/data-race-patterns-in-go/ https://www.uber.com/en-US/blog/data-race-patterns-in-go/ They don't have a specific category that would cover this issue, because most of the time concurrent map or slice accesses are on the same slice and this needs you to exhibit a torn read. So why doesn't it come up more in practice? I dunno. Honestly beats me. I guess people are paranoid enough to avoid this particular pitfall most of the time, kind of like the Technology Connections theory on Americans and extension cords/powerstrips[1]. Re-assigning variables that are known to be used concurrently is obvious enough to be a problem and the language has atomics, channels, mutex locks so I think most people just don't wind up doing that in a concurrent context (or at least certainly not on purpose.) The race detector will definitely find it. For some performance hit, though, the torn reads problem could just be fixed. I think they should probably do it, but I'm not losing sweat over all of the Go code in production. It hasn't really been a big issue. [1]: https://www.youtube.com/watch?v=K_q-xnYRugQ https://www.youtube.com/watch?v=K_q-xnYRugQ

bombela 1y ago
It took months to finally solve a data race in Go. No race detector would see anything. Nobody understood what was happening. It ultimately resulted in a loop counter overflowing, which recomputed the same thing a billion of time (but always the same!). So the visible effect was a request would randomly take 3 min instead of 100ms. I ended up using perf in production, which indirectly lead me to understand the data race. I was called in to help the team because of my experience debugging the weirdest things as a platform dev. Because of this I was exposed to so many races in Go, from my biased point of view, I want Rust everywhere instead. But I guess I am putting myself out of a job? ;)
- jchw 1y ago
  I think the true answer is that the moment you have to do tricky concurrency in Go, it becomes less desirable. I think that Go is still better at tricky concurrency than C, though there are some downsides too (I think it's a bit easier to sneak in a torn read issue in Go due to the presence of fat pointers and slice headers everywhere.) Go is really good at easy concurrency tasks, like things that have almost no shared memory at all, "shared-nothing" architectures, like a typical web server. Share some resources like database handles with a sync.Pool and call it a day. Go lets you write "async" code as if it were sync with no function coloring, making it decidedly nicer than basically anything in its performance class for this use case. Rust, on the other hand, has to contend with function coloring and a myriad of seriously hard engineering tasks to deal with async issues. Async Rust gets better every year, but personally I still (as of last month at least) think it's quite a mess. Rust is absolutely excellent for traditional concurrency, though. Anything where you would've used a mutex lock, Rust is just way better than everything else. It's beautiful. But I struggle to be as productive in Rust as I am in Go, because Rust, the standard library, and its ecosystem gives the programmer so much to worry about. It sometimes reminds me of C++ in that regard, though it's nowhere near as extremely bad (because at least there's a coherent build system and package manager.) And frankly, a lot of software I write is just boring, and Go does fine for a lot of that. I try Rust periodically for things, and romantically it feels like it's the closest language to "the future", but I think the future might still have a place for languages like Go.
  - zozbot234 1y ago
    > And frankly, a lot of software I write is just boring, and Go does fine for a lot of that. I try Rust periodically for things, and romantically it feels like it's the closest language to "the future", but I think the future might still have a place for languages like Go. It's not so much about being "boring" or not; Rust does just fine at writing boring code once you get familiar with the boilerplate patterns (Real-world experience has shown that Rust is not really at a disadvantage wrt. productivity or iteration speed). There is a case for Golang and similar languages, but it has to do with software domains where there literally is no viable alternative to GC, such as when dealing with arbitrary, "spaghetti" reference graphs. Most programs aren't going to look like that though, and starting with Rust will yield a higher quality solution overall.
qcnguy 1y ago
What do Uber mean in that article when they say that Go programs "expose 8x more concurrency compared to Java microservices"? They're using the word concurrency as if it were a countable noun.
- wmf 1y ago
  If the Java version creates 4 concurrent tasks (could be threads, fibers, futures, etc.) but the Go version creates 32 goroutines, that's 8x the concurrency.
ameliaquining 1y ago
I think it's also worth noting that Rust's maintainers acknowledge its various soundness holes as bugs that need to be fixed. It's just that some of them, like https://github.com/rust-lang/rust/issues/25860 https://github.com/rust-lang/rust/issues/25860 (which I assume you're referring to), need major refactors of certain parts of the compiler in order to fix, so it's taking a while.
ralfj 1y ago
Yeah, I can totally believe that this is not a big issue in practice. But I think terms like "memory safety" should have a reasonably strict meaning, and languages that go the extra mile of actually preventing memory corruption even in concurrent programs (which is basically everything typically considered "memory safe" except Go) should not be put into the same bucket as languages that decide not to go through this hassle.
sethammons 1y ago
That Uber article is fantastic. I believe Go fixed the first example recently. We had a rule at my last gig: avoid anonymous functions and always recover from them.

by jchw 1y ago

bombela 1y ago

It took months to finally solve a data race in Go. No race detector would see anything. Nobody understood what was happening. It ultimately resulted in a loop counter overflowing, which recomputed the same thing a billion of time (but always the same!). So the visible effect was a request would randomly take 3 min instead of 100ms. I ended up using perf in production, which indirectly lead me to understand the data race. I was called in to help the team because of my experience debugging the weirdest things as a platform dev. Because of this I was exposed to so many races in Go, from my biased point of view, I want Rust everywhere instead. But I guess I am putting myself out of a job? ;)

jchw 1y ago
I think the true answer is that the moment you have to do tricky concurrency in Go, it becomes less desirable. I think that Go is still better at tricky concurrency than C, though there are some downsides too (I think it's a bit easier to sneak in a torn read issue in Go due to the presence of fat pointers and slice headers everywhere.) Go is really good at easy concurrency tasks, like things that have almost no shared memory at all, "shared-nothing" architectures, like a typical web server. Share some resources like database handles with a sync.Pool and call it a day. Go lets you write "async" code as if it were sync with no function coloring, making it decidedly nicer than basically anything in its performance class for this use case. Rust, on the other hand, has to contend with function coloring and a myriad of seriously hard engineering tasks to deal with async issues. Async Rust gets better every year, but personally I still (as of last month at least) think it's quite a mess. Rust is absolutely excellent for traditional concurrency, though. Anything where you would've used a mutex lock, Rust is just way better than everything else. It's beautiful. But I struggle to be as productive in Rust as I am in Go, because Rust, the standard library, and its ecosystem gives the programmer so much to worry about. It sometimes reminds me of C++ in that regard, though it's nowhere near as extremely bad (because at least there's a coherent build system and package manager.) And frankly, a lot of software I write is just boring, and Go does fine for a lot of that. I try Rust periodically for things, and romantically it feels like it's the closest language to "the future", but I think the future might still have a place for languages like Go.
- zozbot234 1y ago
  > And frankly, a lot of software I write is just boring, and Go does fine for a lot of that. I try Rust periodically for things, and romantically it feels like it's the closest language to "the future", but I think the future might still have a place for languages like Go. It's not so much about being "boring" or not; Rust does just fine at writing boring code once you get familiar with the boilerplate patterns (Real-world experience has shown that Rust is not really at a disadvantage wrt. productivity or iteration speed). There is a case for Golang and similar languages, but it has to do with software domains where there literally is no viable alternative to GC, such as when dealing with arbitrary, "spaghetti" reference graphs. Most programs aren't going to look like that though, and starting with Rust will yield a higher quality solution overall.

qcnguy 1y ago

What do Uber mean in that article when they say that Go programs "expose 8x more concurrency compared to Java microservices"? They're using the word concurrency as if it were a countable noun.

wmf 1y ago
If the Java version creates 4 concurrent tasks (could be threads, fibers, futures, etc.) but the Go version creates 32 goroutines, that's 8x the concurrency.

ameliaquining 1y ago

I think it's also worth noting that Rust's maintainers acknowledge its various soundness holes as bugs that need to be fixed. It's just that some of them, like https://github.com/rust-lang/rust/issues/25860 https://github.com/rust-lang/rust/issues/25860 (which I assume you're referring to), need major refactors of certain parts of the compiler in order to fix, so it's taking a while.

ralfj 1y ago

Yeah, I can totally believe that this is not a big issue in practice. But I think terms like "memory safety" should have a reasonably strict meaning, and languages that go the extra mile of actually preventing memory corruption even in concurrent programs (which is basically everything typically considered "memory safe" except Go) should not be put into the same bucket as languages that decide not to go through this hassle.

sethammons 1y ago

That Uber article is fantastic. I believe Go fixed the first example recently. We had a rule at my last gig: avoid anonymous functions and always recover from them.