5 ms·

I’ll bet your Go code, if it isn’t trivial, has race conditions all over and you just don’t realize it.

by jshen 6y ago

papaf 6y ago
Go has a built in race detector that most teams run during CI. [1] https://golang.org/doc/articles/race_detector.html https://golang.org/doc/articles/race_detector.html
- ben0x539 6y ago
  This roughly detects data races, not all race conditions.
  - gpderetta 6y ago
    And not all possible data races.
- andonisus 6y ago
  This does not run if you have more than 10K go routines :(
xmprt 6y ago
Can't that be said for most languages? Is there anything special that other languages do to make race conditions less likely than they are in Go?
- IshKebab 6y ago
  > Is there anything special that other languages do to make race conditions less likely than they are in Go? Yes, other languages provide you with more tools than the basic nuts and bolts. For example a language might provide you with a "parallel for" primitive that is well tested. The issue is not exactly that Go's concurrency stuff is bad, it's just that it doesn't give you much help so you have to implement it all yourself. Also the lack of generics means it is difficult to write libraries for this stuff.
  - ed25519FUUU 6y ago
    The frustrating part is that the standard lib is just good enough that you also don’t tend to get good adoption for third party libraries, which could easily provide better abstractions.
- continuational 6y ago
  Some languages emphasize persistent data structures, or even enforce immutability, which means you can only introduce race conditions via explicit concurrency. That's a lot less surface area for bugs.
- pizza234 6y ago
  I believe it's correct that that can be "said for most languages". Since you're asking, Rust's ownership/type system, for example, has checks in place that prevent at least some classes of concurrency errors. For example, you can't directly modify a variable from multiple threads, due to the so-called "move" rules: let mut myvar = vec![]; thread::spawn(move || { myvar.push(1); }); myvar.push(1); // can't do this; variable has been moved
- ben0x539 6y ago
  I think Go invites mistakes like this being made over and over, because (in my subjective opinion as someone who writes a lot of Go at work but isn't extremely happy about that) you have to write out the basic low-level concurrency patterns over and over, because the language tries to resist abstraction. In other langs I'd expect to see this code use a generic map-reduce/parallel map helper function or something like that, but in Go people seem to be happier writing for-loops and spawning goroutines by hand every time.
  - iainmerrick 6y ago
    A lot of people are making tangential points in this discussion (“concurrency is always hard!”) but you’ve put your finger on the key point. Concurrency is hard, and one fairly successful approach is to provide battle-tested libraries for these high-level use cases (“run M functions with max parallelism N”). Go gives you low-level concurrency tools, but actively resists the kind of abstraction that would let you make reusable high-level tools, so that robust library doesn’t exist. It’s a strange lack, as it’s meant to be a concurrency-focused language, and the standard library is absolutely terrific in other areas (eg networking and encryption).
    - apta 6y ago
      It just shows golang designers lack of experience in language design (it's stuck in the 70's). They may have written a lot of code, but it doesn't automatically make them good language designers.
      - ben0x539 6y ago
        I'm not really sure about this. I think the language they designed has the properties they desired, in that they consciously forewent a lot of potential benefits because some notion of simplicity is more important to them. So in that sense I think they were successful at designing a language. Like some other comment here says, you run into bugs like this early on and they're not necessarily show-stoppers. Maybe keeping the language simple and ensuring that most code is readable if you only know the language and the stdlib (and not some fancy concurrency library) is worth the occasional bug? Of course that doesn't necessarily mean that Go is the language I want to use for every use case under the sun, but that probably wasn't a goal of Go to begin with.
- thesuperbigfrog 6y ago
  Clojure uses immutable data structures and software transactional memory: https://clojure.org/about/concurrent_programming https://clojure.org/about/concurrent_programming
  - jshen 6y ago
    I love clojure, and I’ve written some high concurrency production apps in it. Concurrency is still hard in clojure, and that’s assuming all of your state and logic can live in a single process. That assumption usually doesn’t hold, and then concurrency gets way harder.
- johnisgood 6y ago
  Yep, Ada/SPARK: https://docs.adacore.com/spark2014-docs/html/ug/en/source/concurrency.html https://docs.adacore.com/spark2014-docs/html/ug/en/source/co... Search for "race condition", or just "race". This section in particular is about preventing data races: https://docs.adacore.com/spark2014-docs/html/ug/en/source/concurrency.html#preventing-data-races https://docs.adacore.com/spark2014-docs/html/ug/en/source/co... This one is about preventing race conditions: https://docs.adacore.com/spark2014-docs/html/ug/en/source/concurrency.html#protected-types-and-protected-objects https://docs.adacore.com/spark2014-docs/html/ug/en/source/co...
naikrovek 6y ago
It's not trivial and I found a lot of races, yes, but I refactored over a couple days to completely isolate work within a goroutine so that data is only passed in and out of them via channels, and with that all races went away. The tool has been used in production extensively for a couple of years now, converting 3D model files from one format to another, and it's running great, and very fast as well. So, you're right, but it has been addressed.

I’ll bet your Go code, if it isn’t trivial, has race conditions all over and you just don’t realize it.

by jshen 6y ago

I’ll bet your Go code, if it isn’t trivial, has race conditions all over and you just don’t realize it.

Go has a built in race detector that most teams run during CI. [1] https://golang.org/doc/articles/race_detector.html https://golang.org/doc/articles/race_detector.html

ben0x539 6y ago
This roughly detects data races, not all race conditions.
- gpderetta 6y ago
  And not all possible data races.
andonisus 6y ago
This does not run if you have more than 10K go routines :(

xmprt 6y ago

Can't that be said for most languages? Is there anything special that other languages do to make race conditions less likely than they are in Go?

IshKebab 6y ago
> Is there anything special that other languages do to make race conditions less likely than they are in Go? Yes, other languages provide you with more tools than the basic nuts and bolts. For example a language might provide you with a "parallel for" primitive that is well tested. The issue is not exactly that Go's concurrency stuff is bad, it's just that it doesn't give you much help so you have to implement it all yourself. Also the lack of generics means it is difficult to write libraries for this stuff.
- ed25519FUUU 6y ago
  The frustrating part is that the standard lib is just good enough that you also don’t tend to get good adoption for third party libraries, which could easily provide better abstractions.
continuational 6y ago
Some languages emphasize persistent data structures, or even enforce immutability, which means you can only introduce race conditions via explicit concurrency. That's a lot less surface area for bugs.
pizza234 6y ago
I believe it's correct that that can be "said for most languages". Since you're asking, Rust's ownership/type system, for example, has checks in place that prevent at least some classes of concurrency errors. For example, you can't directly modify a variable from multiple threads, due to the so-called "move" rules: let mut myvar = vec![]; thread::spawn(move || { myvar.push(1); }); myvar.push(1); // can't do this; variable has been moved
ben0x539 6y ago
I think Go invites mistakes like this being made over and over, because (in my subjective opinion as someone who writes a lot of Go at work but isn't extremely happy about that) you have to write out the basic low-level concurrency patterns over and over, because the language tries to resist abstraction. In other langs I'd expect to see this code use a generic map-reduce/parallel map helper function or something like that, but in Go people seem to be happier writing for-loops and spawning goroutines by hand every time.
- iainmerrick 6y ago
  A lot of people are making tangential points in this discussion (“concurrency is always hard!”) but you’ve put your finger on the key point. Concurrency is hard, and one fairly successful approach is to provide battle-tested libraries for these high-level use cases (“run M functions with max parallelism N”). Go gives you low-level concurrency tools, but actively resists the kind of abstraction that would let you make reusable high-level tools, so that robust library doesn’t exist. It’s a strange lack, as it’s meant to be a concurrency-focused language, and the standard library is absolutely terrific in other areas (eg networking and encryption).
  - apta 6y ago
    It just shows golang designers lack of experience in language design (it's stuck in the 70's). They may have written a lot of code, but it doesn't automatically make them good language designers.
    - ben0x539 6y ago
      I'm not really sure about this. I think the language they designed has the properties they desired, in that they consciously forewent a lot of potential benefits because some notion of simplicity is more important to them. So in that sense I think they were successful at designing a language. Like some other comment here says, you run into bugs like this early on and they're not necessarily show-stoppers. Maybe keeping the language simple and ensuring that most code is readable if you only know the language and the stdlib (and not some fancy concurrency library) is worth the occasional bug? Of course that doesn't necessarily mean that Go is the language I want to use for every use case under the sun, but that probably wasn't a goal of Go to begin with.
thesuperbigfrog 6y ago
Clojure uses immutable data structures and software transactional memory: https://clojure.org/about/concurrent_programming https://clojure.org/about/concurrent_programming
- jshen 6y ago
  I love clojure, and I’ve written some high concurrency production apps in it. Concurrency is still hard in clojure, and that’s assuming all of your state and logic can live in a single process. That assumption usually doesn’t hold, and then concurrency gets way harder.
johnisgood 6y ago
Yep, Ada/SPARK: https://docs.adacore.com/spark2014-docs/html/ug/en/source/concurrency.html https://docs.adacore.com/spark2014-docs/html/ug/en/source/co... Search for "race condition", or just "race". This section in particular is about preventing data races: https://docs.adacore.com/spark2014-docs/html/ug/en/source/concurrency.html#preventing-data-races https://docs.adacore.com/spark2014-docs/html/ug/en/source/co... This one is about preventing race conditions: https://docs.adacore.com/spark2014-docs/html/ug/en/source/concurrency.html#protected-types-and-protected-objects https://docs.adacore.com/spark2014-docs/html/ug/en/source/co...

naikrovek 6y ago

It's not trivial and I found a lot of races, yes, but I refactored over a couple days to completely isolate work within a goroutine so that data is only passed in and out of them via channels, and with that all races went away. The tool has been used in production extensively for a couple of years now, converting 3D model files from one format to another, and it's running great, and very fast as well. So, you're right, but it has been addressed.