3 ms·

I'd argue technically not due to data races on interface values, maps, slices, and strings... but close enough for almost all purposes. PS. Note that unlike mo

by gpm 2y ago

I'd argue technically not due to data races on interface values, maps, slices, and strings... but close enough for almost all purposes. PS. Note that unlike most languages, a datarace on something like an int in go isn't undefined behavior, just non-deterministic and discouraged.

steveklabnik 2y ago
Yes, these issues are real, but as you say, it's not really the same as UB. As such, Go is generally considered a MSL. For anyone not familiar with this, see https://go.dev/ref/mem#restrictions https://go.dev/ref/mem#restrictions Incidentally, Java is very similar: https://docs.oracle.com/javase/specs/jls/se8/html/jls-17.html#jls-17.7 https://docs.oracle.com/javase/specs/jls/se8/html/jls-17.htm...
- EE84M3i 2y ago
  It's been quite a while since I worked with golang details but I seem to remember writes to the same map from multiple goroutines being UB if GOMAXPROCS>1? Did they eventually solve that, or is that excluded from the definition of memory-safe language because it's a different type of safety?
  - gpm 2y ago
    This is my understanding (both that that can occur, and that it does not count as memory safe). I called out the types I did in my post specifically because they're the types where data races can result in arbitrary memory corruption (in the actual implementation, the spec is arguably not even that strict). Per https://go.dev/ref/mem#restrictions https://go.dev/ref/mem#restrictions (the same linke as in Steve's reply to me)
  - steveklabnik 2y ago
    I believe that post 1.6, the runtime will try and detect this, but it's not my area of expertise.
- cyberax 2y ago
  Java is not similar. It guarantees memory safety even for multithreaded code with data races. It can result in stack overflows, infinite loops, but not in memory corruption. That's not the case for Go, it's trivially easy to create memory corruption with multiple threads there.
  - steveklabnik 2y ago
    "Similar" does not mean "the same," you are correct that Java's guarantee is stronger.
- ameliaquining 2y ago
  IIUC word tearing in Java cannot cause arbitrary memory corruption. Any read is guaranteed to see either the old value, or the new value, or half of each. Since these are just numbers we're talking about, not pointers, and the language doesn't otherwise allow memory-unsafe things like pointer arithmetic, a data race resulting in word tearing in Java can at worst result in a runtime crash or a logic bug, not arbitrary memory corruption. By contrast, in Go, all it takes to cause full-blown undefined behavior, in the same sense as C or C++ or Rust, is accessing an interface/map/slice value in a racy way, not having the race detector on, and being sufficiently unlucky. I agree that in practice this doesn't stop people from calling Go a memory-safe language. I think it's reasonable to argue that this statement is just wrong. That said, I think the Carbon people once said something along the lines of, in their experience, it's rare in practice for memory corruption bugs in C++ to be caused solely by data races. So a language that's memory-safe in single-threaded contexts, but where concurrency is not memory-safe, is "good enough". Maybe it's the same in Go. I am still suspicious of this, though. Maybe it's just that concurrent operations are rarer in general because they're so famously hard to get right, and this makes it less likely that you'll have one that's exactly the right kind of wrong to cause memory corruption. Certainly it seems at odds with the notion that you want the exceptions to memory safety to be auditable.
  - steveklabnik 2y ago
    Yes, you are correct that Java's guarantee is stronger. > By contrast, in Go, all it takes to cause full-blown undefined behavior, in the same sense as C or C++ or Rust It's a little more tricky than that. UB in C/C++/Rust is something that the compiler can use to transform the code. This can lead to other issues. Go's compiler will not do those kinds of things. So on one hand, yes, it can lead to arbitrary Bad Things, but on the other hand, it is different. > Maybe it's the same in Go. I was having a discussion about this topic last week, regarding Uber's paper from a few years back trying to analyze the prevalence of these issues: https://news.ycombinator.com/item?id=43336293 https://news.ycombinator.com/item?id=43336293 You'll notice the person replying to me has pointed out some additional mitigations that have happened since.
    - 2y ago
- pcwalton 2y ago
  Java only has tearing for double and long, which are POD types and so are irrelevant to memory safety. Go has tearing for pointer-like objects (interfaces, slices), which causes the memory safety problems. Data races on those pointer-like objects are undefined behavior in Go, whereas there isn't something comparable in Java. The reason why Go is considered a memory safe language is actually pretty mundane: it's simply that the data race issues are pretty hard to exploit in practice for a variety of reasons.
  - steveklabnik 2y ago
    Yes, I certainly agree that Java's guarantee is stronger, and that it's harder to exploit Go's issues than non-MSL's issues.
  - vardump 2y ago
    I haven’t seen any aligned double or long tearing for over 20 years on workstation or server CPUs.
    - mastax 2y ago
      X86 has a strong memory model, so tearing isn’t an issue for 64-bit or smaller quantities. Other architectures (ARM) are different. AIUi
      - vardump 2y ago
        A bit late reply. Memory model strength has nothing to do with tearing. Total store order and total load order does not matter in any possible way. There's no tearing, because all memory operations are wider than 64-bits. Unless someone purposefully performs two 32-bit operations on a single 64-bit value.

by gpm 2y ago

steveklabnik 2y ago

Yes, these issues are real, but as you say, it's not really the same as UB. As such, Go is generally considered a MSL. For anyone not familiar with this, see https://go.dev/ref/mem#restrictions https://go.dev/ref/mem#restrictions Incidentally, Java is very similar: https://docs.oracle.com/javase/specs/jls/se8/html/jls-17.html#jls-17.7 https://docs.oracle.com/javase/specs/jls/se8/html/jls-17.htm...

EE84M3i 2y ago
It's been quite a while since I worked with golang details but I seem to remember writes to the same map from multiple goroutines being UB if GOMAXPROCS>1? Did they eventually solve that, or is that excluded from the definition of memory-safe language because it's a different type of safety?
- gpm 2y ago
  This is my understanding (both that that can occur, and that it does not count as memory safe). I called out the types I did in my post specifically because they're the types where data races can result in arbitrary memory corruption (in the actual implementation, the spec is arguably not even that strict). Per https://go.dev/ref/mem#restrictions https://go.dev/ref/mem#restrictions (the same linke as in Steve's reply to me)
- steveklabnik 2y ago
  I believe that post 1.6, the runtime will try and detect this, but it's not my area of expertise.
cyberax 2y ago
Java is not similar. It guarantees memory safety even for multithreaded code with data races. It can result in stack overflows, infinite loops, but not in memory corruption. That's not the case for Go, it's trivially easy to create memory corruption with multiple threads there.
- steveklabnik 2y ago
  "Similar" does not mean "the same," you are correct that Java's guarantee is stronger.
ameliaquining 2y ago
IIUC word tearing in Java cannot cause arbitrary memory corruption. Any read is guaranteed to see either the old value, or the new value, or half of each. Since these are just numbers we're talking about, not pointers, and the language doesn't otherwise allow memory-unsafe things like pointer arithmetic, a data race resulting in word tearing in Java can at worst result in a runtime crash or a logic bug, not arbitrary memory corruption. By contrast, in Go, all it takes to cause full-blown undefined behavior, in the same sense as C or C++ or Rust, is accessing an interface/map/slice value in a racy way, not having the race detector on, and being sufficiently unlucky. I agree that in practice this doesn't stop people from calling Go a memory-safe language. I think it's reasonable to argue that this statement is just wrong. That said, I think the Carbon people once said something along the lines of, in their experience, it's rare in practice for memory corruption bugs in C++ to be caused solely by data races. So a language that's memory-safe in single-threaded contexts, but where concurrency is not memory-safe, is "good enough". Maybe it's the same in Go. I am still suspicious of this, though. Maybe it's just that concurrent operations are rarer in general because they're so famously hard to get right, and this makes it less likely that you'll have one that's exactly the right kind of wrong to cause memory corruption. Certainly it seems at odds with the notion that you want the exceptions to memory safety to be auditable.
- steveklabnik 2y ago
  Yes, you are correct that Java's guarantee is stronger. > By contrast, in Go, all it takes to cause full-blown undefined behavior, in the same sense as C or C++ or Rust It's a little more tricky than that. UB in C/C++/Rust is something that the compiler can use to transform the code. This can lead to other issues. Go's compiler will not do those kinds of things. So on one hand, yes, it can lead to arbitrary Bad Things, but on the other hand, it is different. > Maybe it's the same in Go. I was having a discussion about this topic last week, regarding Uber's paper from a few years back trying to analyze the prevalence of these issues: https://news.ycombinator.com/item?id=43336293 https://news.ycombinator.com/item?id=43336293 You'll notice the person replying to me has pointed out some additional mitigations that have happened since.
  - 2y ago
pcwalton 2y ago
Java only has tearing for double and long, which are POD types and so are irrelevant to memory safety. Go has tearing for pointer-like objects (interfaces, slices), which causes the memory safety problems. Data races on those pointer-like objects are undefined behavior in Go, whereas there isn't something comparable in Java. The reason why Go is considered a memory safe language is actually pretty mundane: it's simply that the data race issues are pretty hard to exploit in practice for a variety of reasons.
- steveklabnik 2y ago
  Yes, I certainly agree that Java's guarantee is stronger, and that it's harder to exploit Go's issues than non-MSL's issues.
- vardump 2y ago
  I haven’t seen any aligned double or long tearing for over 20 years on workstation or server CPUs.
  - mastax 2y ago
    X86 has a strong memory model, so tearing isn’t an issue for 64-bit or smaller quantities. Other architectures (ARM) are different. AIUi
    - vardump 2y ago
      A bit late reply. Memory model strength has nothing to do with tearing. Total store order and total load order does not matter in any possible way. There's no tearing, because all memory operations are wider than 64-bits. Unless someone purposefully performs two 32-bit operations on a single 64-bit value.