8 ms·
> Note that this means that races on multiword data structures can lead to inconsistent values not corresponding to a single write. When the values depend on th
by twotwotwo 5y ago
> Note that this means that races on multiword data structures can lead to inconsistent values not corresponding to a single write. When the values depend on the consistency of internal (pointer, length) or (pointer, type) pairs, as is the case for interface values, maps, slices, and strings in most Go implementations, such races can in turn lead to arbitrary memory corruption.
That second sentence is one of the painful things about the Go memory model--Go is type and memory safe if you avoid explicitly unsafe things, except for data races on those structures.
I wonder what you can practically do to mitigate that at this point; doesn't seem like you go back and change things to be like Java. Can there be some best-effort detection for races in those specific places that's fast enough to run in prod (like I think they did for maps)? I also recall some Go mailing-list post I can't find suggesting that an SSE 128-bit move, while not guaranteed not to tear, seemed to tear only rarely in their tests on the chips of the time; is that true and is there anything there? I imagine fully preventing the problem with atomic reads/writes of these pairs from the heap is a clear no-go performance-wise, and might involve ABI-changing alignment guarantees.
[Edit: SSE2 behavior is discussed here https://stackoverflow.com/questions/7646018/sse-instructions-which-cpus-can-do-atomic-16b-memory-operations https://stackoverflow.com/questions/7646018/sse-instructions... which is mentioned in a comment at https://research.swtch.com/gorace https://research.swtch.com/gorace which seemed to inspire it. Chance of a race seemed to depend on CPU model at the time, bad on the old Core Duo, better on some server chips another answerer tried. Good on my laptop FWIW!]
Probably hard for anything to happen here because any performance regression is hard to swallow, the race detector is close enough for many, and anything you do won't be a 100% solution anyway. Which is a bummer.
- ngrilly 5y ago> doesn't seem like you go back and change things to be like Java Java is like Go in that regard. That's why you have to use ConcurrentHashMap instead of HashMap for instance.
- deleted 5y ago[deleted]
- ibraheemdev 5y agoJava will throw a ConcurrentModificationException. Go can cause undefined behavior.
- papercrane 5y agoJava might throw a ConcurrentModificationException. The docs are quite clear that it's not guaranteed and best-effort only.
- ngrilly 5y agoThis is not guaranteed at all: > Note that the fail-fast behavior of an iterator cannot be guaranteed as it is, generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification. Fail-fast iterators throw ConcurrentModificationException on a best-effort basis. Therefore, it would be wrong to write a program that depended on this exception for its correctness: the fail-fast behavior of iterators should be used only to detect bugs. Source: https://docs.oracle.com/javase/8/docs/api/java/util/Vector.html https://docs.oracle.com/javase/8/docs/api/java/util/Vector.h... This is conceptually similar to the race detector in Go, which is also a debugging tool, but without strict guarantees.
- twotwotwo 5y agoJava's trick is maintaining memory/type safety in the presence of races. You might crash, and your application might not see the values it expects, but you never get a buffer-overflow-like situation where Java's internal structures might get corrupted in arbitrary ways, for example.
- ngrilly 5y agoI see your point now. Yes, Java won't segfault, unlike Go on slices/maps/interface values. But even without segfaults, undefined behaviors due to race conditions can still be pretty in Java (causing infinite loops, memory leaks, etc.). Maybe it's better to let the program crash with a segfault. https://stackoverflow.com/questions/51102644/what-the-worst-thing-that-can-happen-if-i-write-concurrently-from-multiple-trea https://stackoverflow.com/questions/51102644/what-the-worst-...
- gpderetta 5y agoNo it is not better. Java will preserve the integrity of the runtime. Still memory safety issues in go due to races are probably very hard to exploit so it might not make a huge difference in practice.
- ngrilly 5y agoYes, the JVM will keep executing the program, but I'm still not convinced this is strictly better than segfaulting if a thread is stuck in an infinite loop using CPU for nothing, and another is leaking memory.
- gpderetta 5y agoIt is better than invoking random code because the runtime mistakes some data for a vtable pointer.
- ngrilly 5y agoI agree that a data race on an interface value in Go, is worse than an infinite loop or a memory leak, because it can lead to calling the wrong method on a value, which is hard to debug.
- pierrebai 5y agoI came here to quote and comments on this paragraph. The poke at C/C++ undefined behaviour, claiming Go has no such thing. Except this sentence is exactly undefined behaviour: after the data race, due to possible corruption, anything could happen. There is also equivalently smugness about the language in general. It is one thing to say there are no undefined behaviour related to the memory model, but given a compiler optimizer, you need to do hard analysis that there cannot be once optimized, with all operations re-ordering that can happen. The simplicity of the Go memory model, which is vague on purpose, is taken as making it safe. Is it? It mostly mean it is vague, which provide a convenient cover to hide under. Edit: toward the end, the author talks about adding documentation about compiler optimizations that would be explicitly forbidden. So some of my earlier comments would be addressed.
- lamontcg 5y ago> There is also equivalently smugness about the language in general. This really turns me off about Go. And it seems to attract "tech-splainers" that defend the language choices that are made by talking down to people.
- throwaway894345 5y agoI don't know. A lot of the criticism of Go is just wild over-exaggeration. Is "tech-splaining" just setting the record straight? For example, the guy upthread is insinuating that Go is roughly on-par with C++ as they both have nonzero amounts of undefined behavior. Having used both languages extensively, I can say for certain that you will run into a lot more UB in C++ than in Go, and it will be a lot harder to debug when it happens. Similarly, the guy downthread who is making a mountain out of Go's nullable pointers (and all of the other people who complain about Go's type system)--yeah, they are strictly worse than Rust's enums, but the overwhelming majority of all software ever written has been built with languages that have nullable pointers and a good chunk of that is with languages that have fully dynamic type systems (not only is the type system going to allow your null pointer access, but it will allow all manner of type errors!). Is it "smugness" to put criticism into perspective?
- TheDong 5y ago> Go is type and memory safe if you avoid explicitly unsafe things, except for data races on those structures Note that this isn't actually a real exception. maps are implemented in the go stdlib using "unsafe", in addition to compiler magic for the generics part. I believe that it holds that if you don't use "unsafe" then your program is memory and type safe, it just happens that the go stdlib and runtime use "unsafe", so you can't safely use them either. I get that this is a distinction without a difference.
- throwaway894345 5y agoData races aren't limited to maps, so I'm not sure it's even a distinction at all :)
- TheDong 5y agoI think data-races that can cause undefined behavior are limited to maps and slices and other structures that internally use unsafe. data-races that can't cause undefined behavior are a different class, closer to what java has, and iiuc that's what you get without unsafe. I don't think the distinction matters, but I think it exists.
- gpderetta 5y agoAs far as I understand all interface pointers in go are fat pointers to (vtable, object) and as such can't be updated atomically [1]. Hence any data race on assignment to an interface pointer can lead to UB. [1] unless there is an additional indirection, I don't really know much about go.
- throwaway894345 5y agoThey use `unsafe.Pointer`s under the hood, but I'm pretty sure that implementation detail is unrelated to the race condition (the issue with the race condition is that writes of these types aren't atomic so a concurrent thread that is reading the value could observe it in a partially-updated state (e.g., the pointer field has been updated but not the length field).
- throwaway34241 5y ago> I wonder what you can practically do to mitigate that at this point; doesn't seem like you go back and change things to be like Java. Wouldn't there be an obvious trade off there, requiring an extra heap allocation and indirection every time a slice is used in a data structure? I've always thought of Go as lower-level than Java (since it has explicit pointers, more control over memory layout, etc), which I think makes it a more useful language overall even if not the best fit for every application (since if you do want Java, it already exists and is mature). In other words, I think both Java's and Go's decisions make sense here.
- twotwotwo 5y agoFWIW, I meant race semantics similar to Java's--weird things happen but type/memory safety remains. Not the whole Java runtime model with object headers, etc. It's obviously, uh, quite an Exercise for the Reader(tm) to figure out the best path to avoid memory unsafety given races in line with other Go priorities (as a starting point, imagine a ton of expensive atomics accessing certain pairs on the heap, downgraded to regular loads/stores if the compiler can prove it's safe). But I was trying to say it seems harder to get there now that certain things are set in stone and regressing perf of anything is going to be bad for some user out there.
- throwaway34241 5y agoSure, I was imagining implementing better race semantics the way Java/C# do, by referencing certain multi-word data by pointer to avoid partial update problems. As it is now slices are 192-bit values (on 64 bit systems) embedded directly in objects that can be shared across threads, I don't know how you would efficiently update that atomically.
- twotwotwo 5y agoJust in theory: pointer/capacity pair is atomic, then you read len non-atomically and check it's less than cap before using it. Slice header today puts len before cap, but, like, if we're dreaming. You'd probably hit other walls. I'm def. not trying to say it's straightforward!
- to11mtm 5y agoYeah this can be a hard problem to solve. In most managed languages to 'properly' deal with it you typically need to accept locking on some level (like with ConcurrentDictionary in C# or ConcurrentHashMap in Java). One semi-elegant (yet internally complex) solution I've seen; in C# there is the language-ext library. All the collection types are structs (value types where things like .Add() return a new struct) however the struct itself just holds an implementation to a class. This works well from a user standpoint but the internals of the code is... interesting to say the least and can still lead to a lot of allocations if you're not careful.