4 ms·

"By default, Go programs run with GOMAXPROCS set to the number of cores available; in prior releases it defaulted to 1." This is exciting, but what are the i

by ovis 11y ago

"By default, Go programs run with GOMAXPROCS set to the number of cores available; in prior releases it defaulted to 1." This is exciting, but what are the implications? I remember something once about some 3rd-party code, and maybe even some stdlib code behaving poorly on multiple processors.

vruiz 11y ago
With the new scheduler I believe this is much less of an issue. It is huge though because there might be a bunch of data races in the wild that were hidden until now and are about to see the light of day. Hopefully most people test with -race these days.
- stcredzero 11y ago
  For high concurrency systems, I'd expect false sharing to start rearing its ugly head. I only recently got knocked over the head with -race by the oh-so-diplomatic folks on IRC. It's very good. Might as well use it -- otherwise it's like driving with a broken check engine light. Speaking of false sharing, would it be a good idea to take the Go implementation of Disruptor and use that for buffered channels?
  - pcwalton 11y ago
    > For high concurrency systems, I'd expect false sharing to start rearing its ugly head. I'd be really surprised if that results in programs in parallel mode becoming slower than sequential (which is what you get with GOMAXPROCS=1). Cache effects are important, but not that important in a setting like Go, where few people use concurrent read-write data structures anyway (other than channels, of course) due to the lack of generics. Much more likely is that some programs become slower due to the fact that GOMAXPROCS>1 requires more locks and atomic operations. That's not false sharing, though; that's just synchronization overhead.
    - buerkle 11y ago
      What does lack of generics have to do with concurrent vs non-concurrent data structures? It seems like if the problem calls for it, concurrent data structures would be used regardless of generics.
      - pcwalton 11y ago
        Concurrent data structures (especially lock-free data structures) are really hard to write. Specializing e.g. a lock-free FIFO stack to each type you want to use it with isn't feasible in practice. It's much easier to just protect a built-in slice or map with a mutex, and that's what I believe Go code generally does.
      - dragonwriter 11y ago
        > What does lack of generics have to do with concurrent vs non-concurrent data structures? Generics allow building reusable forms of complex data structures, reducing the cost in developer effort of each specialized use. By not supporting generics, Go increases the cost of each specialized use of complex data structures, which narrows the range of circumstances where the cost will be justified by the benefit. Whether or not "the problem calls for it" is always a cost vs. benefit question, and Go -- compared to languages with support for generics -- increases the cost of this particular solution.
    - stcredzero 11y ago
      I'd be really surprised if that results in programs in parallel mode becoming slower than sequential I'm going to guess that you haven't played too much with executing things in parallel? In Clojure groups, people have heard beginners express surprise about parallel mode programs running slower so often, it's somewhat shaped the reaction to such questions. few people use concurrent read-write data structures anyway (other than channels, of course) One can still get in trouble with channels of pointers to structs. Much more likely is that some programs become slower due to the fact that GOMAXPROCS>1 requires more locks and atomic operations. That's not false sharing, Depending on how the locks and atomic operations are implemented and used, the slowdown with locks and atomic operations could be in large part because of false sharing. (I don't know the details for Go.)
  - vruiz 11y ago
    I'm not familiar with disruptor, but if you want an alternative to buffered channels maybe mangos[0] would interest you, it supports inproc and IPC among other transports. [0] https://github.com/gdamore/mangos https://github.com/gdamore/mangos
azylman 11y ago
Performance is supposed to be much improved: https://docs.google.com/document/d/1At2Ls5_fhJQ59kDK2DFVhFu3g5mATSXqqV5QrxinasI/edit https://docs.google.com/document/d/1At2Ls5_fhJQ59kDK2DFVhFu3...

by ovis 11y ago

vruiz 11y ago

With the new scheduler I believe this is much less of an issue. It is huge though because there might be a bunch of data races in the wild that were hidden until now and are about to see the light of day. Hopefully most people test with -race these days.

stcredzero 11y ago
For high concurrency systems, I'd expect false sharing to start rearing its ugly head. I only recently got knocked over the head with -race by the oh-so-diplomatic folks on IRC. It's very good. Might as well use it -- otherwise it's like driving with a broken check engine light. Speaking of false sharing, would it be a good idea to take the Go implementation of Disruptor and use that for buffered channels?
- pcwalton 11y ago
  > For high concurrency systems, I'd expect false sharing to start rearing its ugly head. I'd be really surprised if that results in programs in parallel mode becoming slower than sequential (which is what you get with GOMAXPROCS=1). Cache effects are important, but not that important in a setting like Go, where few people use concurrent read-write data structures anyway (other than channels, of course) due to the lack of generics. Much more likely is that some programs become slower due to the fact that GOMAXPROCS>1 requires more locks and atomic operations. That's not false sharing, though; that's just synchronization overhead.
  - buerkle 11y ago
    What does lack of generics have to do with concurrent vs non-concurrent data structures? It seems like if the problem calls for it, concurrent data structures would be used regardless of generics.
    - pcwalton 11y ago
      Concurrent data structures (especially lock-free data structures) are really hard to write. Specializing e.g. a lock-free FIFO stack to each type you want to use it with isn't feasible in practice. It's much easier to just protect a built-in slice or map with a mutex, and that's what I believe Go code generally does.
    - dragonwriter 11y ago
      > What does lack of generics have to do with concurrent vs non-concurrent data structures? Generics allow building reusable forms of complex data structures, reducing the cost in developer effort of each specialized use. By not supporting generics, Go increases the cost of each specialized use of complex data structures, which narrows the range of circumstances where the cost will be justified by the benefit. Whether or not "the problem calls for it" is always a cost vs. benefit question, and Go -- compared to languages with support for generics -- increases the cost of this particular solution.
  - stcredzero 11y ago
    I'd be really surprised if that results in programs in parallel mode becoming slower than sequential I'm going to guess that you haven't played too much with executing things in parallel? In Clojure groups, people have heard beginners express surprise about parallel mode programs running slower so often, it's somewhat shaped the reaction to such questions. few people use concurrent read-write data structures anyway (other than channels, of course) One can still get in trouble with channels of pointers to structs. Much more likely is that some programs become slower due to the fact that GOMAXPROCS>1 requires more locks and atomic operations. That's not false sharing, Depending on how the locks and atomic operations are implemented and used, the slowdown with locks and atomic operations could be in large part because of false sharing. (I don't know the details for Go.)
- vruiz 11y ago
  I'm not familiar with disruptor, but if you want an alternative to buffered channels maybe mangos[0] would interest you, it supports inproc and IPC among other transports. [0] https://github.com/gdamore/mangos https://github.com/gdamore/mangos

azylman 11y ago

Performance is supposed to be much improved: https://docs.google.com/document/d/1At2Ls5_fhJQ59kDK2DFVhFu3g5mATSXqqV5QrxinasI/edit https://docs.google.com/document/d/1At2Ls5_fhJQ59kDK2DFVhFu3...