25 ms·
Go 1.5 max procs default
- eblume 11y agoBack when I was learning Go, I was astounded to learn that GOMAXPROCS was set to 1 by default. It didn't seem to make any sense to me given the nature of the claims of the language. Of course when I investigated why the default was 1 I learned the reasons why (which this article talks about), but it still left a bad taste in my mouth. I'm glad that Go has matured to the point where this can be a historical footnote! Good work, Go team.
- stygiansonic 11y agoLike you, I was also initially confused about the default value of GOMAXPROCS - for the same reason. Great to see they have changed this to something that, without any other context, makes more sense.
- cmelbye 11y agoSee also: Concurrency is not Parallelism http://blog.golang.org/concurrency-is-not-parallelism http://blog.golang.org/concurrency-is-not-parallelism You can still benefit from the "claims of the language", even if code isn't running in parallel.
- rsc 11y agoWhile this is technically true, it doesn't seem like that helpful a response to say "well you were getting benefits, just not this one". Until now you've had to know to ask for parallelism; it should have been there by default, and soon (I hope) it will be.
- jlouis 11y agoThis is, incidentally, also the Erlang default. Interrogate the system and if it is SMP capable, run with SMP enabled.
- cmelbye 11y agoOh, I agree 100%. I was just as confused as he was when I first started using Go, and I'm very excited that this change might finally happen. However, I think calling my response "technically true" is selling Go's concurrency primitives a little short. They enable a really great programming model, which makes it much easier to express certain ideas in code. And I/O-bound programs benefit even if GOMAXPROCS=1. I didn't understand that subtlety when I first started with Go, and it seems like GP commenter (and probably other HN readers) didn't either. "Concurrency is not parallelism" is very helpful in explaining that, which is why I linked to it.
- tossawayppp 11y agoA distinction without meaning is kind of dumb. In popular parlance parallelism and concurrency are approximately the same thing.
- Jtsummers 11y agoYou aren't wrong, but it does make it difficult to discuss with people. At least within the programming field it's useful to have two terms with distinct meanings to discuss the topic. Consider the terms accuracy and precision [0], where again the popular understanding is that they're (essentially) synonymous. But within the scientific and engineering world that uses the terms the distinction is critical. EDIT: Verification and validation [1] are another pair of terms where the distinction is important for industry, but popular understanding often mixes them up. [0] http://en.wikipedia.org/wiki/Accuracy_and_precision http://en.wikipedia.org/wiki/Accuracy_and_precision [1] http://en.wikipedia.org/wiki/Verification_and_validation http://en.wikipedia.org/wiki/Verification_and_validation
- enneff 11y agoWhen having a technical discussion we should interpret technical terms with their precise meaning. Rob Pike gave a good talk about this: http://blog.golang.org/concurrency-is-not-parallelism http://blog.golang.org/concurrency-is-not-parallelism
- bkeroack 11y agoIt wasn't very long ago (before the multicore era) that essentially nothing running on personal computers was truly parallel. Yet multithreaded applications (concurrency) worked just fine.
- Igglyboo 11y agoIt does have meaning. An obvious difference is that you can have concurrency with one processor but not parallelism requires at least two.
- chrisseaton 11y ago> parallelism requires at least two Interestingly there is one example of parallelism without concurrency and with just one processor and just one core - vector instructions. Although it would be a breathtakingly good compiler that could emit vector instructions to schedule two goroutines in parallel
- ubernostrum 11y agoWhat I do not want in a programming language is for it to advertise features which are subject to confusion in common parlance, then say "I did what I said, you just weren't smart enough to understand what I said". My reaction is somewhat like this XKCD: https://xkcd.com/169/ https://xkcd.com/169/
- tankenmate 11y agoThe debate over concurrency vs parallelism hit its peak in the mid 1990s, well before Go came along. I remember having arguments with fellow students (and the occasional professor) at university about how it affects memory allocations (stack vs heap), virtual memory layout (the thread vs coroutine stack being one of the bigger issues; Linux's early introduction of mremap was particularly handy) the OS scheduler, etc. It's not just a Go / language feature thing.
- aikah 11y ago> It didn't seem to make any sense to me given the nature of the claims of the language I disagree, it makes total sense when you understand the consequences of GOMAXPROCS set to anything but 1.
- acdha 11y agoConcurrency was the primary selling point for Go. Setting GOMAXPROCS=1 not only failed to deliver on that but also ran the risk of people not noticing concurrency issues because all of their development happened without parallel execution.
- aikah 11y agosure, but now some programs will break. This is a breaking change.
- eikenberry 11y agoConcurrency in Go is a programming model, specifically CSP. They have been very clear about this from the beginning and saying they failed to deliver on it is disingenuous.
- coldtea 11y agoNo, they also talked from the start about how Go makes it easy to leverage many CPUs...
- smegel 11y ago> It didn't seem to make any sense to me given the nature of the claims of the language. You mean that it was good for writing network servers capable of handling many tens of thousands of connections? You usually don't need more than one core to handle IO, and if you are doing processor intensive stuff a few more cores is not going to help if you have many thousands of requests - in this case a better pattern is a work queue and a way of letting the client know their work is done. Concurrency/threading is generally considered a "hard" topic within software dev, and if you lack a solid grounding in the basic concepts you are destined to misuse or misapply the paradigm.
- amelius 11y agoSometimes a server does more than just I/O.
- smegel 11y agoWell if it "does more" for every request when you are handling many thousands of concurrent requests, that is going to be one hell of a busy server.
- paulfurtado 11y agoI'm not sure what you're trying to argue here. If you want to serve many requests that do significant CPU work in parallel, multiple cores are necessary no matter how efficiently Go manages the connections. Additionally, even rendering HTML templates can take quite a bit of CPU time and when Go's concurrency is limited to a single-core, that means only one template can be rendered in parallel, blocking other requests.
- smegel 11y ago> If you want to serve many requests that do significant CPU work in parallel For what definition of "many"? If you have 16 cores that is 16 requests being processed in parallel. Go hasn't even got out of bed by the time it is handling 16 connections. When Go is being used for it's intended purpose - handling thousands of concurrent connections, you need to be a lot smarter than just running on all the cores you have. I'm not saying you wouldn't use all cores to maximize utilization - what I am saying if that if you are scaling beyond one core, then you would already be thinking about scaling across multiple servers behind a load balancer - i.e. you are doing ops work on a production scale cluster. And if you are doing production ops work, you are going to be tuning all the processes under your control in detail, not just making rash assumptions about how they may or may not utilization multiple cores. The point I was arguing here is that someone who said "Back when I was learning Go" and who understands the "nature of the claims of the language" is not going to be running into these sorts of scalability issues, and would not be bitterly disappointed to learn of a default threading value of 1. Unless they had some misconstrued understanding of what Go's concurrency support was all about, and made an incorrect assumption that it was primarily about parallel computation, rather than it's true purpose of having many thousands of lightweight processes handling lots of network I/O.
- tptacek 11y agoThis probably always should have been the default, since bumping GOMAXPROCS from 1 to N can reveal bugs.
- akavel 11y agoWorth to note that they address this concern in a way in the doc: > Correctness. There is one non-performance concern. Increased parallelism could make bugs in racy programs more likely to cause crashes or other problems. The setting of GOMAXPROCS=1 may have thus far let those bugs go undetected. Raising it may therefore make buggy programs less reliable. We hope that both the race detector and the introduction of goroutine preemption and interleaving in Go 1.1 has already helped programmers identify and eliminate many such bugs. In any event, we can’t reasonably hobble the performance of working Go programs for fear of breaking buggy ones. If such buggy programs do turn up, the authors can set GOMAXPROCS=1 explicitly to keep them working until the bugs can be fixed.
- twoodfin 11y agoAs someone with little experience in Go, I'm curious: How do racy programs end up crashing? Do they just end up in some inconsistent state and panic or can you actually trigger a CPU fault?
- kasey_junk 11y agoSetting something to nil and then referencing it is a trivial race condition to create which can cause a crash.
- Rexxar 11y agoStrangely, on my computer, Firefox reports a security error on the site. No error in Chrome. An error occurred during a connection to golang.org. security library: DER encoded incorrectly formatted message. (Error code: sec_error_bad_der) I didn't have this the last time I went on golang.org.
- lukevers 11y agoSame here. If you go to the link without ssl it'll load for you though.
- httpseverywhere 11y agoHTTPS Everywhere is the bane of my crap-configured-ssl-server existence :(.
- tyho 11y agoClick on the icon in the menu bar. It allows you to disable the rules for domains where it is broken.
- stygiansonic 11y agoThe certificate for golang.org was recently (re)issued. I see a "Not Valid Before" of "Thursday, May 21, 2015 at 5:39:30 AM Eastern Daylight Time". I wonder if there is a Firefox-unsupported certificate extension in the new certificate? EDIT: I wonder if it's related to this Firefox issue: Secure connection failed (sec_error_bad_der) due to certs with SAN dNSName entries incorrectly containing IP addresses[0] However, it doesn't appear the golang.org certificate has any subjectAltName DNS entries using an IP address. 0. https://bugzilla.mozilla.org/show_bug.cgi?id=1148766 https://bugzilla.mozilla.org/show_bug.cgi?id=1148766
- throwaway2048 11y agoThe most frustrating part about this is when you want to access router/etc machines via https and an ip address it is just no longer even possible if they dont support this particular certificate extension. Firefox doesn't even give you a way to bypass the erorr, even the error itself gives absolutely zero indication of what the issue actually IS. It's extraordinarily obnoxious.
- zerr 11y agoAny updates regarding generics and error handling?
- Thaxll 11y agoCould people stop trolling about generics on every Golang topics?
- zerr 11y agoThe Golang FAQ mentions from the day one that they are going to consider adding ge<censored>ics someday in the future. So I was wandering if there are any updates regarding this. I didn't expect such a fanboysm though...
- rsc 11y agoEven if there were news, it's certainly off topic for this post.
- andrewchambers 11y agoIf you ever implement it, I suggest a release on april 1st.
- artursapek 11y agoThat would be my favorite April Fools ever.
- kjksf 11y agoThe FAQ also says that Go 1 will not introduce backwards-incompatible changes. Generics would be such a backwards-incompatible change. So the answer to "are there generics in 1.x release" will always be "no". http://golang.org/doc/faq#What_is_the_status_of_the_project http://golang.org/doc/faq#What_is_the_status_of_the_project http://golang.org/doc/go1compat http://golang.org/doc/go1compat Go 2 is years away (if it even happens).
- vruiz 11y agoDirect link for Firefox users since there is a certificate error: https://docs.google.com/document/d/1At2Ls5_fhJQ59kDK2DFVhFu3g5mATSXqqV5QrxinasI/preview?sle=true https://docs.google.com/document/d/1At2Ls5_fhJQ59kDK2DFVhFu3...
- PopsiclePete 11y agoGo 1.5 is shaping to be a pretty major release! Definitely feels more major than 1.3 and 1.4 were. Really happy it's progressing so nicely. Can't wait to see the GC improvements in 1.6+.
- staunch 11y ago> We hope that both the race detector and the introduction of goroutine preemption and interleaving in Go 1.1 has already helped programmers identify and eliminate many such bugs. I recall that it took me a while to discover the race detector when I started using Go. Now I always set GOMAXPROCS=runtime.NumCPU() and use the race detector. https://golang.org/doc/articles/race_detector.html https://golang.org/doc/articles/race_detector.html
- fuddle 11y agoFirefox users use - http://golang.org/s/go15gomaxprocs http://golang.org/s/go15gomaxprocs
- higherpurpose 11y agoI thought it was set to 1 CPU because that's the only way for Go to guarantee thread-safety. Is that fixed in 1.5, or does Google just choose to ignore that in this release due to the performance benefits?
- bradfitz 11y agoNo, that was one of the reasons for GOMAXPROCS=1 on App Engine, which is a hardened environment. For your own code, we trust you to fix your races or deal with the consequences.
- rsc 11y agoI don't know what you mean by "guarantee thread-safety". Whether your program is thread-safe is up to you, not Go.
- tokenizerrr 11y agoThey address this in the document: > Correctness. There is one non-performance concern. Increased parallelism could make bugs in racy programs more likely to cause crashes or other problems. The setting of GOMAXPROCS=1 may have thus far let those bugs go undetected. Raising it may therefore make buggy programs less reliable. We hope that both the race detector and the introduction of goroutine preemption and interleaving in Go 1.1 has already helped programmers identify and eliminate many such bugs. In any event, we can’t reasonably hobble the performance of working Go programs for fear of breaking buggy ones. If such buggy programs do turn up, the authors can set GOMAXPROCS=1 explicitly to keep them working until the bugs can be fixed.
- spullara 11y agoMy prediction is that it is going to reveal tons of bugs in libraries. They really should have defaulted it to 2.
- rco8786 11y agoCurious, why? Seems that anything over 1 would reveal the same bugs.
- darshan 11y agoI'm interpreting the suggestion to mean that the original default with Go 1.0 should have been 2, because that is the lowest value (and thus would have the least of the the performance problems documented here) that would, as you say, reveal the same bugs. That way raising it to runtime.NumCPU wouldn't be a potentially breaking change. Personally, I've found the issue to be documented well enough, and it is easy enough to set GOMAXPROCS to whatever works best for a given program, that I'm content with how the team has handled it. And I'm thrilled at the work that is being done now to fix the issue once and for all.
- davidw 11y ago> we propose to change the default setting of GOMAXPROCS to the number of CPUs available, as determined by runtime That's what Erlang does, too.
- amelius 11y agoI hope that at least the standard libraries that come with Go are thread safe (?) Do they test them on a regular basis, appropriately, with GOMAXPROCS > 1?