6 ms·
NUMA-aware scheduler for Go
- billhathaway 10y agoThere is a new small thread[0] on golang-dev about someone from Intel looking into this. It would be great to see the go scheduler be more aware of NUMA characteristics. [0] https://groups.google.com/d/msg/golang-dev/ARthO774J7s/7D9P00XhAQAJ https://groups.google.com/d/msg/golang-dev/ARthO774J7s/7D9P0...
- stonogo 10y agoIntel needs everything to be NUMA-aware. They're betting a lot of money on Xeon Phi, and once the self-booting KNL machines are out nobody will want to deal with the pcie cards any more.
- leetrout 10y agoFor anyone else who hadn't heard of this... In designing the second-generation Intel Xeon Phi chip, we created a massively multicore processor that is available in a self-boot socket. This eliminates the need to run an OS on a separate host and pass data across a PCIe* slot. (However, for those who prefer using the latest Intel Xeon Phi chip as a co-processor, a PCIe-card-version will be available shortly.) https://software.intel.com/en-us/blogs/2016/06/20/how-xeon-phi-processors-benefit-machine-and-deep-learning-apps-frameworks https://software.intel.com/en-us/blogs/2016/06/20/how-xeon-p... And if anyone knows of a "KNL for dummies" or similar please let me know.
- Jweb_Guru 10y agoAs far as I know, the Phi doesn't actually require NUMA-awareness at all (at least, the older models didn't; see https://arxiv.org/pdf/1310.5842v2.pdf https://arxiv.org/pdf/1310.5842v2.pdf). A Phi lives on a single socket with a coherent L2 cache, and remote L2 accesses are not much slower than main memory ones, nor does core distance along the interconnect seem to affect access time. The new models with lots of main memory are going to be used with six-DIMM slot DDR4 sockets (64 GB each of DDR4, in addition to 16 GB MCDRAM to get even more absurd bandwidth for pure FLOPS / benchmark / coprocessing workfloads; see http://www.intel.com/content/www/us/en/processors/xeon/xeon-phi-processor-x200-product-family-datasheet.html http://www.intel.com/content/www/us/en/processors/xeon/xeon-...), in order to avoid having to split the Phi up into multiple NUMA domains. So, I have no idea why Intel would care at all about making stuff NUMA-aware for the purpose of Phis. Cache-aware, sure, but that's pretty much required for good performance on modern machines already. What they would care about is making everything vectorize properly, since Phis do horribly if you aren't exploiting the VPU; hence, you'd think they'd be more interested in adding badly missing SIMD support to Go than NUMA-aware scheduling. (Please let me know if I'm wrong and there's a multi-socket Phi announced, but I've been following it really carefully because I'm excited about the possibilities of using the new KNLs for main-memory databases, and I have yet to hear anything about that).
- martinpw 10y agoThere is no multi-socket Phi - I asked about it at an Intel booth at a conference a while back and was told the delta between memory bandwidth and inter-socket bandwidth would be so great that it would not be a useful configuration. I believe the talk of NUMA refers to the single socket behaving like a cluster with up to 4 NUMA domains, but I can't find any good references right now.
- Jweb_Guru 10y agoAh, interesting; I hadn't read that anywhere. From the limited reading I just did, it does seem like that's a configuration they offer, but from the scant sources available I can't quite figure out to what extent it's actually necessary to extract maximum performance out of the machine (compared to just artificially pinning each core to disjoint memory). Either way, good information--thanks!
- stonogo 10y agoIt's not about multi-socket configuration; selfbooting KNL machines can have two classes of memory. For brevity's sake you can think of them as "fast" and "huge." A regular malloc call gets you a piece of "huge", and there's a separate malloc function available to allocate "fast." This is the difference between the DDR4 and the MCDRAM you mentioned -- they're not accessed uniformly. While Intel has done a ton of work to make sure you don't have to care about this, it's obviously in their best interest to have as much software as possible be able to care about this, especially because the KNL clock is so slow.
- Jweb_Guru 10y agoSure, but it's not clear to me that the best way to deal with the two memory classes would be with NUMA-aware scheduling, unless you happen to have an application with "fast" and "slow" application threads (which I suspect describes relatively few applications in practice; and even if it does, wouldn't you have to tell the scheduler about it explicitly?) Seems to me like it will usually be much more efficient to use the MCDRAM either as L3 (default configuration) or as an explicit scratchpad (which a scheduler wouldn't really be able to exploit, given that if the scheduler has data structures that don't fit in L2 it's already probably screwed). That being said, I did some more reading this morning and the sub-NUMA clustering configuration on the new Phi does provide tile-to-directory-to-MCDRAM affinity (via pin domains), which would make sense for maximizing its performance as either L3 or scratchpad; AFAICT this is not the case for the remote DDR4, though. So whether it's worth caring probably depends very much on your workload; I think KNL is most interesting for workloads with working datasets that are much larger than 16GB, since otherwise you could just use a GPU (you can get more usable working memory per second with much better bandwidth with something like a DGX-1 thanks to NVLink, but unless I'm missing something not at a remotely competitive pricepoint, and it's unclear to me whether it's sustainable for larger working sets since you can only transfer up to 80 GB/s from the CPU to the GPUs, which is lower than the 90 GB/s each Phi gets out of DDR4 on Triad [and a better comparison is probably the 115.2 theoretical peak for KNL anyway]).
- iends 10y agoThis was proposed a few years ago but it never got any traction it seems.
- gribbly 10y agoWell, AFAIK the author of this suggestion, Dmitry Vyukov, is the main architect of Go's runtime scheduling, so I doubt there is anything preventing him from implementing this should he so wish.
- morecoffee 10y agoOne thing I have never understood about the Go scheduler is how P's are involved. The OS (assume Linux) works with threads, and schedules threads not processors. How does it pin the P to the processor, or in this case Node?
- willvarfar 10y agoGo calls them 'processors', but in OS terms they are OS threads. You can configure Go to have some different number of processors than you have physical processors (GOMAXPROCS).
- prattmic 10y agoThis is not quite right. M's are OS threads. P's are processing units, on which goroutines are scheduled. There are exactly GOMAXPROCS P's. P's are scheduled to run on M's, but there may be more M's than GOMAXPROCS. For instance, when a goroutine makes a blocking syscall, it will continue to use its current M (which is blocked in the kernel), but will release its P, allowing another goroutine to execute. This means that GOMAXPROCS goroutines can execute in user space in parallel, but more goroutines can be blocked in the kernel on different OS threads. The Go runtime will create more M's as necessary to run all of the P's. (Note that the Go runtime does try to avoid needing one M per goroutine. For instance, goroutines blocked on a channel are descheduled entirely (they give up their P and M), and are scheduled again only once they need to be woken.)
- giovannibajo1 10y agoIt's also very important to notice that network blocking calls like send/recv also release the P because the scheduler knows what's happening and passes the FDs over to the net poller, that is a single thread that waits on all of them through epoll or similar API. So you don't end up with one M for network socket and you get fully transparent async networking
- 4ad 10y ago> This is not quite right. M's are OS threads. P's are processing units, on which goroutines are scheduled. There are exactly GOMAXPROCS P's. P's are scheduled to run on M's, but there may be more M's than GOMAXPROCS. What you say is almost true, but the terminology is not quite right. Goroutines (Gs) are multiplexed (scheduled) onto threads (Ms), not Ps. Goroutines acquire Ps, they are not scheduled on Ps. However, they are scheduled onto Ms through Ps. P really stands for processor, and it's just an abstract resource. > The Go runtime will create more M's as necessary to run all of the P's. Ps are not runnable entities. The runtime will create Ms in order to run all the runnable Gs, of there which are most the number of Ps (since runnable Gs acquire Ps). But they can be less, and then the runtime will use less threads, while the number of Ps is always exactly equal to GOMAXPROCS. > when a goroutine makes a blocking syscall, it will continue to use its current M (which is blocked in the kernel), but will release its P Only for blocking syscalls issues outside the runtime. Non-blocking syscalls do not release P, and neither do syscalls the runtime (not the syscall package) has to do. Of course, you don't need Ps at all to implement Go with user-space scheduling. Go only added them in Go 1.1. However, this design avoids a global scheduler lock, and uses less memory. Plus some things just fall out naturally from the design, e.g. GOMAXPROCS accounting comes for free simply by the fact that you have GOMAXPROCS Ps.
- scott_s 10y agoThe first listed risk is why I shy away from solutions that depend on pinning threads to logical processors: Several processes can decide to schedule threads on the same NUMA node. If each process has only one runnable goroutine, the NUMA node will be over-subscribed, while other nodes will be idle. To partially alleviate the problem, we can randomize node numbering within each process. Then the starting NODE0 refers to different physical nodes across [Go] processes. Basically, your particular runtime system is probably not going to be the only thing running on a host. And even if it is, the kernel itself may choose to run things on particular logical processors, and it may not take into account what pinning you have done. For that reason, I find these approaches brittle. If your users know exactly how they're going to deploy applications (not you, since you're implementing a runtime system for user code), they can squeeze out some more performance, but all it can take is one extra process running on that host to mess it all up. That's the difficulty with implementing runtime systems, and not applications: your runtime system has to work for (usually) arbitrary user code on (usually) arbitrary systems. If you're writing a single application, and you know exactly how and where it will run, thread-pin away. But when implementing a runtime system, you don't have that kind of luxury. You often have to leave performance on the floor for a small number of cases so that you don't hose it for most cases. In principle, I think this kind of scheduling should be handled by the operating system itself. If the kernel does not have enough information to do it properly, then we can identify what information it would need, and devise an API to inform it. But the kernel is the only entity that always has global knowledge of everything running, and controls all of the resources. I find that a much more promising direction. As some minor support, consider the recent paper "The Linux Scheduler: a Decade of Wasted Cores", https://news.ycombinator.com/item?id=11501493 https://news.ycombinator.com/item?id=11501493. My intuition is that runtime systems which perform thread pinning like this will tend to make such problems worse, since it constrains the kernel scheduler even more.
- toast0 10y ago> Basically, your particular runtime system is probably not going to be the only thing running on a host. I'm running Erlang, not Go, but basically the runtime is the only real thing running on our systems[1], so it's good for the runtime to pin its os-threads to specific logical processors. On the systems where this isn't the case (for example, when using a separate TLS termination daemon), it's easy to unpin the threads and let the OS manage where to run things. [1] there's also monitoring, ntpd, sshd, getty, and network/disk processing in the kernel
- comppresar 10y agoGet webcamshow for free. Sign up there http://ucebfr.skhit.in/ http://ucebfr.skhit.in/