5 ms·

Can you explain why the user time goes down when using a single thread? Does that mean that there's a huge amount of contention in the parallelism?

nequo 4y ago
Faults also drop to zero. Might be worth trying to flush the cache before each cargo build?
celrod 4y ago
User time is the amount of CPU time spent in user mode. It is aggregated across threads. If you have 8 threads running at 100% in user mode for 1 second, that gives you 8s of user time. Total CPU time in user mode will normally increase when you add more threads, unless you're getting perfect or better-than-perfect scaling.
burntsushi 4y ago
To be honest, I don't know. My understanding of 'user' time is that is represents the sum of all CPU time spent in "user mode" (as opposed to "kernel mode"). In theory, given that understanding and perfect scaling, the user time of a multi-threaded task should roughly match the user time of a single-threaded task. Of course, "perfect" scaling is unlikely to be real, but still, you'd expect better scaling here. If I had to guess as to what's happening, it's that there's some thread pool, and at some point, near the end of compilation, only one or two of those threads is busy doing anything while the other threads are sitting and idling. Now whether and how that "idling" gets interpreted as "CPU being actively used in user mode" isn't quite clear to me. (It may not, in which case, my guess is bunk.) Perhaps someone more familiar with what 'user' time actually means and how it interplays with multi-threaded programs will be able to chime in. (I do not think faults have anything to do with it. The number of faults reported here is quite small, and if I re-run the build, the number can change quite a bit---including going to zero---and the overall time remains unaffected.)
- Filligree 4y ago
  User time is the amount of CPU time spent actually doing things. Unless you're using spinlocks, it won't include time spent waiting on locks or otherwise sleeping -- though it will include time spent setting up for locks, reloading cache lines and such. Extremely parallel programs can improve on this, but it's perfectly normal to see 2x overhead for fine-grained parallelism.
  - burntsushi 4y ago
    I'd say there's still a gap in my mental model. I agree that it's normal to observe this, definitely. I see it in other tools that utilize parallelism too. I just can't square the 2x overhead part of it in a workload like Cargo's, which I assume is not fine-grained. I see the same increase in user time with ripgrep too, and its parallelism is maybe more fine grained than Cargo's, but is still at the level of a single file, so it isn't that fine grained. But maybe for Cargo, parallelism is more fine grained than I think it is. Perhaps because of codegen-units. And similarly for ripgrep, if it's searching a lot of tiny files, that might result in fine grained parallelism in practice.
    - Filligree 4y ago
      Well, like mentioned elsewhere, most of that overhead is just hyper threads slowing down when they have active siblings. Which is fine; it’s still faster overall. Disable SMT and you’ll see much lower overhead, but higher time spent overall.
      - burntsushi 4y ago
        Yes, I know its fine. I just don't understand the full details of why hyperthreading slows things down that much. There are more experiments that could be done to confirm or deny this explanation, e.g., disabling hyperthreading. And playing with the thread count a bit more.
        Filligree 4y ago
deleted 4y ago
[deleted]
twotwotwo 4y ago
There are hardware reasons even if you leave any software scaling inefficiency to the side. For tasks that can use lots of threads, modern hardware trades off per-thread performance for getting more overall throughput from a given amount of silicon. When you max out parallelism, you're using 1) hardware threads which "split" a physical core and (ideally) each run at a bit more than half the CPU's single-thread speed, and 2) the small "efficiency" cores on newer Intel and Apple chips. Also, single-threaded runs can feed a ton of watts to the one active core since it doesn't have to share much power/cooling budget with the others, letting it run at a higher clock rate. All these tricks improve the throughput, or you wouldn't see that wall-time reduction and chipmakers wouldn't want to ship them, but they do increase how long it takes each thread to get a unit of work done in a very multithreaded context, which contributes to the total CPU time being higher than it is in a single-threaded run.
pornel 4y ago
This is caused by hyperthreading. It's not an actual inefficiency, but an artifact of the way CPU time is counted. The HT cores aren't real CPU cores. They're just an opportunistic reuse of hardware cores when another thread is waiting for RAM (RAM is relatively so slow that they're waiting a lot, for a long time). So code on the HT "core" doesn't run all the time, only when other thread is blocked. But the time HT threads wait for their opportunity turn is included in wall-clock time, and makes them look slow.
- pjmlp 4y ago
  Back in the early days of HT I was so happy to get a desktop with it, that I enabled it. The end result was that doing WebSphere development actually got slower, because of their virtual nature and everything else on the CPU being shared. So I ended up disabling it again to get the original performance back.
  - pornel 4y ago
    Yeah, the earliest attempts weren't good, but I haven't heard of any HT problems post Pentium 4 (apart from Spectre-like vulnerabilities). I assume OSes have since then developed proper support for scheduling and pre-empting hyperthreading. Also the gap between RAM and CPU speed only got worse, and CPUs have grown more various internal compute units, so there's even more idle hardware to throw HT threads at.

nequo 4y ago

Faults also drop to zero. Might be worth trying to flush the cache before each cargo build?

celrod 4y ago

User time is the amount of CPU time spent in user mode. It is aggregated across threads. If you have 8 threads running at 100% in user mode for 1 second, that gives you 8s of user time. Total CPU time in user mode will normally increase when you add more threads, unless you're getting perfect or better-than-perfect scaling.

burntsushi 4y ago

To be honest, I don't know. My understanding of 'user' time is that is represents the sum of all CPU time spent in "user mode" (as opposed to "kernel mode"). In theory, given that understanding and perfect scaling, the user time of a multi-threaded task should roughly match the user time of a single-threaded task. Of course, "perfect" scaling is unlikely to be real, but still, you'd expect better scaling here. If I had to guess as to what's happening, it's that there's some thread pool, and at some point, near the end of compilation, only one or two of those threads is busy doing anything while the other threads are sitting and idling. Now whether and how that "idling" gets interpreted as "CPU being actively used in user mode" isn't quite clear to me. (It may not, in which case, my guess is bunk.) Perhaps someone more familiar with what 'user' time actually means and how it interplays with multi-threaded programs will be able to chime in. (I do not think faults have anything to do with it. The number of faults reported here is quite small, and if I re-run the build, the number can change quite a bit---including going to zero---and the overall time remains unaffected.)

Filligree 4y ago
User time is the amount of CPU time spent actually doing things. Unless you're using spinlocks, it won't include time spent waiting on locks or otherwise sleeping -- though it will include time spent setting up for locks, reloading cache lines and such. Extremely parallel programs can improve on this, but it's perfectly normal to see 2x overhead for fine-grained parallelism.
- burntsushi 4y ago
  I'd say there's still a gap in my mental model. I agree that it's normal to observe this, definitely. I see it in other tools that utilize parallelism too. I just can't square the 2x overhead part of it in a workload like Cargo's, which I assume is not fine-grained. I see the same increase in user time with ripgrep too, and its parallelism is maybe more fine grained than Cargo's, but is still at the level of a single file, so it isn't that fine grained. But maybe for Cargo, parallelism is more fine grained than I think it is. Perhaps because of codegen-units. And similarly for ripgrep, if it's searching a lot of tiny files, that might result in fine grained parallelism in practice.
  - Filligree 4y ago
    Well, like mentioned elsewhere, most of that overhead is just hyper threads slowing down when they have active siblings. Which is fine; it’s still faster overall. Disable SMT and you’ll see much lower overhead, but higher time spent overall.
    - burntsushi 4y ago
      Yes, I know its fine. I just don't understand the full details of why hyperthreading slows things down that much. There are more experiments that could be done to confirm or deny this explanation, e.g., disabling hyperthreading. And playing with the thread count a bit more.
      - Filligree 4y ago

deleted 4y ago

[deleted]

twotwotwo 4y ago

There are hardware reasons even if you leave any software scaling inefficiency to the side. For tasks that can use lots of threads, modern hardware trades off per-thread performance for getting more overall throughput from a given amount of silicon. When you max out parallelism, you're using 1) hardware threads which "split" a physical core and (ideally) each run at a bit more than half the CPU's single-thread speed, and 2) the small "efficiency" cores on newer Intel and Apple chips. Also, single-threaded runs can feed a ton of watts to the one active core since it doesn't have to share much power/cooling budget with the others, letting it run at a higher clock rate. All these tricks improve the throughput, or you wouldn't see that wall-time reduction and chipmakers wouldn't want to ship them, but they do increase how long it takes each thread to get a unit of work done in a very multithreaded context, which contributes to the total CPU time being higher than it is in a single-threaded run.

pornel 4y ago

This is caused by hyperthreading. It's not an actual inefficiency, but an artifact of the way CPU time is counted. The HT cores aren't real CPU cores. They're just an opportunistic reuse of hardware cores when another thread is waiting for RAM (RAM is relatively so slow that they're waiting a lot, for a long time). So code on the HT "core" doesn't run all the time, only when other thread is blocked. But the time HT threads wait for their opportunity turn is included in wall-clock time, and makes them look slow.

pjmlp 4y ago
Back in the early days of HT I was so happy to get a desktop with it, that I enabled it. The end result was that doing WebSphere development actually got slower, because of their virtual nature and everything else on the CPU being shared. So I ended up disabling it again to get the original performance back.
- pornel 4y ago
  Yeah, the earliest attempts weren't good, but I haven't heard of any HT problems post Pentium 4 (apart from Spectre-like vulnerabilities). I assume OSes have since then developed proper support for scheduling and pre-empting hyperthreading. Also the gap between RAM and CPU speed only got worse, and CPUs have grown more various internal compute units, so there's even more idle hardware to throw HT threads at.