3 ms·
It's a thousand clock cycles more than a function call (seriously, I'm not making this up: time it sometime). I guess whether that is "expensive" or not is con
by ajross 18y ago
It's a thousand clock cycles more than a function call (seriously, I'm not making this up: time it sometime). I guess whether that is "expensive" or not is context dependent. For some applications (e.g. database servers) that make lots of random access reads from large files, using a mapping can be overwhelmingly faster.
I guess I'm stunned that this turns out to have been such a controversial notion...
- tptacek 18y agoCan you be specific about which part of the system call you're talking about? Because you said "swapping the stack" (which doesn't take a thousand clock cycles) and "flushing the TLB" (which doesn't happen in OSs where the kernel occupies a permanent part of the VM address space). I'm sorry, this isn't controversial. All I said was, "mmap isn't faster than read in typical use cases". But then we all got really specific talking about ESP and CS and CR3 and now we have something to go back and forth on, which is kind of fun, and I might learn something new. Didn't mean to snipe at you.
- ajross 18y agoTo be clear: I didn't say "flush the TLB". Nonetheless unless you happen to be lucky and have those kernel mapping in there, those TLB entries need to be faulted in. That's expensive. Likewise the new stack isn't in L1 cache and needs to be read in from main memory. Likewise the kernel code to execute the handler needs to be read into the instruction cache, etc... Given that main memory reads are pushing 100 cycles on a modern box, all those things add up. A context switch (my usage: meaning a bounce to a non-local, non-current execution environment) is really expensive.
- tptacek 18y agoCome on. Userland code constantly blows L1 cache. Almost any library call will churn the icache. For that matter, any memory access can collide the TLB. None of those things are syscall-specific. If you were concerned about L1 and icache, you needed to write your code meticulously to accomodate them anyways. Yes, cache is important. But there's a world of difference between the "flush the caches and the TLB" behavior that system calls used to incur and the "expensive relative to local variable access" behavior we're talking about here. "A bounce to non-local, non-current execution environment" literally doesn't mean anything. You could be talking about the CR3 change that swaps page hierarchies, the CPL3->0 change that allows privileged instructions, the CS change that got you there, or even the ESP change to "swap stacks". Which of these are expensive? I'm chasing you down because it's fun, not because I think you don't know what you're talking about. But, let's make this relevant: if system call overhead is drowned by disk I/O overhead, then one argument for mmap goes away.
- ajross 18y agoWhy do you insist on making this a flame war? Sit down and write a program that makes 10M trivial system calls (getpid() is a good choice) and one that calls a function to return a constant integer 10M times. I'm sorry, but you seem to have a wildly inflated idea of the speed of system calls on modern OSs. The real world just doesn't work like that. This is my last post on this thread.
- tptacek 18y agoHere you go: http://pastie.org/230109 http://pastie.org/230109 Want the punch line? OSX getpid averages under 100 cycles; snprintf (no IO) averages over 1000.
- ajross 18y agoAre you sure getpid() is doing a system call and not reading TLS?
- tptacek 18y agoWhat a great question! I just checked. Nope: Breakpoint 1, 0x96210c44 in getpid () (gdb) disp/i $eip 2: x/i $eip 0x96210c44 <getpid>: call 0x96210c49 <getpid+5> (gdb) stepi 0x96210c49 in getpid () 2: x/i $eip 0x96210c49 <getpid+5>: pop %ecx 0x96210c4a in getpid () 2: x/i $eip 0x96210c4a <getpid+6>: lea 0xa6427a3(%ecx),%ecx 0x96210c50 in getpid () 2: x/i $eip 0x96210c50 <getpid+12>: mov (%ecx),%eax 0x96210c52 in getpid () 2: x/i $eip 0x96210c52 <getpid+14>: test %eax,%eax 0x96210c54 in getpid () 2: x/i $eip 0x96210c54 <getpid+16>: jle 0x96210c57 <getpid+19> 0x96210c57 in getpid () 2: x/i $eip 0x96210c57 <getpid+19>: mov $0x14,%eax 0x96210c5c in getpid () 2: x/i $eip 0x96210c5c <getpid+24>: call 0x961e8cf4 <_sysenter_trap>
- tptacek 18y agoBy the way, you asked me to time getpid against "a function that returns a constant integer". Obviously that's a silly comparison (getpid isn't really returning a constant; it's a vectored call and it has to pull the PID out of a structure offset from %%fs). But here's a fairer fight: getpid vs. a trivial LFSR: s = (s >> 1) ^ (-(s & 1) & 0xd0000001); I also timed it the way you asked, with wall clock time over 100M iterations (10M runs too fast to time). Surprising even me, getpid wins by almost a full second. Got another challenge for me to write code for?
- dhess 18y ago"TLB entries need to be faulted in?" You mean TLB entries which map kernel pages? Typically kernel pages (and the TLB entries which cover them) are "pinned." You don't want to take a TLB exception in the middle of an interrupt service routine, let alone a page fault. Most architectures couldn't even handle the nested exception, and would panic.
- tptacek 18y agoTLB misses don't cause exceptions on X86. The TLB is part of the microarchitecture.
- dhess 18y agoYeah, also on IA-64 (optionally). I was generalizing. Still, on IA-64 at least, you would typically pin kernel mappings by putting them in translation registers instead of translation cache registers (i.e., the TLB). I haven't kept up with x86 on Linux; does Linux enable the global page extensions? I would assume it does for kernel mappings.