3 ms·
A system call is a pair of context switches: swap the registers, swap the stack, change the TLB mappings, on some architectures flush the cache. Even the faste
by ajross 18y ago
A system call is a pair of context switches: swap the registers, swap the stack, change the TLB mappings, on some architectures flush the cache. Even the fastest system calls (Linux on amd64) take a thousand clock cycles or more.
- tptacek 18y agoTechnically, a system call (on i386) is only changing CS, EIP, and ESP. You don't have to change CR3 or flush the TLB. "Swapping stacks" isn't expensive.
- ajross 18y agoIt's a thousand clock cycles more than a function call (seriously, I'm not making this up: time it sometime). I guess whether that is "expensive" or not is context dependent. For some applications (e.g. database servers) that make lots of random access reads from large files, using a mapping can be overwhelmingly faster. I guess I'm stunned that this turns out to have been such a controversial notion...
- tptacek 18y agoCan you be specific about which part of the system call you're talking about? Because you said "swapping the stack" (which doesn't take a thousand clock cycles) and "flushing the TLB" (which doesn't happen in OSs where the kernel occupies a permanent part of the VM address space). I'm sorry, this isn't controversial. All I said was, "mmap isn't faster than read in typical use cases". But then we all got really specific talking about ESP and CS and CR3 and now we have something to go back and forth on, which is kind of fun, and I might learn something new. Didn't mean to snipe at you.
- ajross 18y agoTo be clear: I didn't say "flush the TLB". Nonetheless unless you happen to be lucky and have those kernel mapping in there, those TLB entries need to be faulted in. That's expensive. Likewise the new stack isn't in L1 cache and needs to be read in from main memory. Likewise the kernel code to execute the handler needs to be read into the instruction cache, etc... Given that main memory reads are pushing 100 cycles on a modern box, all those things add up. A context switch (my usage: meaning a bounce to a non-local, non-current execution environment) is really expensive.
- tptacek 18y agoCome on. Userland code constantly blows L1 cache. Almost any library call will churn the icache. For that matter, any memory access can collide the TLB. None of those things are syscall-specific. If you were concerned about L1 and icache, you needed to write your code meticulously to accomodate them anyways. Yes, cache is important. But there's a world of difference between the "flush the caches and the TLB" behavior that system calls used to incur and the "expensive relative to local variable access" behavior we're talking about here. "A bounce to non-local, non-current execution environment" literally doesn't mean anything. You could be talking about the CR3 change that swaps page hierarchies, the CPL3->0 change that allows privileged instructions, the CS change that got you there, or even the ESP change to "swap stacks". Which of these are expensive? I'm chasing you down because it's fun, not because I think you don't know what you're talking about. But, let's make this relevant: if system call overhead is drowned by disk I/O overhead, then one argument for mmap goes away.
- ajross 18y agoWhy do you insist on making this a flame war? Sit down and write a program that makes 10M trivial system calls (getpid() is a good choice) and one that calls a function to return a constant integer 10M times. I'm sorry, but you seem to have a wildly inflated idea of the speed of system calls on modern OSs. The real world just doesn't work like that. This is my last post on this thread.
- dhess 18y agoWhy would you flush the cache on a system call?? The only case I can think of where a cache flush is needed at all is if you have a virtually-addressed (or -indexed) cache and you're switching address spaces. Some ARM implementations may do this, if I recall correctly. But entering the kernel doesn't require an address space switch because the kernel is mapped into every process's address space, at the same virtual address. Also, as I pointed out below, typically you don't even need to change any TLB entries on kernel entry, because the address space doesn't change and the kernel TLB mappings are usually pinned. In other words, any load from or store to a kernel page will always hit the TLB. On most architectures, you probably don't even need to swap registers; not the general-purpose ones, anyway. Sorry, but you're confusing switching processes with switching privilege, and greatly overestimating the cost of a system call.