3 ms·
Why not remove syscall instructions altogether? When libc wants to do something, it traps on an undefined instruction and then the kernel looks at the program c
by saagarjha 3y ago
Why not remove syscall instructions altogether? When libc wants to do something, it traps on an undefined instruction and then the kernel looks at the program counter to see what it should do. Seems like this would be the ultimate application of this line of thought…
- nine_k 3y agoAre there any performance implications on such traps on x64 or ARM? IIRC, PDP11 and VAX used traps as the way to call the supervisor, and it was pretty cheap.
- monocasa 3y agoI don't think that helps much. OpenBSD already only allows syscalls originating out of the libc .text section, so whether the trap itself comes from a syscall instruction or some other trap mechanism doesn't really improve security AFAICT.
- saagarjha 3y agoYeah but it sounds super cool doesn’t it!
- monocasa 3y agoEvery time I've seen a dev team go down that road, it's come with rather unfortunate unintended side effects. https://devblogs.microsoft.com/oldnewthing/20041215-00/?p=37003 https://devblogs.microsoft.com/oldnewthing/20041215-00/?p=37...
- saagarjha 3y agoYeah but think of the attacks I thought of in the shower that it mitigates
- jcranmer 3y agoMy understanding is this was done by one system, which then made the architecture's life hard because now they lost half their opcode encoding space due to it being used as pseudo-syscalls by their largest customer.
- saagarjha 3y agoJust one encoding, or maybe we’re thinking of different systems?
- Veserv 3y agoThat is basically what we used to do, int 0x80 and such. I mean, I guess you do not read the PC since that is much more brittle than just having the caller say what they want to do, but it is structurally the same. Turns out, having a dedicated syscall instruction and trap pathway is just better design. Unlike a regular exception, this is a deliberate change of control to the kernel, so you can enforce a much stronger ABI requirement. In particular, you can define it to use a standard function call ABI with respect to preserved and non-preserved registers making it literally look like a standard function call. For similar reasons, having a dedicated hardware pathway like on x86-64 is also just better design. System calls are a synchronous, voluntary transfer of control that is expected to return in contrast to (1) interrupts which are a asynchronous involuntary transfer of control and (2) instruction stream exceptions which are a synchronous involuntary transfer of control with no guarantee of return. This fundamental distinction can be leveraged for more efficient and simpler implementations.