5 ms·

I was just discussing this sort of thing with some colleagues. Because the stack frame for main contains a bunch of other stuff - environment variables, cli arg

I was just discussing this sort of thing with some colleagues. Because the stack frame for main contains a bunch of other stuff - environment variables, cli args, etc - it makes it unreliable to try to instrument Linux systems and collect that information. A process can change its name, args, env, at any time. That sort of information is really helpful for forensics. Currently your only option is to pull data directly from kernel structures, which is fine, but most people aren't doing that. And then of course there's other cool stuff like being able to 'lock' system calls to the system provided libc, which openbsd can do since they already only support their provided libc. Unfortunately none of this is likely to get to Linux at any point because: a) I bet some insane userland processes fuck with their stacks b) Linux doesn't mandate any libc edit: Seems like there's some "what is this" being asked. Here's my loose understand! For starters, libc is the interface to the kernel. Very few programs make syscalls directly (except go programs i guess? lol) on Linux, but on openbsd it's just flat out not allowed. OpenBSD doesn't have the "GNU/Linux" dichotomy - it's all one package deal. As such, they get to mandate how userland calls into the kernel. Of course, you don't have to listen to openbsd today, because you can just... make those system calls directly. Easy. And this is relevant for security because attackers will do something called ROP, which effectively "reuses" bits of your already-executable code to issue system calls (you can google "return to libc" or "ret2libc"). So, first thing's first, have the kernel actually ensure that the sycall is issued from the memory that libc is mapped into. Cool, solved sort of. But what if the attacker overwrites that libc? Now you've verified that the call is coming from libc but you don't have integrity. With immutability the kernel will now enforce that system calls come from libc and that the code can't be overwritten. Of course, this can extend beyond libc, but I'm assuming that's the main point. This would presumably be a general system call that programs can use themselves to do all sorts of fun things. edit: Can't respond to replies, HN rate limited me :) sorry. Sounds like I need to read up a bit more, this doesn't address the use case I'd had in mind sadly, but still very cool nonetheless.

jart 4y ago
> Linux doesn't mandate any libc Neither does OpenBSD. You're misunderstanding how msyscall() works. It's like Highlander. There can be only one.
- staticassertion 4y ago
  Ah, interesting, I'll have to dig into it more :) I've never used OpenBSD so I've just had to watch on the sidelines. Anywhere I can read more about this?
  - jart 4y ago
    The msyscall() man page. The Cosmopolitan Libc source code. In the cosmoverse, we produce static executables that run on six operating systems. The only Libc we use is obviously our own. So when our executables get loaded on OpenBSD, we do the same thing OpenBSD Libc does, which is calling msyscall() so that only a privileged subset of the application is able to use SYSCALL. In our static binary world, that's basically any function that uses the `privileged` keyword. It makes things a little dicey if we want to dynamically link OpenBSD Libc since we have to pull out some aggressive hacks like Xed in that case. But overall it's fine.
eqvinox 4y ago
> Because the stack frame for main contains a bunch of other stuff - environment variables, cli args, etc - […] > a) I bet some insane userland processes fuck with their stacks The new OpenBSD `mimmutable()` call, when applied on the stack, does absolutely nothing to prevent writing or reading the stack. It prevents changing the (virtual memory/pagetable) mapping itself, i.e. mapping in something else, removing the mapping, or changing the mapping attributes (primarily RWX). Even if some userland process requires an executable stack, that would be indicated with ELF flags and set up by the loader - and then frozen in place.
- jart 4y ago
  Well it technically does if you make a new stack with mmap(MAP_STACK) (or simply round down the stack pointer by a page size) and remove the write access to the top of the original stack and mimmutable it.
  - eqvinox 4y ago
    Problem is that those "magic variables" are defined by ABI to be directly above main()'s stack frame… if you're throwing that out, you may equally well leave the stack alone and put the magic stuff in some new place :) … Actually, now that I think about it, I guess you could try to put the boundary between main() and the magic bits right on a page boundary, and then change attrs on the magic bits?
    - jart 4y ago
      You mean _start()'s stack pointer. Libc can secure them and it's fine. It would probably break the ANSI C standard though if you can't modify argv and environ though. So Libc would need to copy them. But that limits the usefulness of the original tooling proposal unless the operating system has something like /proc/pid/maps that tells you the extent of the system provided stack. I don't think OpenBSD has that. Please prove me wrong since if it does, I'd love to know how.
      - eqvinox 4y ago
saagarjha 4y ago
> And this is relevant for security because attackers will do something called ROP, which effectively "reuses" bits of your already-executable code to issue system calls (you can google "return to libc" or "ret2libc"). You’ve just described why system call integrity is useless without strong CFI :)
leni536 4y ago
> a) I bet some insane userland processes fuck with their stacks What do you mean by "insane" here? In standard C you get char** argv, and the arguments live on the stack. I believe you are allowed to modify those cli arguments through the pointers, it is not undefined behavior.
- staticassertion 4y ago
  I never called it undefined behavior. But you shouldn't modify that stack space directly, you should either use setenv via libc or you should use prctl.
10000truths 4y ago
> Very few programs make syscalls directly (except go programs i guess? lol) on Linux You’re forgetting statically compiled binaries, which are very important for ease of deployment and distro-agnostic execution.
- staticassertion 4y ago
  Statically compiled binaries almost always still use a libc, either by dynamically linking it or by statically linking it.

jart 4y ago

> Linux doesn't mandate any libc Neither does OpenBSD. You're misunderstanding how msyscall() works. It's like Highlander. There can be only one.

staticassertion 4y ago
Ah, interesting, I'll have to dig into it more :) I've never used OpenBSD so I've just had to watch on the sidelines. Anywhere I can read more about this?
- jart 4y ago
  The msyscall() man page. The Cosmopolitan Libc source code. In the cosmoverse, we produce static executables that run on six operating systems. The only Libc we use is obviously our own. So when our executables get loaded on OpenBSD, we do the same thing OpenBSD Libc does, which is calling msyscall() so that only a privileged subset of the application is able to use SYSCALL. In our static binary world, that's basically any function that uses the `privileged` keyword. It makes things a little dicey if we want to dynamically link OpenBSD Libc since we have to pull out some aggressive hacks like Xed in that case. But overall it's fine.

eqvinox 4y ago

> Because the stack frame for main contains a bunch of other stuff - environment variables, cli args, etc - […] > a) I bet some insane userland processes fuck with their stacks The new OpenBSD `mimmutable()` call, when applied on the stack, does absolutely nothing to prevent writing or reading the stack. It prevents changing the (virtual memory/pagetable) mapping itself, i.e. mapping in something else, removing the mapping, or changing the mapping attributes (primarily RWX). Even if some userland process requires an executable stack, that would be indicated with ELF flags and set up by the loader - and then frozen in place.

jart 4y ago
Well it technically does if you make a new stack with mmap(MAP_STACK) (or simply round down the stack pointer by a page size) and remove the write access to the top of the original stack and mimmutable it.
- eqvinox 4y ago
  Problem is that those "magic variables" are defined by ABI to be directly above main()'s stack frame… if you're throwing that out, you may equally well leave the stack alone and put the magic stuff in some new place :) … Actually, now that I think about it, I guess you could try to put the boundary between main() and the magic bits right on a page boundary, and then change attrs on the magic bits?
  - jart 4y ago
    You mean _start()'s stack pointer. Libc can secure them and it's fine. It would probably break the ANSI C standard though if you can't modify argv and environ though. So Libc would need to copy them. But that limits the usefulness of the original tooling proposal unless the operating system has something like /proc/pid/maps that tells you the extent of the system provided stack. I don't think OpenBSD has that. Please prove me wrong since if it does, I'd love to know how.
    - eqvinox 4y ago

saagarjha 4y ago

> And this is relevant for security because attackers will do something called ROP, which effectively "reuses" bits of your already-executable code to issue system calls (you can google "return to libc" or "ret2libc"). You’ve just described why system call integrity is useless without strong CFI :)

leni536 4y ago

> a) I bet some insane userland processes fuck with their stacks What do you mean by "insane" here? In standard C you get char** argv, and the arguments live on the stack. I believe you are allowed to modify those cli arguments through the pointers, it is not undefined behavior.

staticassertion 4y ago
I never called it undefined behavior. But you shouldn't modify that stack space directly, you should either use setenv via libc or you should use prctl.

10000truths 4y ago

> Very few programs make syscalls directly (except go programs i guess? lol) on Linux You’re forgetting statically compiled binaries, which are very important for ease of deployment and distro-agnostic execution.

staticassertion 4y ago
Statically compiled binaries almost always still use a libc, either by dynamically linking it or by statically linking it.