5 ms·

> the reason why you'd pick Fil-C is that it's actually memory safe for real, unlike the alternatives, which just give you safety for newly written code but pul

by josephg 11mo ago

> the reason why you'd pick Fil-C is that it's actually memory safe for real, unlike the alternatives, which just give you safety for newly written code but pull in a ton of unsafe depenendencies. Right; that makes a lot of sense. So currently the only implementation of pam is written in Yolo-C. If you want to build a safe, secure tool which depends on pam - like sudo - then you either need to trust pam (like sudo-rs does). Or you need some way to compile pam to safe code. Fil-C is a way to do that. Just to name them - I can think of two other solutions to this problem: 1. Sandbox pam within a wasm container. This is probably less convenient than Fil-C (although it would allow that wasm container to be linked from yolo-c, rust, go, js, etc). For pam in particular, this would also be much less safe than Fil-C, since memory bugs in pam would still be possible, and might still result in privilege escalation. Wasm only sandboxes memory at the boundary. It wouldn't protect pam itself from its own bugs like Fil-C does. 2. Rewrite pam itself in a safer language, like C#, Go or Rust. Then sudo-rs doesn't need to link to yolo C code. I suspect someone will do this soon. I think this also reaffirms the role I see for Fil-C: as a tool to make legacy C code (like pam) safe, at the cost of performance. For new code, I'd rather pick a fully GC language if I don't care about performance, or use rust if performance matters. Again, it would be nice if I could have the best of both worlds - use fil-c to wrap and sandbox pam, but write the rest of my program in rust or C# or Go or something else that is already safe without needing to pay fil-c's cost. > But, unlike Rust, Fil-C does not attempt to ensure race-freedom for your own logic. It depends what you're doing, but this can be a pretty big deal. I suppose this puts Fil-C in about the same safety and performance camp as Go, Java and C#. Ie, it has some protections that rust is missing - like there aren't any unsafe blocks. And its missing some protections rust has - like better thread safety. The unique selling point is it works with existing C code, like old C libraries that we don't want to rewrite. Thanks for your work on it!

pizlonator 11mo ago
> 2. Rewrite pam itself in a safer language, like C#, Go or Rust. Then sudo-rs doesn't need to link to yolo C code. I suspect someone will do this soon. OK, let's think about that for a moment. Go and Rust do not have dynamic linking. Well, Rust sort of does, if you're OK with using C ABI (i.e. `unsafe`). C# only has dynamic loading if you aren't using the AOT. That means you're JITing. I don't think JITing scales to every process on your system doing it (you get no sharing of code between processes, and if you consider that I've got >500 processes on the machine I'm using right now, you can imagine how much of a memory burden JIT-in-everything would be). And pam is engineered around dynamic linking. That's sort of the whole point of how it works. Your pam stack refers to modules that get dynamically loaded, which then allows pam modules to pull in things loads of other libraries. So: there is currently no scalable alternative to Fil-C for making pam safe.
- josephg 11mo ago
  Does Fil-C support dynamic linking to "yolo C" code or only modules that have also been compiled using fil-C? Would all the pam modules (eg pam_systemd.so, pam_selinux.so, etc) also need to be compiled with fil-c? I assume so, to make it safe? In general dynamic linking support is something that rust, go and C# could definitely improve upon. As I understand it, Rust does support dynamic linking for pure rust code. It just doesn't have a stable ABI yet. So dynamic linking is only supported within the same version of the rust compiler. Taking a look at pam, it looks like there's 2 parts: - The configuration parser & dynamic loader, which would be trivial to rewrite in rust or C# or something. - The actual dynamically loaded modules, and the ABI bridge to call into them. I think the obvious way to port pam to rust or C# would be to rewrite the core. Then just statically link in all the standard, built-in pam modules (like pam_unix). And - at least for now - handle "external" modules in exactly the same "yolo C" dyld way they are handled today. Yes, that means a dozen lines of boilerplate unsafe rust code across the C ABI bridge. It wouldn't be quite as safe as Fil-C or C# JITting for everything. But its within my personal safety paranoia tolerance. It would certainly be much safer than pam is today, given currently every single line is in yolo mode. It would also mean those modules wouldn't need to all be rewritten in rust all at once, or at all. If you want a fully safe replacement for the pam bridge maybe wasm+wasi could work here? I'm not sure. I certainly hear you that this is a problem in rust, go and c#, and I'm not entirely sure what a good solution would look like. Well, other than rust stabilizing its ABI. At the pace rust moves at, that could be any decade now. > I don't think JITing scales to every process on your system doing it (you get no sharing of code between processes, and if you consider that I've got >500 processes on the machine I'm using right now, you can imagine how much of a memory burden JIT-in-everything would be). Forget every program for a moment. C#'s JIT would certainly be fast enough for pam. And most linux programs don't make such heavy use of dynamic linking to work. So they could be AOT compiled. On the topic of performance, I'm not sure I want to pay the fil-C performance overhead for all 500+ processes on my system either. I assume the performance ceiling for fil-c is going to be somewhere around the performance of other well optimized GC runtimes. I don't know of any GC language that does better than about a 2x memory usage overhead. Call me crazy, but for general computing, I like the speed of native code.
  - 11mo ago
binary132 11mo ago
I think the whole point is to _not_ rewrite anything.
- josephg 11mo ago
  I mean, that's a real advantage. But personally I'm not so keen for all my system software to have the performance of garbage collected languages. I don't want my software to suddenly get 1.5x-4x slower and start using twice as much RAM. For pam and sudo it wouldn't matter, sure. But across the whole linux system? I'd rather a slow and careful rewrite in rust over that sort of system wide slowdown.
  - drnick1 11mo ago
    There will never be a slow and careful rewrite in Rust. Nobody wants to do it, and doing it without introducing new bugs and security issues (not linked to memory) is even more difficult.
    - josephg 11mo ago
      > There will never be a slow and careful rewrite in Rust. Nobody wants to do it Lots of people want to do it. Whether you want to run any of their code is an open question - but rewriting old things from scratch is a very popular passtime. In fact, people are already doing it. For example, the recent sudo-rs rewrite. Or the uutils coreutils package: https://github.com/uutils/coreutils https://github.com/uutils/coreutils - Which passes almost all conformance tests now by the way. (Though they could certainly use a few more tests): https://uutils.github.io/coreutils/docs/test_coverage.html https://uutils.github.io/coreutils/docs/test_coverage.html > doing it without introducing new bugs and security issues (not linked to memory) is even more difficult. Difficult, but not impossible. Plenty of alternate C compilers exist. And alternate SQL databases. And some web browsers. And email servers. And TLS implementations. And on and on. Arguably there already are several reimplementations of the "linux userland", with various tweaks. They're just called Freebsd. And Illumos. And MacOS. Why would the linux userland be uniquely impossible to reimplement in another language? I feel like people celebrate when someone makes a new text editor or gameboy emulator. Why so much hate at the idea of more implementations of unix tools?

by josephg 11mo ago

pizlonator 11mo ago

> 2. Rewrite pam itself in a safer language, like C#, Go or Rust. Then sudo-rs doesn't need to link to yolo C code. I suspect someone will do this soon. OK, let's think about that for a moment. Go and Rust do not have dynamic linking. Well, Rust sort of does, if you're OK with using C ABI (i.e. `unsafe`). C# only has dynamic loading if you aren't using the AOT. That means you're JITing. I don't think JITing scales to every process on your system doing it (you get no sharing of code between processes, and if you consider that I've got >500 processes on the machine I'm using right now, you can imagine how much of a memory burden JIT-in-everything would be). And pam is engineered around dynamic linking. That's sort of the whole point of how it works. Your pam stack refers to modules that get dynamically loaded, which then allows pam modules to pull in things loads of other libraries. So: there is currently no scalable alternative to Fil-C for making pam safe.

josephg 11mo ago
Does Fil-C support dynamic linking to "yolo C" code or only modules that have also been compiled using fil-C? Would all the pam modules (eg pam_systemd.so, pam_selinux.so, etc) also need to be compiled with fil-c? I assume so, to make it safe? In general dynamic linking support is something that rust, go and C# could definitely improve upon. As I understand it, Rust does support dynamic linking for pure rust code. It just doesn't have a stable ABI yet. So dynamic linking is only supported within the same version of the rust compiler. Taking a look at pam, it looks like there's 2 parts: - The configuration parser & dynamic loader, which would be trivial to rewrite in rust or C# or something. - The actual dynamically loaded modules, and the ABI bridge to call into them. I think the obvious way to port pam to rust or C# would be to rewrite the core. Then just statically link in all the standard, built-in pam modules (like pam_unix). And - at least for now - handle "external" modules in exactly the same "yolo C" dyld way they are handled today. Yes, that means a dozen lines of boilerplate unsafe rust code across the C ABI bridge. It wouldn't be quite as safe as Fil-C or C# JITting for everything. But its within my personal safety paranoia tolerance. It would certainly be much safer than pam is today, given currently every single line is in yolo mode. It would also mean those modules wouldn't need to all be rewritten in rust all at once, or at all. If you want a fully safe replacement for the pam bridge maybe wasm+wasi could work here? I'm not sure. I certainly hear you that this is a problem in rust, go and c#, and I'm not entirely sure what a good solution would look like. Well, other than rust stabilizing its ABI. At the pace rust moves at, that could be any decade now. > I don't think JITing scales to every process on your system doing it (you get no sharing of code between processes, and if you consider that I've got >500 processes on the machine I'm using right now, you can imagine how much of a memory burden JIT-in-everything would be). Forget every program for a moment. C#'s JIT would certainly be fast enough for pam. And most linux programs don't make such heavy use of dynamic linking to work. So they could be AOT compiled. On the topic of performance, I'm not sure I want to pay the fil-C performance overhead for all 500+ processes on my system either. I assume the performance ceiling for fil-c is going to be somewhere around the performance of other well optimized GC runtimes. I don't know of any GC language that does better than about a 2x memory usage overhead. Call me crazy, but for general computing, I like the speed of native code.
- 11mo ago

binary132 11mo ago

I think the whole point is to _not_ rewrite anything.

josephg 11mo ago
I mean, that's a real advantage. But personally I'm not so keen for all my system software to have the performance of garbage collected languages. I don't want my software to suddenly get 1.5x-4x slower and start using twice as much RAM. For pam and sudo it wouldn't matter, sure. But across the whole linux system? I'd rather a slow and careful rewrite in rust over that sort of system wide slowdown.
- drnick1 11mo ago
  There will never be a slow and careful rewrite in Rust. Nobody wants to do it, and doing it without introducing new bugs and security issues (not linked to memory) is even more difficult.
  - josephg 11mo ago
    > There will never be a slow and careful rewrite in Rust. Nobody wants to do it Lots of people want to do it. Whether you want to run any of their code is an open question - but rewriting old things from scratch is a very popular passtime. In fact, people are already doing it. For example, the recent sudo-rs rewrite. Or the uutils coreutils package: https://github.com/uutils/coreutils https://github.com/uutils/coreutils - Which passes almost all conformance tests now by the way. (Though they could certainly use a few more tests): https://uutils.github.io/coreutils/docs/test_coverage.html https://uutils.github.io/coreutils/docs/test_coverage.html > doing it without introducing new bugs and security issues (not linked to memory) is even more difficult. Difficult, but not impossible. Plenty of alternate C compilers exist. And alternate SQL databases. And some web browsers. And email servers. And TLS implementations. And on and on. Arguably there already are several reimplementations of the "linux userland", with various tweaks. They're just called Freebsd. And Illumos. And MacOS. Why would the linux userland be uniquely impossible to reimplement in another language? I feel like people celebrate when someone makes a new text editor or gameboy emulator. Why so much hate at the idea of more implementations of unix tools?