4 ms·

> 2. Rewrite pam itself in a safer language, like C#, Go or Rust. Then sudo-rs doesn't need to link to yolo C code. I suspect someone will do this soon. OK, le

> 2. Rewrite pam itself in a safer language, like C#, Go or Rust. Then sudo-rs doesn't need to link to yolo C code. I suspect someone will do this soon. OK, let's think about that for a moment. Go and Rust do not have dynamic linking. Well, Rust sort of does, if you're OK with using C ABI (i.e. `unsafe`). C# only has dynamic loading if you aren't using the AOT. That means you're JITing. I don't think JITing scales to every process on your system doing it (you get no sharing of code between processes, and if you consider that I've got >500 processes on the machine I'm using right now, you can imagine how much of a memory burden JIT-in-everything would be). And pam is engineered around dynamic linking. That's sort of the whole point of how it works. Your pam stack refers to modules that get dynamically loaded, which then allows pam modules to pull in things loads of other libraries. So: there is currently no scalable alternative to Fil-C for making pam safe.

josephg 11mo ago
Does Fil-C support dynamic linking to "yolo C" code or only modules that have also been compiled using fil-C? Would all the pam modules (eg pam_systemd.so, pam_selinux.so, etc) also need to be compiled with fil-c? I assume so, to make it safe? In general dynamic linking support is something that rust, go and C# could definitely improve upon. As I understand it, Rust does support dynamic linking for pure rust code. It just doesn't have a stable ABI yet. So dynamic linking is only supported within the same version of the rust compiler. Taking a look at pam, it looks like there's 2 parts: - The configuration parser & dynamic loader, which would be trivial to rewrite in rust or C# or something. - The actual dynamically loaded modules, and the ABI bridge to call into them. I think the obvious way to port pam to rust or C# would be to rewrite the core. Then just statically link in all the standard, built-in pam modules (like pam_unix). And - at least for now - handle "external" modules in exactly the same "yolo C" dyld way they are handled today. Yes, that means a dozen lines of boilerplate unsafe rust code across the C ABI bridge. It wouldn't be quite as safe as Fil-C or C# JITting for everything. But its within my personal safety paranoia tolerance. It would certainly be much safer than pam is today, given currently every single line is in yolo mode. It would also mean those modules wouldn't need to all be rewritten in rust all at once, or at all. If you want a fully safe replacement for the pam bridge maybe wasm+wasi could work here? I'm not sure. I certainly hear you that this is a problem in rust, go and c#, and I'm not entirely sure what a good solution would look like. Well, other than rust stabilizing its ABI. At the pace rust moves at, that could be any decade now. > I don't think JITing scales to every process on your system doing it (you get no sharing of code between processes, and if you consider that I've got >500 processes on the machine I'm using right now, you can imagine how much of a memory burden JIT-in-everything would be). Forget every program for a moment. C#'s JIT would certainly be fast enough for pam. And most linux programs don't make such heavy use of dynamic linking to work. So they could be AOT compiled. On the topic of performance, I'm not sure I want to pay the fil-C performance overhead for all 500+ processes on my system either. I assume the performance ceiling for fil-c is going to be somewhere around the performance of other well optimized GC runtimes. I don't know of any GC language that does better than about a 2x memory usage overhead. Call me crazy, but for general computing, I like the speed of native code.
- pizlonator 11mo ago
  > As I understand it, Rust does support dynamic linking for pure rust code. It just doesn't have a stable ABI yet. Not quite. To safely run a Rust program against a Rust dynamic library, you had to have compiled and linked that program against exactly that version of the Rust library. This is necessary for fundamental Rust type system reasons. It’s not simply a matter of stabilizing ABI. It’s not like updating the library without relinking would work if you used exactly the same rustc version - it would still be unsafe. So no, Rust doesn’t support dynamic linking in any meaningful way. > And most linux programs don't make such heavy use of dynamic linking to work. This is not true at all. Linux relies on dynamic linking extensively in two ways: - A dynamic library may be updated independently of its dependents. You can’t do this in Rust, see above. - Lots of Linux programs rely on the dynamically loaded module architecture like what pam does. All of the scripting language runtimes do this for example.
  - gpm 11mo ago
    The ability to replace libraries without recompiling downstream dependents is not a necessary component of dynamic linking. However if rust wanted to gain this ability, it basically is just a matter of a stable ABI. The only other thing you would need is to get a hash of all the parts of the system that are equivalent to C header files and have the program check the hash in the dylib matched the version it was compiled against at load time. I believe in fact the sdylib crate type in nightly rust does (or at least is intended to do, not sure it's fully implemented) exactly this.
    - pizlonator 11mo ago
      > The ability to replace libraries without recompiling downstream dependents is not a necessary component of dynamic linking. Yes it is, in the sense that not having this is a massive bug. Dynamic linking becomes horrifically unsafe without this property, rendering it mostly useless.
      - gpm 11mo ago
        It's not a bug, it's just a lack of an additional feature on top of dynamic linking. In fact it prevents bugs as it provides a clear bar on the misuse of dynamic linking as is so common in C/C++/(as designed) Fil-C to replace libraries with likely-ABI-incompatible replacements. Fil-C limits the blast radius of bugs caused by this, but doesn't prevent the bugs. Just saying "you can't do that" like current rust actually prevents them. (Alternatively growing that feature properly, including checking ABI compatibility, as intended with sdylib's would also prevent them. To the best of my knowledge no C-like language does this today) Dynmaic linking used responsibly to reduce linking times in development cycles, and to reduce memory usage when you have memory things using the same library, remains perfectly safe without this feature. The former is reasonably common in the rust ecosystem today, and I've literally never hit or heard of anyone else hitting a safety issue as a result. Minimizing memory usage is a niche use case in the modern world... but it's there if you want it. You just have to not modify the application post compilation - just like you have to not modify any application post compilation to retain memory safety guarantees.
gpm 11mo ago
> Well, Rust sort of does, if you're OK with using C ABI (i.e. `unsafe`). Rust has full blown safe dynamic linking - so long as you use rust, and the the same compiler version, for everything. What it doesn't have is runtime libdl support for that. Which would be a problem for the current pam architecture. I'm not particularly convinced you couldn't just re-architect pam to use static linking though. Or a scripting language.
- pizlonator 11mo ago
  > Rust has full blown safe dynamic linking - so long as you use rust, and the the same compiler version, for everything. This is UB in rust as far as I understand: 1) you compile bar using version 1.5 of libfoo.so at link/compile time 2) you update libfoo.so to version 1.6. It has a lot of changes, but is still source compatible and some kind of reasonable precautions were taken, like that struct layouts are the same 3) you run bar with the new libfoo.so version 1.6 This is UB even if you use the same compiler. If this is UB then that’s not “full blown safe dynamic linking”. Note that Fil-C does safely support this. This is safe in Fil-C even if you don’t take appropriate precautions (like if you change function signatures, change struct layouts, etc - it’ll still be safe and have well defined outcomes in Fil-C).
  - gpm 11mo ago
    What you're looking for is a stable ABI, not just dynamic linking. Honestly not sure if rust guarantees a stable ABI under some set of "reasonable" precautions... likely not but it's never been something I've needed when I've even wanted in the context of matching rust's other constraint of matching compiler versions.
pjmlp 11mo ago
C# supports dynamic loading in AOT compilation. Native AOT, and the oldie NGEN only do dynamic linking. https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/libraries https://learn.microsoft.com/en-us/dotnet/core/deploying/nati... .NET Native can do both, Applications are statically compiled, libraries are compiled into WinRT components, COM's evolution, introduced in Windows 8. https://learn.microsoft.com/en-us/windows/uwp/dotnet-native/net-native-and-compilation https://learn.microsoft.com/en-us/windows/uwp/dotnet-native/... Just watched the 2024 talk by the way, quite interesting.
- neonsunset 11mo ago
  .NET Native is essentially dead. NGEN is specific to .NET Framework. It is possible to dynamically link multiple binaries with the same runtime with NativeAOT but it is very exotic and undocumented path that is not guaranteed to be stable. Not much worse than Go's dynamic linking which also requires the same compiler version.
  - pjmlp 11mo ago
    In theory, in practice the almost resourless WinUI/WinAppSDK team is yet to change the minds of the few companies that believed in UWP to move away from it, due to the amount of missing features after 5 years of Project Reunion. In fact, the ones that care to port their applications might create Web versions than bother with Native AOT.

josephg 11mo ago

Does Fil-C support dynamic linking to "yolo C" code or only modules that have also been compiled using fil-C? Would all the pam modules (eg pam_systemd.so, pam_selinux.so, etc) also need to be compiled with fil-c? I assume so, to make it safe? In general dynamic linking support is something that rust, go and C# could definitely improve upon. As I understand it, Rust does support dynamic linking for pure rust code. It just doesn't have a stable ABI yet. So dynamic linking is only supported within the same version of the rust compiler. Taking a look at pam, it looks like there's 2 parts: - The configuration parser & dynamic loader, which would be trivial to rewrite in rust or C# or something. - The actual dynamically loaded modules, and the ABI bridge to call into them. I think the obvious way to port pam to rust or C# would be to rewrite the core. Then just statically link in all the standard, built-in pam modules (like pam_unix). And - at least for now - handle "external" modules in exactly the same "yolo C" dyld way they are handled today. Yes, that means a dozen lines of boilerplate unsafe rust code across the C ABI bridge. It wouldn't be quite as safe as Fil-C or C# JITting for everything. But its within my personal safety paranoia tolerance. It would certainly be much safer than pam is today, given currently every single line is in yolo mode. It would also mean those modules wouldn't need to all be rewritten in rust all at once, or at all. If you want a fully safe replacement for the pam bridge maybe wasm+wasi could work here? I'm not sure. I certainly hear you that this is a problem in rust, go and c#, and I'm not entirely sure what a good solution would look like. Well, other than rust stabilizing its ABI. At the pace rust moves at, that could be any decade now. > I don't think JITing scales to every process on your system doing it (you get no sharing of code between processes, and if you consider that I've got >500 processes on the machine I'm using right now, you can imagine how much of a memory burden JIT-in-everything would be). Forget every program for a moment. C#'s JIT would certainly be fast enough for pam. And most linux programs don't make such heavy use of dynamic linking to work. So they could be AOT compiled. On the topic of performance, I'm not sure I want to pay the fil-C performance overhead for all 500+ processes on my system either. I assume the performance ceiling for fil-c is going to be somewhere around the performance of other well optimized GC runtimes. I don't know of any GC language that does better than about a 2x memory usage overhead. Call me crazy, but for general computing, I like the speed of native code.

pizlonator 11mo ago
> As I understand it, Rust does support dynamic linking for pure rust code. It just doesn't have a stable ABI yet. Not quite. To safely run a Rust program against a Rust dynamic library, you had to have compiled and linked that program against exactly that version of the Rust library. This is necessary for fundamental Rust type system reasons. It’s not simply a matter of stabilizing ABI. It’s not like updating the library without relinking would work if you used exactly the same rustc version - it would still be unsafe. So no, Rust doesn’t support dynamic linking in any meaningful way. > And most linux programs don't make such heavy use of dynamic linking to work. This is not true at all. Linux relies on dynamic linking extensively in two ways: - A dynamic library may be updated independently of its dependents. You can’t do this in Rust, see above. - Lots of Linux programs rely on the dynamically loaded module architecture like what pam does. All of the scripting language runtimes do this for example.
- gpm 11mo ago
  The ability to replace libraries without recompiling downstream dependents is not a necessary component of dynamic linking. However if rust wanted to gain this ability, it basically is just a matter of a stable ABI. The only other thing you would need is to get a hash of all the parts of the system that are equivalent to C header files and have the program check the hash in the dylib matched the version it was compiled against at load time. I believe in fact the sdylib crate type in nightly rust does (or at least is intended to do, not sure it's fully implemented) exactly this.
  - pizlonator 11mo ago
    > The ability to replace libraries without recompiling downstream dependents is not a necessary component of dynamic linking. Yes it is, in the sense that not having this is a massive bug. Dynamic linking becomes horrifically unsafe without this property, rendering it mostly useless.
    - gpm 11mo ago
      It's not a bug, it's just a lack of an additional feature on top of dynamic linking. In fact it prevents bugs as it provides a clear bar on the misuse of dynamic linking as is so common in C/C++/(as designed) Fil-C to replace libraries with likely-ABI-incompatible replacements. Fil-C limits the blast radius of bugs caused by this, but doesn't prevent the bugs. Just saying "you can't do that" like current rust actually prevents them. (Alternatively growing that feature properly, including checking ABI compatibility, as intended with sdylib's would also prevent them. To the best of my knowledge no C-like language does this today) Dynmaic linking used responsibly to reduce linking times in development cycles, and to reduce memory usage when you have memory things using the same library, remains perfectly safe without this feature. The former is reasonably common in the rust ecosystem today, and I've literally never hit or heard of anyone else hitting a safety issue as a result. Minimizing memory usage is a niche use case in the modern world... but it's there if you want it. You just have to not modify the application post compilation - just like you have to not modify any application post compilation to retain memory safety guarantees.

gpm 11mo ago

> Well, Rust sort of does, if you're OK with using C ABI (i.e. `unsafe`). Rust has full blown safe dynamic linking - so long as you use rust, and the the same compiler version, for everything. What it doesn't have is runtime libdl support for that. Which would be a problem for the current pam architecture. I'm not particularly convinced you couldn't just re-architect pam to use static linking though. Or a scripting language.

pizlonator 11mo ago
> Rust has full blown safe dynamic linking - so long as you use rust, and the the same compiler version, for everything. This is UB in rust as far as I understand: 1) you compile bar using version 1.5 of libfoo.so at link/compile time 2) you update libfoo.so to version 1.6. It has a lot of changes, but is still source compatible and some kind of reasonable precautions were taken, like that struct layouts are the same 3) you run bar with the new libfoo.so version 1.6 This is UB even if you use the same compiler. If this is UB then that’s not “full blown safe dynamic linking”. Note that Fil-C does safely support this. This is safe in Fil-C even if you don’t take appropriate precautions (like if you change function signatures, change struct layouts, etc - it’ll still be safe and have well defined outcomes in Fil-C).
- gpm 11mo ago
  What you're looking for is a stable ABI, not just dynamic linking. Honestly not sure if rust guarantees a stable ABI under some set of "reasonable" precautions... likely not but it's never been something I've needed when I've even wanted in the context of matching rust's other constraint of matching compiler versions.

pjmlp 11mo ago

C# supports dynamic loading in AOT compilation. Native AOT, and the oldie NGEN only do dynamic linking. https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/libraries https://learn.microsoft.com/en-us/dotnet/core/deploying/nati... .NET Native can do both, Applications are statically compiled, libraries are compiled into WinRT components, COM's evolution, introduced in Windows 8. https://learn.microsoft.com/en-us/windows/uwp/dotnet-native/net-native-and-compilation https://learn.microsoft.com/en-us/windows/uwp/dotnet-native/... Just watched the 2024 talk by the way, quite interesting.

neonsunset 11mo ago
.NET Native is essentially dead. NGEN is specific to .NET Framework. It is possible to dynamically link multiple binaries with the same runtime with NativeAOT but it is very exotic and undocumented path that is not guaranteed to be stable. Not much worse than Go's dynamic linking which also requires the same compiler version.
- pjmlp 11mo ago
  In theory, in practice the almost resourless WinUI/WinAppSDK team is yet to change the minds of the few companies that believed in UWP to move away from it, due to the amount of missing features after 5 years of Project Reunion. In fact, the ones that care to port their applications might create Web versions than bother with Native AOT.