5 ms·

Does Fil-C support dynamic linking to "yolo C" code or only modules that have also been compiled using fil-C? Would all the pam modules (eg pam_systemd.so, pam_

by josephg 11mo ago

Does Fil-C support dynamic linking to "yolo C" code or only modules that have also been compiled using fil-C? Would all the pam modules (eg pam_systemd.so, pam_selinux.so, etc) also need to be compiled with fil-c? I assume so, to make it safe? In general dynamic linking support is something that rust, go and C# could definitely improve upon. As I understand it, Rust does support dynamic linking for pure rust code. It just doesn't have a stable ABI yet. So dynamic linking is only supported within the same version of the rust compiler. Taking a look at pam, it looks like there's 2 parts: - The configuration parser & dynamic loader, which would be trivial to rewrite in rust or C# or something. - The actual dynamically loaded modules, and the ABI bridge to call into them. I think the obvious way to port pam to rust or C# would be to rewrite the core. Then just statically link in all the standard, built-in pam modules (like pam_unix). And - at least for now - handle "external" modules in exactly the same "yolo C" dyld way they are handled today. Yes, that means a dozen lines of boilerplate unsafe rust code across the C ABI bridge. It wouldn't be quite as safe as Fil-C or C# JITting for everything. But its within my personal safety paranoia tolerance. It would certainly be much safer than pam is today, given currently every single line is in yolo mode. It would also mean those modules wouldn't need to all be rewritten in rust all at once, or at all. If you want a fully safe replacement for the pam bridge maybe wasm+wasi could work here? I'm not sure. I certainly hear you that this is a problem in rust, go and c#, and I'm not entirely sure what a good solution would look like. Well, other than rust stabilizing its ABI. At the pace rust moves at, that could be any decade now. > I don't think JITing scales to every process on your system doing it (you get no sharing of code between processes, and if you consider that I've got >500 processes on the machine I'm using right now, you can imagine how much of a memory burden JIT-in-everything would be). Forget every program for a moment. C#'s JIT would certainly be fast enough for pam. And most linux programs don't make such heavy use of dynamic linking to work. So they could be AOT compiled. On the topic of performance, I'm not sure I want to pay the fil-C performance overhead for all 500+ processes on my system either. I assume the performance ceiling for fil-c is going to be somewhere around the performance of other well optimized GC runtimes. I don't know of any GC language that does better than about a 2x memory usage overhead. Call me crazy, but for general computing, I like the speed of native code.

pizlonator 11mo ago
> As I understand it, Rust does support dynamic linking for pure rust code. It just doesn't have a stable ABI yet. Not quite. To safely run a Rust program against a Rust dynamic library, you had to have compiled and linked that program against exactly that version of the Rust library. This is necessary for fundamental Rust type system reasons. It’s not simply a matter of stabilizing ABI. It’s not like updating the library without relinking would work if you used exactly the same rustc version - it would still be unsafe. So no, Rust doesn’t support dynamic linking in any meaningful way. > And most linux programs don't make such heavy use of dynamic linking to work. This is not true at all. Linux relies on dynamic linking extensively in two ways: - A dynamic library may be updated independently of its dependents. You can’t do this in Rust, see above. - Lots of Linux programs rely on the dynamically loaded module architecture like what pam does. All of the scripting language runtimes do this for example.
- gpm 11mo ago
  The ability to replace libraries without recompiling downstream dependents is not a necessary component of dynamic linking. However if rust wanted to gain this ability, it basically is just a matter of a stable ABI. The only other thing you would need is to get a hash of all the parts of the system that are equivalent to C header files and have the program check the hash in the dylib matched the version it was compiled against at load time. I believe in fact the sdylib crate type in nightly rust does (or at least is intended to do, not sure it's fully implemented) exactly this.
  - pizlonator 11mo ago
    > The ability to replace libraries without recompiling downstream dependents is not a necessary component of dynamic linking. Yes it is, in the sense that not having this is a massive bug. Dynamic linking becomes horrifically unsafe without this property, rendering it mostly useless.
    - gpm 11mo ago
      It's not a bug, it's just a lack of an additional feature on top of dynamic linking. In fact it prevents bugs as it provides a clear bar on the misuse of dynamic linking as is so common in C/C++/(as designed) Fil-C to replace libraries with likely-ABI-incompatible replacements. Fil-C limits the blast radius of bugs caused by this, but doesn't prevent the bugs. Just saying "you can't do that" like current rust actually prevents them. (Alternatively growing that feature properly, including checking ABI compatibility, as intended with sdylib's would also prevent them. To the best of my knowledge no C-like language does this today) Dynmaic linking used responsibly to reduce linking times in development cycles, and to reduce memory usage when you have memory things using the same library, remains perfectly safe without this feature. The former is reasonably common in the rust ecosystem today, and I've literally never hit or heard of anyone else hitting a safety issue as a result. Minimizing memory usage is a niche use case in the modern world... but it's there if you want it. You just have to not modify the application post compilation - just like you have to not modify any application post compilation to retain memory safety guarantees.

by josephg 11mo ago

pizlonator 11mo ago

> As I understand it, Rust does support dynamic linking for pure rust code. It just doesn't have a stable ABI yet. Not quite. To safely run a Rust program against a Rust dynamic library, you had to have compiled and linked that program against exactly that version of the Rust library. This is necessary for fundamental Rust type system reasons. It’s not simply a matter of stabilizing ABI. It’s not like updating the library without relinking would work if you used exactly the same rustc version - it would still be unsafe. So no, Rust doesn’t support dynamic linking in any meaningful way. > And most linux programs don't make such heavy use of dynamic linking to work. This is not true at all. Linux relies on dynamic linking extensively in two ways: - A dynamic library may be updated independently of its dependents. You can’t do this in Rust, see above. - Lots of Linux programs rely on the dynamically loaded module architecture like what pam does. All of the scripting language runtimes do this for example.

gpm 11mo ago
The ability to replace libraries without recompiling downstream dependents is not a necessary component of dynamic linking. However if rust wanted to gain this ability, it basically is just a matter of a stable ABI. The only other thing you would need is to get a hash of all the parts of the system that are equivalent to C header files and have the program check the hash in the dylib matched the version it was compiled against at load time. I believe in fact the sdylib crate type in nightly rust does (or at least is intended to do, not sure it's fully implemented) exactly this.
- pizlonator 11mo ago
  > The ability to replace libraries without recompiling downstream dependents is not a necessary component of dynamic linking. Yes it is, in the sense that not having this is a massive bug. Dynamic linking becomes horrifically unsafe without this property, rendering it mostly useless.
  - gpm 11mo ago
    It's not a bug, it's just a lack of an additional feature on top of dynamic linking. In fact it prevents bugs as it provides a clear bar on the misuse of dynamic linking as is so common in C/C++/(as designed) Fil-C to replace libraries with likely-ABI-incompatible replacements. Fil-C limits the blast radius of bugs caused by this, but doesn't prevent the bugs. Just saying "you can't do that" like current rust actually prevents them. (Alternatively growing that feature properly, including checking ABI compatibility, as intended with sdylib's would also prevent them. To the best of my knowledge no C-like language does this today) Dynmaic linking used responsibly to reduce linking times in development cycles, and to reduce memory usage when you have memory things using the same library, remains perfectly safe without this feature. The former is reasonably common in the rust ecosystem today, and I've literally never hit or heard of anyone else hitting a safety issue as a result. Minimizing memory usage is a niche use case in the modern world... but it's there if you want it. You just have to not modify the application post compilation - just like you have to not modify any application post compilation to retain memory safety guarantees.