6 ms·

I wonder why it is so hard for Rust to implement its own safe stdlib independent of C.

zanderwohl 2y ago
It would be a tremendous amount of work, and would take years. Meanwhile, the problems are avoidable. It's not exactly the "rust way" to just remember and avoid problems, but everything in language design is compromises.
- IshKebab 2y ago
  "Impossibru!!" https://github.com/sunfishcode/eyra https://github.com/sunfishcode/eyra Oh look: > Why use Eyra? It fixes Rust's set_var unsoundness issue. The environment-variable implementation leaks memory internally (it is optional, but enabled by default), so setenv etc. are thread-safe.
  - kbolino 2y ago
    That's quite a trade-off
    - IshKebab 2y ago
      What is? Leaking memory? It's going to be a few kB at absolute most. Not an issue unless you are doing something very weird.
    - mmastrac 2y ago
      I think glibc made the same trade-off. It makes sense for most types of programs, but there's certainly a lot of classes of programs that wouldn't take it.
  - sunshowers 2y ago
    That only works on Linux though right?
  - zanderwohl 2y ago
    > Why not use Eyra? Well, that's a lot of caveats. As I said, it would take years to complete. And it looks like it's well on its way but not near complete.
  - dvtkrlbs 2y ago
    If I understand it correctly this still doesnt help with downstream dependencies.
do_not_redeem 2y ago
The crash in the article happened when Python called C's getenv. Rust could very well throw away libc, but then it would also be throwing away its great C interop story. Rust can't force Python to use its own stdlib instead of libc.
dgrunwald 2y ago
How exactly would that help in this situation? If both Rust and C have independent standard libraries loaded into the same process, each would have an independent set of environment variables. So setting a variable from Rust wouldn't make it visible to the C code, which would break the article's usecase of configuring OpenSSL. The only real solution is to have the operating system provide a thread-safe way of managing environment variables. Windows does so; but in Linux that's the job of libc, which refuses to provide thread-safety.
- rcxdude 2y ago
  If there was a libc implemented in rust (like https://github.com/redox-os/relibc https://github.com/redox-os/relibc), you could use that for the C code in the process, and you'd be sharing the relevant state.
steveklabnik 2y ago
Linux is an unusual platform in that it allows you to call into it via assembly. Most other platforms require you to go through libc to do so. It's not really in Rust's hands.
- PaulDavisThe1st 2y ago
  This is not unusual at all. Windows allowed it for years before Linux came along. It was also true of some other *nix systems - IIRC, Ultrix (DEC) allowed this, and so did Dynix (Sequent). *BSD allows it too, or used as of 2022. What is unusual about Linux is that it guarantees a syscall ABI, meaning that if you follow it, you can make a system call "portably" across "any" version of Linux.
  - steveklabnik 2y ago
    Sure, I’m speaking about platforms that are relevant today, not historical ones. Windows, MacOS, {Free,Open,Net}BSD, Solaris, illumos, none of these do.
    - eqvinox 2y ago
      It's quite easy to find out the actual situation on this since Go decided to do it their way. Last I checked, OpenBSD is the only OS where they go through libc, but I haven't really kept up.
      - steveklabnik 2y ago
        In my understanding, Go initially disregarded various platforms' rules here, and have ended up walking it back. I could be wrong though. It's hard to find good details here, but here's a mailing list thread from 2019 mentioning libc usage: https://groups.google.com/g/golang-nuts/c/uX8eUeyuuAY/m/CfhlczbrAwAJ https://groups.google.com/g/golang-nuts/c/uX8eUeyuuAY/m/Cfhl... > On Solaris (and Windows), and more recently in macOS as well we link with libc (or equivalent). > Go used to do raw system calls on macOS, and binaries were occasionally broken by kernel updates. Now Go uses libc on macOS.
kbolino 2y ago
They did, it's called core. But it assumes no operating system at all, and environment variables require an operating system.
- nomel 2y ago
  > and environment variables require an operating system Is that true? It's just a process global string -> string map, that can be pre-loaded with values before the process starts, with a copy of the current state being passed to any sub-process. This could be trivially implemented with batch processing/supervisory programs.
  - panzi 2y ago
    Well, it's used by the OS when exec-ing a new process, but at least the Linux syscall for that takes the environment as an explicit parameter. So it could be managed in whatever way by the runtime until execve() is called.
  - kbolino 2y ago
    Sure, there's a broader concept here, which doesn't require any operating system. But any alternate string->string map you define won't answer to C code calling getenv, won't be passed to child processes created with fork, won't be visible through /proc/$PID/environ, etc.
    - nomel 2y ago
      This is the context: > They did, it's called core. But it assumes no operating system at all, and environment variables require an operating system. I think there's some confusion here. The C standard library is an abstraction layer that exists to implement standard behavior on hardware. It's entirely unrelated to the existence of an OS. Things like "/proc/$PID/environ" have nothing to do with C. There are many standard libraries, for embedded, that implement these things, like getenv, on bare metal [1]. Standard C libraries exist to implement functionality. It does not define how to implement the functionality. That's the whole point of C: it's an abstraction that has very little requirements. The implementation of environment variables don't require an OS. If they made this "core", they could trivially implement the concept. [1] https://en.wikipedia.org/wiki/Newlib https://en.wikipedia.org/wiki/Newlib [2] getenv: https://sourceware.org/newlib/libc.html https://sourceware.org/newlib/libc.html
v3xro 2y ago
There are rust libc implementations e.g. one by Redox: https://gitlab.redox-os.org/redox-os/relibc https://gitlab.redox-os.org/redox-os/relibc

It would be a tremendous amount of work, and would take years. Meanwhile, the problems are avoidable. It's not exactly the "rust way" to just remember and avoid problems, but everything in language design is compromises.

IshKebab 2y ago
"Impossibru!!" https://github.com/sunfishcode/eyra https://github.com/sunfishcode/eyra Oh look: > Why use Eyra? It fixes Rust's set_var unsoundness issue. The environment-variable implementation leaks memory internally (it is optional, but enabled by default), so setenv etc. are thread-safe.
- kbolino 2y ago
  That's quite a trade-off
  - IshKebab 2y ago
    What is? Leaking memory? It's going to be a few kB at absolute most. Not an issue unless you are doing something very weird.
  - mmastrac 2y ago
    I think glibc made the same trade-off. It makes sense for most types of programs, but there's certainly a lot of classes of programs that wouldn't take it.
- sunshowers 2y ago
  That only works on Linux though right?
- zanderwohl 2y ago
  > Why not use Eyra? Well, that's a lot of caveats. As I said, it would take years to complete. And it looks like it's well on its way but not near complete.
- dvtkrlbs 2y ago
  If I understand it correctly this still doesnt help with downstream dependencies.

do_not_redeem 2y ago

The crash in the article happened when Python called C's getenv. Rust could very well throw away libc, but then it would also be throwing away its great C interop story. Rust can't force Python to use its own stdlib instead of libc.

dgrunwald 2y ago

How exactly would that help in this situation? If both Rust and C have independent standard libraries loaded into the same process, each would have an independent set of environment variables. So setting a variable from Rust wouldn't make it visible to the C code, which would break the article's usecase of configuring OpenSSL. The only real solution is to have the operating system provide a thread-safe way of managing environment variables. Windows does so; but in Linux that's the job of libc, which refuses to provide thread-safety.

rcxdude 2y ago
If there was a libc implemented in rust (like https://github.com/redox-os/relibc https://github.com/redox-os/relibc), you could use that for the C code in the process, and you'd be sharing the relevant state.

steveklabnik 2y ago

Linux is an unusual platform in that it allows you to call into it via assembly. Most other platforms require you to go through libc to do so. It's not really in Rust's hands.

PaulDavisThe1st 2y ago
This is not unusual at all. Windows allowed it for years before Linux came along. It was also true of some other *nix systems - IIRC, Ultrix (DEC) allowed this, and so did Dynix (Sequent). *BSD allows it too, or used as of 2022. What is unusual about Linux is that it guarantees a syscall ABI, meaning that if you follow it, you can make a system call "portably" across "any" version of Linux.
- steveklabnik 2y ago
  Sure, I’m speaking about platforms that are relevant today, not historical ones. Windows, MacOS, {Free,Open,Net}BSD, Solaris, illumos, none of these do.
  - eqvinox 2y ago
    It's quite easy to find out the actual situation on this since Go decided to do it their way. Last I checked, OpenBSD is the only OS where they go through libc, but I haven't really kept up.
    - steveklabnik 2y ago
      In my understanding, Go initially disregarded various platforms' rules here, and have ended up walking it back. I could be wrong though. It's hard to find good details here, but here's a mailing list thread from 2019 mentioning libc usage: https://groups.google.com/g/golang-nuts/c/uX8eUeyuuAY/m/CfhlczbrAwAJ https://groups.google.com/g/golang-nuts/c/uX8eUeyuuAY/m/Cfhl... > On Solaris (and Windows), and more recently in macOS as well we link with libc (or equivalent). > Go used to do raw system calls on macOS, and binaries were occasionally broken by kernel updates. Now Go uses libc on macOS.

kbolino 2y ago

They did, it's called core. But it assumes no operating system at all, and environment variables require an operating system.

nomel 2y ago
> and environment variables require an operating system Is that true? It's just a process global string -> string map, that can be pre-loaded with values before the process starts, with a copy of the current state being passed to any sub-process. This could be trivially implemented with batch processing/supervisory programs.
- panzi 2y ago
  Well, it's used by the OS when exec-ing a new process, but at least the Linux syscall for that takes the environment as an explicit parameter. So it could be managed in whatever way by the runtime until execve() is called.
- kbolino 2y ago
  Sure, there's a broader concept here, which doesn't require any operating system. But any alternate string->string map you define won't answer to C code calling getenv, won't be passed to child processes created with fork, won't be visible through /proc/$PID/environ, etc.
  - nomel 2y ago
    This is the context: > They did, it's called core. But it assumes no operating system at all, and environment variables require an operating system. I think there's some confusion here. The C standard library is an abstraction layer that exists to implement standard behavior on hardware. It's entirely unrelated to the existence of an OS. Things like "/proc/$PID/environ" have nothing to do with C. There are many standard libraries, for embedded, that implement these things, like getenv, on bare metal [1]. Standard C libraries exist to implement functionality. It does not define how to implement the functionality. That's the whole point of C: it's an abstraction that has very little requirements. The implementation of environment variables don't require an OS. If they made this "core", they could trivially implement the concept. [1] https://en.wikipedia.org/wiki/Newlib https://en.wikipedia.org/wiki/Newlib [2] getenv: https://sourceware.org/newlib/libc.html https://sourceware.org/newlib/libc.html

v3xro 2y ago

There are rust libc implementations e.g. one by Redox: https://gitlab.redox-os.org/redox-os/relibc https://gitlab.redox-os.org/redox-os/relibc