7 ms·

I'm not trying to be rude, but I'm sorry... Are you seriously suggesting that rewriting an implementation of the C standard library in a language that isn't C

I'm not trying to be rude, but I'm sorry... Are you seriously suggesting that rewriting an implementation of the C standard library in a language that isn't C is something that makes any bit of sense? I think you have a fundemental understanding of the roll the C-standard library plays in the C language. Not to mention UNIX in general. Maybe you're suggesting we need an OS based in Rust instead of C (which is more reasonable), but in that case why wouldn't you use Rust's standard library as the backbone of that system? Let me put it this way, if the GNU/Linux C standard library gets rewritten in Rust or Idris (sigh... seriously?) I will switch to Windows 10 with default settings for the rest of my life. I would rather have the NSA log my every key stroke than participate in a community where that is considered a good idea.

arielb1 11y ago
Rust can have the same ABI as C - rewriting NSS in Rust should not pose ABI problems.
- autoreleasepool 11y ago
  It's still completely pointless. How would mmap be any different if it was implemented in Rust rather than in C? You need unsafe to implement it! The safety mechanisms are practically useless at such a low level. Wrap existing C with safe Rust, not the other way around! We don't rewrite unsafe C code in Unsafe Rust so that it can be called from all the unsafe C code in UNIX. Such a translation is so pointless and error prone. I can't even believe I'm the one being called backwards here.
  - dbaupp 11y ago
    > How would mmap be any different if it was implemented in Rust rather than in C? You need unsafe to implement it! The safety mechanisms are practically useless at such a low level. They're definitely not. The power of Rust is not avoiding all `unsafe` ever, but wrapping that unsafe into finite-scope, safe wrappers. Something like mmap would presumably interface with the OS's internal memory management routines, which would definitely have to have `unsafe` somewhere since they're hitting the hardware, but it seems very reasonable for the amount of unsafe exposed to much reduced (possibly even to zero). See, for instance, http://os.phil-opp.com/modifying-page-tables.html http://os.phil-opp.com/modifying-page-tables.html . Basically, as soon as you're above the absolute raw hardware level, you can start introducing extra help to reduce unsafety/catch bugs. In any case, code inside `unsafe` still benefits from all the conventional Rust checks, e.g. iterating over a slice won't go out of bounds due to a typo in the loop even inside `unsafe`, nor will references accidentally become dangling. (Of course, the `unsafe` block may do something explicitly that causes either of these problems, but this exact same risk is pervasive in all C code, rather than just around explicitly marked areas.) > Wrap existing C with safe Rust, not the other way around! We don't rewrite unsafe C code in Unsafe Rust so that it can be called from all the unsafe C code in UNIX. Such a translation is so pointless and error prone. Note that this is exactly what is done now, the features of Rust for making safe interfaces makes Rust often a far nicer way to use C interfaces than C itself, without overhead (IMO, of course). See http://blog.rust-lang.org/2015/04/24/Rust-Once-Run-Everywhere.html#safe-abstractions http://blog.rust-lang.org/2015/04/24/Rust-Once-Run-Everywher... , for example. Of course, zero-overhead Rust is only as safe as the C code it wraps, it can't (in general) protect against the C code not correctly implementing its stated contract(s).
    - tomjakubowski 11y ago
      It seems hard (impossible?) to write a safe, full-featured mmap wrapper in Rust, because of the potential for multiple processes sharing mutable access to the memory and defeating Rust's aliasing checks. Maybe you could separately map shared RefCells to guard the pages returned by the mmap wrapper? If the region is backed by a file, you'd need to check those RefCells on every IO access too, though.
      - eddyb 11y ago
        What you can do for mmap is have several abstractions (or one using generics and phantom types), one for each different set of usecases, with different access modes. Examples would be: * read-only: &ROMemMap -> &[u8] * read-write: &mut RWMemMap -> &mut [u8] * write-only: fn set(self: &mut WOMemMap, i: usize, b: u8), or more generally: &mut WOMemMap -> &mut [WriteOnly<u8>] where WriteOnly<T> has fn set(&mut self, T) Now for the shared case, consider this: aliasing rules can be avoided with atomic operations, i.e. Arc<AtomicUsize> is shared and can be safely (atomically) read/written by multiple threads. In the multi-process case, you could provide an atomic API, although we don't currently seem to expose byte-level atomics (likely not present on some platforms) so if you wanted to write a demo you'd need to use the unstable intrinsics atm. FWIW restricted to single-threaded code, this results in the Cell get/set API which is not hardware-atomic but cannot overlap with other accesses to the same memory, as Cell doesn't implement Share so any threading abstraction will block you from doing any kind of sharing of Cells. That is, you can share &[Cell<u8>] pointing to any bag of bytes that sits in read-write memory, and anything in the same thread can read or write to it, safely.
      - dbaupp 11y ago
        I was more focusing on implementing the internals of mmap, since that's what I assumed the question was. For that task, the actual memory/pointer returned is essentially a black-box (i.e. it's never read nor written by mmap itself), mmap is just interacting with the operating system. However, as you say, exposing a safe mmap version for libraries to call is a different problem with different difficulties. eddyb's sibling suggestion of teasing out the multiple entangled uses of mmap sounds like a good one, although someone would have to experiment with it to be sure.
  - umanwizard 11y ago
    To add to what others have said: mmap is a system call; it's not something provided by libc.
- arielb1 11y ago
  You need unsafe to implement the C API part of a DNS resolver, but the API can be given a safe wrapper without much effort. Then parsing and buffer management bugs would occur only in safe code and only affect it.
umanwizard 11y ago
Your post is just a bunch of pathos and doesn't explain why you actually think it's a bad idea. There's no technical reason why the C standard library can't be written in a language other than C. Lots of language libraries are written in languages other than the one they're primarily used with and work fine.
- autoreleasepool 11y ago
  I mean, what problem is this solving? There's no technical reason why it can't be done, true. It's still a hilariously bad idea that would never succeed. That's aside from the fact that's a completely pointless wheel-reinventing endeavor. Rewrites in general are pretty difficult to justify. Sorry, I guess my aggressively pragmatic side is showing today. The idea is really dumb. To be honest, "Let's rewrite the C standard library in Rust" is one of the worst pieces of bikeshedding drivel I've ever read. And I like Rust 100 times more than I like C.
  - zebra1287 11y ago
    It will be finished together with GNU HURD.
  - theseoafs 11y ago
    The problem that it's solving is security vulnerabilities like this one.
pjmlp 11y ago
Are you are that Visual Studio C library is written in C++ with extern "C" and for Microsoft C is legacy, with .NET Native and C++ with compiler static analysis being the way forward?
- autoreleasepool 11y ago
  IIRC, that's because it's a C++ library, not a real C library. Aside from a few exceptions, it only includes the parts of the C standard that are required by the C++ standard. Hence the dismal C99 support on that platform.
  - pjmlp 11y ago
    As I mentioned, C is legacy for Microsoft. The way forward for C on Windows from Microsoft own SDK is to use the clang frontend + Visual C++ backend, or another C compiler from someone else.
    - autoreleasepool 11y ago
      I definitely support that initiative. The more C++ programmers we can get to stop writing C the better.
      - pjmlp 11y ago
        Me too. Yes, C++ is as unsafe as C, given its copy-paste compatibility, but at least it provides the tools to program safely (if one chooses to) and the C++ community cares much more about safety than the C one.

arielb1 11y ago

Rust can have the same ABI as C - rewriting NSS in Rust should not pose ABI problems.

autoreleasepool 11y ago
It's still completely pointless. How would mmap be any different if it was implemented in Rust rather than in C? You need unsafe to implement it! The safety mechanisms are practically useless at such a low level. Wrap existing C with safe Rust, not the other way around! We don't rewrite unsafe C code in Unsafe Rust so that it can be called from all the unsafe C code in UNIX. Such a translation is so pointless and error prone. I can't even believe I'm the one being called backwards here.
- dbaupp 11y ago
  > How would mmap be any different if it was implemented in Rust rather than in C? You need unsafe to implement it! The safety mechanisms are practically useless at such a low level. They're definitely not. The power of Rust is not avoiding all `unsafe` ever, but wrapping that unsafe into finite-scope, safe wrappers. Something like mmap would presumably interface with the OS's internal memory management routines, which would definitely have to have `unsafe` somewhere since they're hitting the hardware, but it seems very reasonable for the amount of unsafe exposed to much reduced (possibly even to zero). See, for instance, http://os.phil-opp.com/modifying-page-tables.html http://os.phil-opp.com/modifying-page-tables.html . Basically, as soon as you're above the absolute raw hardware level, you can start introducing extra help to reduce unsafety/catch bugs. In any case, code inside `unsafe` still benefits from all the conventional Rust checks, e.g. iterating over a slice won't go out of bounds due to a typo in the loop even inside `unsafe`, nor will references accidentally become dangling. (Of course, the `unsafe` block may do something explicitly that causes either of these problems, but this exact same risk is pervasive in all C code, rather than just around explicitly marked areas.) > Wrap existing C with safe Rust, not the other way around! We don't rewrite unsafe C code in Unsafe Rust so that it can be called from all the unsafe C code in UNIX. Such a translation is so pointless and error prone. Note that this is exactly what is done now, the features of Rust for making safe interfaces makes Rust often a far nicer way to use C interfaces than C itself, without overhead (IMO, of course). See http://blog.rust-lang.org/2015/04/24/Rust-Once-Run-Everywhere.html#safe-abstractions http://blog.rust-lang.org/2015/04/24/Rust-Once-Run-Everywher... , for example. Of course, zero-overhead Rust is only as safe as the C code it wraps, it can't (in general) protect against the C code not correctly implementing its stated contract(s).
  - tomjakubowski 11y ago
    It seems hard (impossible?) to write a safe, full-featured mmap wrapper in Rust, because of the potential for multiple processes sharing mutable access to the memory and defeating Rust's aliasing checks. Maybe you could separately map shared RefCells to guard the pages returned by the mmap wrapper? If the region is backed by a file, you'd need to check those RefCells on every IO access too, though.
    - eddyb 11y ago
      What you can do for mmap is have several abstractions (or one using generics and phantom types), one for each different set of usecases, with different access modes. Examples would be: * read-only: &ROMemMap -> &[u8] * read-write: &mut RWMemMap -> &mut [u8] * write-only: fn set(self: &mut WOMemMap, i: usize, b: u8), or more generally: &mut WOMemMap -> &mut [WriteOnly<u8>] where WriteOnly<T> has fn set(&mut self, T) Now for the shared case, consider this: aliasing rules can be avoided with atomic operations, i.e. Arc<AtomicUsize> is shared and can be safely (atomically) read/written by multiple threads. In the multi-process case, you could provide an atomic API, although we don't currently seem to expose byte-level atomics (likely not present on some platforms) so if you wanted to write a demo you'd need to use the unstable intrinsics atm. FWIW restricted to single-threaded code, this results in the Cell get/set API which is not hardware-atomic but cannot overlap with other accesses to the same memory, as Cell doesn't implement Share so any threading abstraction will block you from doing any kind of sharing of Cells. That is, you can share &[Cell<u8>] pointing to any bag of bytes that sits in read-write memory, and anything in the same thread can read or write to it, safely.
    - dbaupp 11y ago
      I was more focusing on implementing the internals of mmap, since that's what I assumed the question was. For that task, the actual memory/pointer returned is essentially a black-box (i.e. it's never read nor written by mmap itself), mmap is just interacting with the operating system. However, as you say, exposing a safe mmap version for libraries to call is a different problem with different difficulties. eddyb's sibling suggestion of teasing out the multiple entangled uses of mmap sounds like a good one, although someone would have to experiment with it to be sure.
- umanwizard 11y ago
  To add to what others have said: mmap is a system call; it's not something provided by libc.
arielb1 11y ago
You need unsafe to implement the C API part of a DNS resolver, but the API can be given a safe wrapper without much effort. Then parsing and buffer management bugs would occur only in safe code and only affect it.

umanwizard 11y ago

Your post is just a bunch of pathos and doesn't explain why you actually think it's a bad idea. There's no technical reason why the C standard library can't be written in a language other than C. Lots of language libraries are written in languages other than the one they're primarily used with and work fine.

autoreleasepool 11y ago
I mean, what problem is this solving? There's no technical reason why it can't be done, true. It's still a hilariously bad idea that would never succeed. That's aside from the fact that's a completely pointless wheel-reinventing endeavor. Rewrites in general are pretty difficult to justify. Sorry, I guess my aggressively pragmatic side is showing today. The idea is really dumb. To be honest, "Let's rewrite the C standard library in Rust" is one of the worst pieces of bikeshedding drivel I've ever read. And I like Rust 100 times more than I like C.
- zebra1287 11y ago
  It will be finished together with GNU HURD.
- theseoafs 11y ago
  The problem that it's solving is security vulnerabilities like this one.

pjmlp 11y ago

Are you are that Visual Studio C library is written in C++ with extern "C" and for Microsoft C is legacy, with .NET Native and C++ with compiler static analysis being the way forward?

autoreleasepool 11y ago
IIRC, that's because it's a C++ library, not a real C library. Aside from a few exceptions, it only includes the parts of the C standard that are required by the C++ standard. Hence the dismal C99 support on that platform.
- pjmlp 11y ago
  As I mentioned, C is legacy for Microsoft. The way forward for C on Windows from Microsoft own SDK is to use the clang frontend + Visual C++ backend, or another C compiler from someone else.
  - autoreleasepool 11y ago
    I definitely support that initiative. The more C++ programmers we can get to stop writing C the better.
    - pjmlp 11y ago
      Me too. Yes, C++ is as unsafe as C, given its copy-paste compatibility, but at least it provides the tools to program safely (if one chooses to) and the C++ community cares much more about safety than the C one.