5 ms·

A potential use case I see is for security auditing. Even if you cannot port an existing C codebase to Rust, you could run this tool to examine the unsafe hotsp

A potential use case I see is for security auditing. Even if you cannot port an existing C codebase to Rust, you could run this tool to examine the unsafe hotspots. Any place where the translation has to rely upon unsafe is a region of the code more likely to contain any of the mistakes Rust is designed to prevent. Of course, this pre-supposes that 90% of the translation does not have to lean on unsafe annotation.

kazinator 4y ago
I suspect that this relies on unsafe pretty much everywhere. Even handling argc and argv in your C main function, in idiomatic ways, is unsafe. There is no 80/20 rule for C unsafety, other than maybe an inverted one: 80% of the unsafety of a large C program might be spread into 80% or more of the code. :)
- counttheforks 4y ago
  > Even handling argc and argv in your C main function, in idiomatic ways, is unsafe. Not knowing C very well, could you clarify what makes it unsafe? Thanks!
  - initplus 4y ago
    C arrays are just sugar for pointer arithmetic. [] just calculates the sum and deferences the result. arr[n] == *(arr + n) == n[arr] All these forms are valid C and gcc will happily compile them all without complaining.
  - tinco 4y ago
    The word unsafe has a specific meaning in Rust. It doesn't mean every C program that uses argc and argv is unsafe. In this specific case however I don't think it would actually require much unsafe. The only unsafe thing I'd introduce is a way of casting the *argv[] to a type that safely deals with null terminated strings. Maybe such a type is already in Rust's standard library and I wouldn't even need that. edit: eh sorry I wasn't thinking straight, you of course need unsafe to cast the argv itself to a type that has a seperate argc as well. Assuming such a type is available, if it's not its implementation would also have unsafe all over the place. Maybe to answer the underlying question. What makes it unsafe is that in C it is assumed the programmer knows to keep all indexes into argv under argc. In Rust such an assumption must be made explicit by specifying "unsafe". It is idiomatic Rust to have all instances of "unsafe" in libraries whose implementation is vetted by the community, so ideally there are little to no instances of "unsafe" in the application logic itself. Rust's compiler and type system have various tricks that reduce the amount of "unsafe" you would think you'd need for even quite complex problems.
    - kazinator 4y ago
      Not every C program that processes argv gets it wrong. They are all unsafe. Unsafe != wrong.
  - insanitybit 4y ago
    You'd have to access the stack frame above `main` and then treat some of the bytes within that frame as your env. This means forging pointers/bounds based on the inputs. `execve` basically sets that up for you but Rust doesn't know about that. Then if you wanted to handle dynamically set environment variables you'll need to call into your libc implementation, which crosses an ffi boundary, which means rust doesn't know about what that code is doing and therefor it requires `unsafe`. edit: Question for others - is main a separate stackframe? I actually don't recall.
  - hn92726819 4y ago
    "safe" and "unsafe" in rust are well defined, but in my opinion it's very confusing and I wish they used different terms. In rust, unsafe means accessing memory that's already freed or unallocated or things like that. You can look up the definition for the full definition. I think the comment you replied to mistakenly used the term "unsafe" (that's part of the reason I dislike the term; it can mean multiple things). In rust context though, it isn't unsafe to index an array that's out of bounds. I.e. if argc=10 and you call argv[99], that will crash your program but isn't considered "unsafe".
  - pjmlp 4y ago
    Paraphrasing a common meme, how much time do you have? Just scratching the surface, we have: - The language doesn't really have vector and strings as data types, they are pointers to memory sections without any kind of protection - All functions on the standard library deemed as safe, added as mitgation to fix possible memory corruptions have gotchas on their use, there isn't a single one that is safe, specially because all of them expect the developer to never get the buffer size parameters wrong. - Enumerations are not type safe, decay implicitly to integers when used in numeric context, and all numeric values can be converted into an enumeration, even if there isn't a mapping available - Implicit numeric convertions everywhere, and since there is no overflow/underflow checking, every single numeric operation can wrap around, or be the source for clever compiler optimizations - ISO C documents at least around 200 cases of UB, where the compiler can take the liberty to optimize the code as it pleases - Type casts that convert complex data types into others can be a source of surprises when moving across compilers and platforms - Speaking of which, even if you restrict yourself to ISO C, without any compiler specific extensions, there are behaviours that are implementation defined, which can vary across compilers and platforms. - Variables defined as const, aren't really constant and one can subvert their value - There is no null checking, so whatever happens depends on the platform. This is just a short overview, open the man page for GCC or clang and go through the list of all warnings that you can enable to try to write safer code, specially all that are enabled via -Wall and -Wextra. All the above flaws are also present in Objective-C, Objective-C++ and C++, due to their copy-paste compatibility with C (yes C++ isn't 100% compatible).
    - int_19h 4y ago
      IIRC, in C++ at least, mutating an object that is originally const (whether it's a variable declared as such, or a heap object created with "new const ..."), is UB regardless of how you do it - pointer casts etc.
      - pjmlp 4y ago
        Mutating an object that is originally const is anyway UB regardless of the programming language. Compilers have the freedom to place them into readonly memory segments, either in the case of static data, in the heap (mark the pages as such for safety), in embedded static const data can even be mapped into ROM chips, so who knows what happens if the compiler has decided to implement const data that way, or what the linker scripts are doing.
  - kazinator 4y ago
    For instance it means that an expression like argv[i], even though correct, could be wrong in a way that won't be diagnosed. Code us "unsafe" to the extent that is predictable behavior depends only on the programmer.
- moomin 4y ago
  You’re right, but they’re hoping to improve upon that.
- a1369209993 4y ago
  > There is no 80/20 rule for C unsafety, Actually, there is; the problem is that the transpiler can't tell the difference between code that relies on unsafety for its semantics, versus code that would still work if appropiate annotations (potentially causing function to not be callable in intended contexts), run-time checks (possibly causing code to error out on what were intended to be valid inputs), etc, were added to make it safe. 80% of the code contains 20% of the cases where safety would require deviating from the intended semantics, not just the incidental ones. (A general-purpose transpiler can't (in general) tell the difference between intended semantics and incidental ones, so it has to conservatively assume all semantics are intended, and write everything as the most-general (ie most-unsafe) interpretation.)
  - kazinator 4y ago
    Any C code that performs a calculation which would silently be wrong or crash if the values were not correct (even though they are) is inherently unsafe.
    - turminal 4y ago
      That's a weird way to put it. If your function assumes some constraints on the input it gets and you give it data that violates its constraints, it's going to fail in some way. Sure, C makes it worse by making it harder to verify the assumptions and constraints, but by your definition every function that operates on sorted arrays and doesn't verify the input is sorted is inherently unsafe, regardless of the language.
      - jsjohnst 4y ago
        > but by your definition every function that operates on sorted arrays and doesn't verify the input is sorted is inherently unsafe Are you claiming it’s not? verify all input should be the baseline to call something safe imho.
        turminal 4y ago
electroly 4y ago
This only produces unsafe code. Every translated function has the unsafe keyword. It's up to the programmer to clean it up afterwards.
WalterBright 4y ago
Since 90% (a wild guesstimate) of C code is pointers, I suspect this is hopeless. I've translated a lot of C code to D, and manually converting `*` to `ref` (D's safe pointers), and converting to slices, cleans up most of the C code nicely and you get buffer overflow checks for free.

kazinator 4y ago

I suspect that this relies on unsafe pretty much everywhere. Even handling argc and argv in your C main function, in idiomatic ways, is unsafe. There is no 80/20 rule for C unsafety, other than maybe an inverted one: 80% of the unsafety of a large C program might be spread into 80% or more of the code. :)

counttheforks 4y ago
> Even handling argc and argv in your C main function, in idiomatic ways, is unsafe. Not knowing C very well, could you clarify what makes it unsafe? Thanks!
- initplus 4y ago
  C arrays are just sugar for pointer arithmetic. [] just calculates the sum and deferences the result. arr[n] == *(arr + n) == n[arr] All these forms are valid C and gcc will happily compile them all without complaining.
- tinco 4y ago
  The word unsafe has a specific meaning in Rust. It doesn't mean every C program that uses argc and argv is unsafe. In this specific case however I don't think it would actually require much unsafe. The only unsafe thing I'd introduce is a way of casting the *argv[] to a type that safely deals with null terminated strings. Maybe such a type is already in Rust's standard library and I wouldn't even need that. edit: eh sorry I wasn't thinking straight, you of course need unsafe to cast the argv itself to a type that has a seperate argc as well. Assuming such a type is available, if it's not its implementation would also have unsafe all over the place. Maybe to answer the underlying question. What makes it unsafe is that in C it is assumed the programmer knows to keep all indexes into argv under argc. In Rust such an assumption must be made explicit by specifying "unsafe". It is idiomatic Rust to have all instances of "unsafe" in libraries whose implementation is vetted by the community, so ideally there are little to no instances of "unsafe" in the application logic itself. Rust's compiler and type system have various tricks that reduce the amount of "unsafe" you would think you'd need for even quite complex problems.
  - kazinator 4y ago
    Not every C program that processes argv gets it wrong. They are all unsafe. Unsafe != wrong.
- insanitybit 4y ago
  You'd have to access the stack frame above `main` and then treat some of the bytes within that frame as your env. This means forging pointers/bounds based on the inputs. `execve` basically sets that up for you but Rust doesn't know about that. Then if you wanted to handle dynamically set environment variables you'll need to call into your libc implementation, which crosses an ffi boundary, which means rust doesn't know about what that code is doing and therefor it requires `unsafe`. edit: Question for others - is main a separate stackframe? I actually don't recall.
- hn92726819 4y ago
  "safe" and "unsafe" in rust are well defined, but in my opinion it's very confusing and I wish they used different terms. In rust, unsafe means accessing memory that's already freed or unallocated or things like that. You can look up the definition for the full definition. I think the comment you replied to mistakenly used the term "unsafe" (that's part of the reason I dislike the term; it can mean multiple things). In rust context though, it isn't unsafe to index an array that's out of bounds. I.e. if argc=10 and you call argv[99], that will crash your program but isn't considered "unsafe".
- pjmlp 4y ago
  Paraphrasing a common meme, how much time do you have? Just scratching the surface, we have: - The language doesn't really have vector and strings as data types, they are pointers to memory sections without any kind of protection - All functions on the standard library deemed as safe, added as mitgation to fix possible memory corruptions have gotchas on their use, there isn't a single one that is safe, specially because all of them expect the developer to never get the buffer size parameters wrong. - Enumerations are not type safe, decay implicitly to integers when used in numeric context, and all numeric values can be converted into an enumeration, even if there isn't a mapping available - Implicit numeric convertions everywhere, and since there is no overflow/underflow checking, every single numeric operation can wrap around, or be the source for clever compiler optimizations - ISO C documents at least around 200 cases of UB, where the compiler can take the liberty to optimize the code as it pleases - Type casts that convert complex data types into others can be a source of surprises when moving across compilers and platforms - Speaking of which, even if you restrict yourself to ISO C, without any compiler specific extensions, there are behaviours that are implementation defined, which can vary across compilers and platforms. - Variables defined as const, aren't really constant and one can subvert their value - There is no null checking, so whatever happens depends on the platform. This is just a short overview, open the man page for GCC or clang and go through the list of all warnings that you can enable to try to write safer code, specially all that are enabled via -Wall and -Wextra. All the above flaws are also present in Objective-C, Objective-C++ and C++, due to their copy-paste compatibility with C (yes C++ isn't 100% compatible).
  - int_19h 4y ago
    IIRC, in C++ at least, mutating an object that is originally const (whether it's a variable declared as such, or a heap object created with "new const ..."), is UB regardless of how you do it - pointer casts etc.
    - pjmlp 4y ago
      Mutating an object that is originally const is anyway UB regardless of the programming language. Compilers have the freedom to place them into readonly memory segments, either in the case of static data, in the heap (mark the pages as such for safety), in embedded static const data can even be mapped into ROM chips, so who knows what happens if the compiler has decided to implement const data that way, or what the linker scripts are doing.
- kazinator 4y ago
  For instance it means that an expression like argv[i], even though correct, could be wrong in a way that won't be diagnosed. Code us "unsafe" to the extent that is predictable behavior depends only on the programmer.
moomin 4y ago
You’re right, but they’re hoping to improve upon that.
a1369209993 4y ago
> There is no 80/20 rule for C unsafety, Actually, there is; the problem is that the transpiler can't tell the difference between code that relies on unsafety for its semantics, versus code that would still work if appropiate annotations (potentially causing function to not be callable in intended contexts), run-time checks (possibly causing code to error out on what were intended to be valid inputs), etc, were added to make it safe. 80% of the code contains 20% of the cases where safety would require deviating from the intended semantics, not just the incidental ones. (A general-purpose transpiler can't (in general) tell the difference between intended semantics and incidental ones, so it has to conservatively assume all semantics are intended, and write everything as the most-general (ie most-unsafe) interpretation.)
- kazinator 4y ago
  Any C code that performs a calculation which would silently be wrong or crash if the values were not correct (even though they are) is inherently unsafe.
  - turminal 4y ago
    That's a weird way to put it. If your function assumes some constraints on the input it gets and you give it data that violates its constraints, it's going to fail in some way. Sure, C makes it worse by making it harder to verify the assumptions and constraints, but by your definition every function that operates on sorted arrays and doesn't verify the input is sorted is inherently unsafe, regardless of the language.
    - jsjohnst 4y ago
      > but by your definition every function that operates on sorted arrays and doesn't verify the input is sorted is inherently unsafe Are you claiming it’s not? verify all input should be the baseline to call something safe imho.
      - turminal 4y ago

electroly 4y ago

This only produces unsafe code. Every translated function has the unsafe keyword. It's up to the programmer to clean it up afterwards.

WalterBright 4y ago

Since 90% (a wild guesstimate) of C code is pointers, I suspect this is hopeless. I've translated a lot of C code to D, and manually converting `*` to `ref` (D's safe pointers), and converting to slices, cleans up most of the C code nicely and you get buffer overflow checks for free.