9 ms·

Because your 'safe' implementation will certainly have a performance cost, and won't be the default. This is why, despite C++ providing std::array, you'll still

Because your 'safe' implementation will certainly have a performance cost, and won't be the default. This is why, despite C++ providing std::array, you'll still find buffer overflows in C++ code. C++'s std::array provides the safe 'at' function but you're opting into a performance penalty and it's not the more familiar [] syntax. Rust arrays/ vectors are safe-by-default. To use the unchecked, unsafe version requires using the 'unsafe' keyword. let v = vec![0, 1, 2]; unsafe { let x = v.get_unchecked(5); } This means you can basically grep audit for vulnerabilities, and the above code should be very rare.

sidlls 10y ago
How do you suppose runtime bounds checks are done in Rust? They certainly also incur a performance penalty in not-trivial cases. Also, "safe by grep audit" means "safe according to a human." The argument of course is that it lowers the surface area of what a human must be trusted to verify. I'm still not convinced by that argument, because human error is a thing. And for actual systems programming, "very rare" may not be true.
- staticassertion 10y ago
  > How do you suppose runtime bounds checks are done in Rust? They certainly also incur a performance penalty in not-trivial cases. Certainly. I didn't intend to imply otherwise. > Also, "safe by grep audit" means "safe according to a human." Again, totally correct here. > The argument of course is that it lowers the surface area of what a human must be trusted to verify. I'm still not convinced by that argument, because human error is a thing. And for actual systems programming, "very rare" may not be true. Well, given a codebase where both safe and unsafe code exists, the amount of unsafe code is strictly less than the amount of both safe and unsafe code. So it does reduce the amount of code needed to audit, even in a very atypical case where a ton of the code is unsafe. It's true that a project may use egregious amounts of unsafe. That would be unfortunate. Rust is still safer than C in that case, since it just defines more behavior (like arithmetic overflow), but I certainly wouldn't pretend that the rust code should be trusted. When writing rust one should certainly strive to write less unsafe code, and to always document the invariants required for unsafe code to be safe. Rust is not 100% safe 100% of the time, I'm only arguing that safe defaults are critical, and that grep auditing is a powerful tool.
  - EugeneOZ 10y ago
    Rust is safe in terms of memory usage and race conditions, nobody claims that Rust compiler will catch all 100% of bugs human can invent.
    - sidlls 10y ago
      Rust is safe by default in terms of memory usage. It is not, however, strictly memory safe. It is trivial to overflow a buffer in Rust, for example. I haven't discovered a trivial way to hide it, though.
      - steveklabnik 10y ago
        If Rust is not memory safe in safe code, then you've found a bug. Please report it to https://www.rust-lang.org/security.html https://www.rust-lang.org/security.html
        sidlls 10y ago
        Let me clarify: in Rust it is trivial to use memory unsafely. It is not, so far as I have found, trivial to hide that fact because it is required to use "unsafe" syntax decoration" to do so.
        Manishearth 10y ago
        The issue here is that that definition of "safe" language basically excludes all practical languages, including languages like Python, because FFI is possible. In general when talking about safety in a language it's about the level of explicitness required to trigger unsafety. I like the distinction made in the nomicon (https://doc.rust-lang.org/stable/nomicon/meet-safe-and-unsafe.html https://doc.rust-lang.org/stable/nomicon/meet-safe-and-unsaf...) -- Rust comprises of two distinct languages. You have everyday Rust, which is completely memory safe, and "unsafe Rust", which looks similar to everyday Rust but is not safe. `unsafe {}` blocks are your FFI between the two. Looking at unsafe blocks as FFI is IMO a very useful mental model especially for understanding the changes to invariants involved.
- rtpg 10y ago
  "Actual systems programming" mostly does not involve unsafe code. For example, most OS code is _not_ interrupt hooks or malloc but the rest of the OS. Most of Postgres is not reading data quickly, but higher-level abstractions. Large-scale systems programming will always be mostly higher-level abstractions, because that's the only way to write large programs. Name any "systems programming" OSS project, choose a random C file, and you'll see that most code does not require pointer arithmetic except because that's how you do things in C. You don't bit-twiddle for 200k lines of code, so being able to limit dangerous stuff like that to the 5k lines that actually need it makes work an order of magnitude easier.
  - anarazel 10y ago
    > "Actual systems programming" mostly does not involve unsafe code. True. Or slightly rephrased: Most of the unsafe code is centralized to some core pieces. From the POV of a PG dev: The big problem using something like rust for something like PostgreSQL is its it's portability, stability and uncertainty about where things are going. We do five years of back-branch releases (and for many that's not even enough!). Language and tooling around the new crop of languages simply aren't mature enough for that yet.
jdmichal 10y ago
In addition, the Rust compiler can also remove the built-in indexing checks if it can prove the code is safe. So, say, iterator loops over an array won't have any index checking.
- ori_b 10y ago
  So can the C compiler. The optimization is called 'value range propagation'.
- dbaupp 10y ago
  As others have said, this is a fairly standard optimisation: compilers (Rust, C, C++, Java, PHP, whatever) will remove branches if they can see that it can never be taken. Iterators are slightly different in that they don't have any indexing checks at all: an array iterator yields a plain reference, and these are always dereferenced like plain pointers.
- staticassertion 10y ago
  C/ C++ compilers are also capable of doing this, though rust's iterator syntax tends to make it a pretty natural optimization.
  - Animats 10y ago
    Neither C nor C++ knows, at the language level, the size of an array, unless that size is fixed. The subscript checking variants of C and C++ have to use "fat pointers" which carry along size information. The overhead for this is large and nobody uses that. Fat pointers used to be a feature you could turn on in gcc, but it's somewhat abandoned now.
    - sirclueless 10y ago
      > Neither C nor C++ knows, at the language level, the size of an array, unless that size is fixed. Neither does Rust. > The subscript checking variants of C and C++ have to use "fat pointers" which carry along size information. So do Rust's Slices. > The overhead for this is large and nobody uses that. People use std::vector all the time for this purpose in C++. It has about the performance you'd expect, with very little overhead except where you want it in bounds-checking. I don't think there's actually a performance difference here. Rust's default is safer because it requires dropping to unsafe code to do something dangerous, but the same optimizations are available in both.
lacampbell 10y ago
Would you care to provide a short rust implementation of reading an arbitrary length string from standard input? I have rust installed. I would be curious to benchmark it and see if is indeed faster than the same thing in C, using a user-defined bounds checked array.
- _sh 10y ago
  It's not so much a question of benchmarks, it's that one of the standard C tools for reading an arbitrary string from standard input is gets(). And if you reach for that from the standard toolbox, you've failed before you've started.
  - lacampbell 10y ago
    Many rust advocates talk about the performance cost of doing things safely in C, and inform me that rust has "Zero-cost abstractions". So it's fairly natural I ask for something I can benchmark.
    - tjr 10y ago
      Having used C (on and off) for about twenty years, including writing aerospace software in C, I strongly agree that writing safe C is possible. Rust (which I have thus far only tinkered with) looks pretty appealing to me, though. Do you suggest using C instead of Rust? If so, why?
      - lacampbell 10y ago
        Rust (which I have thus far only tinkered with) looks pretty appealing to me, though. Do you suggest using C instead of Rust? If so, why? I'd probably suggest a higher level language if you can get away with it (portability, performance, etc), but I am not really advocating for using one language over another here.. What I am advocating is writing a thin layer of abstraction to avoid buffer overflows in the already extant code base. I suspect it would be a better choice in terms of cost/benefit than re-writing the code.
      - 10y ago
sbov 10y ago
This is everyone's favorite excuse to trot out. But in reality the vast majority of projects I've seen never actually measure WHAT the performance cost would be and whether or not it's acceptable. So in the usual case the answer tends to be a mix of laziness and "that's how it's always been done."
mistaken 10y ago
To be fair the [] syntax is an overridable operator and you could just point it to use the "at" method. Not sure why they've implemented it as an usafe function.
- loup-vaillant 10y ago
  With C++, the default is zero overhead, not safety.
  - pjmlp 10y ago
    Not 100% true when using the standard library and compilers like VC++, which do provide such checks in debug builds, while allowing to selectively turn them on, on release builds.
  - temac 10y ago
    If people start to consider bound checking branches as overhead (which, in some extremely rare limited cases, they are in the right to do so), they should as well understand what happen with e.g. some largely used calling convention such as the one of Windows. Even in optimised builds, a unique_ptr<> for example can have an overhead compared to a raw pointer, IIRC because it forces going through the stack. If I remember my typical latencies correctly, for modern Intel CPU and probably a lot of high perf CPU, that can actually in some cases have a greater impact than an extra bound check... So if you are in a so performance critical section that you start to care about the "zero overhead" kind of stuff and the cost of your (predicted) bound checks, you might be impacted by this non-zero overhead... (that is falsely widely believed to be zero!) And don't get me started about debug builds with all mainstream compilers. The performance is then complete utter shit. This is made worse by the fact such an unsafe language needs debug builds more... C++ was an interesting experiment in its domain, and has been and still is a success in some aspects, but honestly given core language modifications are needed to significantly extend the behaviour of core types (like vectors, unique_ptr, strings, etc. -- e.g. with the introduction of rvalue references and all the associated machinery and default constructors and so over), I'm starting to believe there are little advantages of this approach over integrating such fundamental types/concepts in the core language (I'm not advocating to do that for C++, I'm thinking about other/future languages). Then you can have more true "zero overhead" stuffs, whatever that means.
EugeneOZ 10y ago
No, Rust arrays are not safe by default: https://is.gd/iY5lPQ https://is.gd/iY5lPQ
- steveklabnik 10y ago
  Panics are definitely safe. They are significantly different than the segfault you'd get out of the C code.
  - EugeneOZ 10y ago
    Besides of memory safety, what is different in panicking? I'm not trying to argue - I don't know it. For me result is the same - process failed. What is difference in consequences for memory?
    - gue5t 10y ago
      Not all violations of memory safety result in immediate program termination. There's an intervening period of execution during which your program simply performs unintended operations such as overwriting unrelated memory (which could be a memory-mapped file), terminating "correctly" (and performing whatever operations that entails) with corrupt state, and generally an unbounded set of other potential consequences. An immediate segfault is the absolute best hope for a program without memory safety, since you both see no unintended effects performed on the program's behalf and you learn that your program needs fixing.
      - EugeneOZ 10y ago
        Got it, thank you.
    - gpderetta 10y ago
      Panic behaviour is well defined. Segfaults are non-reliable (i.e. it might or might not crash) and often exploitable.
  - ue_ 10y ago

sidlls 10y ago

How do you suppose runtime bounds checks are done in Rust? They certainly also incur a performance penalty in not-trivial cases. Also, "safe by grep audit" means "safe according to a human." The argument of course is that it lowers the surface area of what a human must be trusted to verify. I'm still not convinced by that argument, because human error is a thing. And for actual systems programming, "very rare" may not be true.

staticassertion 10y ago
> How do you suppose runtime bounds checks are done in Rust? They certainly also incur a performance penalty in not-trivial cases. Certainly. I didn't intend to imply otherwise. > Also, "safe by grep audit" means "safe according to a human." Again, totally correct here. > The argument of course is that it lowers the surface area of what a human must be trusted to verify. I'm still not convinced by that argument, because human error is a thing. And for actual systems programming, "very rare" may not be true. Well, given a codebase where both safe and unsafe code exists, the amount of unsafe code is strictly less than the amount of both safe and unsafe code. So it does reduce the amount of code needed to audit, even in a very atypical case where a ton of the code is unsafe. It's true that a project may use egregious amounts of unsafe. That would be unfortunate. Rust is still safer than C in that case, since it just defines more behavior (like arithmetic overflow), but I certainly wouldn't pretend that the rust code should be trusted. When writing rust one should certainly strive to write less unsafe code, and to always document the invariants required for unsafe code to be safe. Rust is not 100% safe 100% of the time, I'm only arguing that safe defaults are critical, and that grep auditing is a powerful tool.
- EugeneOZ 10y ago
  Rust is safe in terms of memory usage and race conditions, nobody claims that Rust compiler will catch all 100% of bugs human can invent.
  - sidlls 10y ago
    Rust is safe by default in terms of memory usage. It is not, however, strictly memory safe. It is trivial to overflow a buffer in Rust, for example. I haven't discovered a trivial way to hide it, though.
    - steveklabnik 10y ago
      If Rust is not memory safe in safe code, then you've found a bug. Please report it to https://www.rust-lang.org/security.html https://www.rust-lang.org/security.html
      - sidlls 10y ago
        Let me clarify: in Rust it is trivial to use memory unsafely. It is not, so far as I have found, trivial to hide that fact because it is required to use "unsafe" syntax decoration" to do so.
        Manishearth 10y ago
        The issue here is that that definition of "safe" language basically excludes all practical languages, including languages like Python, because FFI is possible. In general when talking about safety in a language it's about the level of explicitness required to trigger unsafety. I like the distinction made in the nomicon (https://doc.rust-lang.org/stable/nomicon/meet-safe-and-unsafe.html https://doc.rust-lang.org/stable/nomicon/meet-safe-and-unsaf...) -- Rust comprises of two distinct languages. You have everyday Rust, which is completely memory safe, and "unsafe Rust", which looks similar to everyday Rust but is not safe. `unsafe {}` blocks are your FFI between the two. Looking at unsafe blocks as FFI is IMO a very useful mental model especially for understanding the changes to invariants involved.
rtpg 10y ago
"Actual systems programming" mostly does not involve unsafe code. For example, most OS code is _not_ interrupt hooks or malloc but the rest of the OS. Most of Postgres is not reading data quickly, but higher-level abstractions. Large-scale systems programming will always be mostly higher-level abstractions, because that's the only way to write large programs. Name any "systems programming" OSS project, choose a random C file, and you'll see that most code does not require pointer arithmetic except because that's how you do things in C. You don't bit-twiddle for 200k lines of code, so being able to limit dangerous stuff like that to the 5k lines that actually need it makes work an order of magnitude easier.
- anarazel 10y ago
  > "Actual systems programming" mostly does not involve unsafe code. True. Or slightly rephrased: Most of the unsafe code is centralized to some core pieces. From the POV of a PG dev: The big problem using something like rust for something like PostgreSQL is its it's portability, stability and uncertainty about where things are going. We do five years of back-branch releases (and for many that's not even enough!). Language and tooling around the new crop of languages simply aren't mature enough for that yet.

jdmichal 10y ago

In addition, the Rust compiler can also remove the built-in indexing checks if it can prove the code is safe. So, say, iterator loops over an array won't have any index checking.

ori_b 10y ago
So can the C compiler. The optimization is called 'value range propagation'.
dbaupp 10y ago
As others have said, this is a fairly standard optimisation: compilers (Rust, C, C++, Java, PHP, whatever) will remove branches if they can see that it can never be taken. Iterators are slightly different in that they don't have any indexing checks at all: an array iterator yields a plain reference, and these are always dereferenced like plain pointers.
staticassertion 10y ago
C/ C++ compilers are also capable of doing this, though rust's iterator syntax tends to make it a pretty natural optimization.
- Animats 10y ago
  Neither C nor C++ knows, at the language level, the size of an array, unless that size is fixed. The subscript checking variants of C and C++ have to use "fat pointers" which carry along size information. The overhead for this is large and nobody uses that. Fat pointers used to be a feature you could turn on in gcc, but it's somewhat abandoned now.
  - sirclueless 10y ago
    > Neither C nor C++ knows, at the language level, the size of an array, unless that size is fixed. Neither does Rust. > The subscript checking variants of C and C++ have to use "fat pointers" which carry along size information. So do Rust's Slices. > The overhead for this is large and nobody uses that. People use std::vector all the time for this purpose in C++. It has about the performance you'd expect, with very little overhead except where you want it in bounds-checking. I don't think there's actually a performance difference here. Rust's default is safer because it requires dropping to unsafe code to do something dangerous, but the same optimizations are available in both.

lacampbell 10y ago

Would you care to provide a short rust implementation of reading an arbitrary length string from standard input? I have rust installed. I would be curious to benchmark it and see if is indeed faster than the same thing in C, using a user-defined bounds checked array.

_sh 10y ago
It's not so much a question of benchmarks, it's that one of the standard C tools for reading an arbitrary string from standard input is gets(). And if you reach for that from the standard toolbox, you've failed before you've started.
- lacampbell 10y ago
  Many rust advocates talk about the performance cost of doing things safely in C, and inform me that rust has "Zero-cost abstractions". So it's fairly natural I ask for something I can benchmark.
  - tjr 10y ago
    Having used C (on and off) for about twenty years, including writing aerospace software in C, I strongly agree that writing safe C is possible. Rust (which I have thus far only tinkered with) looks pretty appealing to me, though. Do you suggest using C instead of Rust? If so, why?
    - lacampbell 10y ago
      Rust (which I have thus far only tinkered with) looks pretty appealing to me, though. Do you suggest using C instead of Rust? If so, why? I'd probably suggest a higher level language if you can get away with it (portability, performance, etc), but I am not really advocating for using one language over another here.. What I am advocating is writing a thin layer of abstraction to avoid buffer overflows in the already extant code base. I suspect it would be a better choice in terms of cost/benefit than re-writing the code.
    - 10y ago

sbov 10y ago

This is everyone's favorite excuse to trot out. But in reality the vast majority of projects I've seen never actually measure WHAT the performance cost would be and whether or not it's acceptable. So in the usual case the answer tends to be a mix of laziness and "that's how it's always been done."

mistaken 10y ago

To be fair the [] syntax is an overridable operator and you could just point it to use the "at" method. Not sure why they've implemented it as an usafe function.

loup-vaillant 10y ago
With C++, the default is zero overhead, not safety.
- pjmlp 10y ago
  Not 100% true when using the standard library and compilers like VC++, which do provide such checks in debug builds, while allowing to selectively turn them on, on release builds.
- temac 10y ago
  If people start to consider bound checking branches as overhead (which, in some extremely rare limited cases, they are in the right to do so), they should as well understand what happen with e.g. some largely used calling convention such as the one of Windows. Even in optimised builds, a unique_ptr<> for example can have an overhead compared to a raw pointer, IIRC because it forces going through the stack. If I remember my typical latencies correctly, for modern Intel CPU and probably a lot of high perf CPU, that can actually in some cases have a greater impact than an extra bound check... So if you are in a so performance critical section that you start to care about the "zero overhead" kind of stuff and the cost of your (predicted) bound checks, you might be impacted by this non-zero overhead... (that is falsely widely believed to be zero!) And don't get me started about debug builds with all mainstream compilers. The performance is then complete utter shit. This is made worse by the fact such an unsafe language needs debug builds more... C++ was an interesting experiment in its domain, and has been and still is a success in some aspects, but honestly given core language modifications are needed to significantly extend the behaviour of core types (like vectors, unique_ptr, strings, etc. -- e.g. with the introduction of rvalue references and all the associated machinery and default constructors and so over), I'm starting to believe there are little advantages of this approach over integrating such fundamental types/concepts in the core language (I'm not advocating to do that for C++, I'm thinking about other/future languages). Then you can have more true "zero overhead" stuffs, whatever that means.

EugeneOZ 10y ago

No, Rust arrays are not safe by default: https://is.gd/iY5lPQ https://is.gd/iY5lPQ

steveklabnik 10y ago
Panics are definitely safe. They are significantly different than the segfault you'd get out of the C code.
- EugeneOZ 10y ago
  Besides of memory safety, what is different in panicking? I'm not trying to argue - I don't know it. For me result is the same - process failed. What is difference in consequences for memory?
  - gue5t 10y ago
    Not all violations of memory safety result in immediate program termination. There's an intervening period of execution during which your program simply performs unintended operations such as overwriting unrelated memory (which could be a memory-mapped file), terminating "correctly" (and performing whatever operations that entails) with corrupt state, and generally an unbounded set of other potential consequences. An immediate segfault is the absolute best hope for a program without memory safety, since you both see no unintended effects performed on the program's behalf and you learn that your program needs fixing.
    - EugeneOZ 10y ago
      Got it, thank you.
  - gpderetta 10y ago
    Panic behaviour is well defined. Segfaults are non-reliable (i.e. it might or might not crash) and often exploitable.
- ue_ 10y ago