5 ms·

I don't mean this to sound superior or critical. I found this article a little surprising because the authors seem to have an intuition about the compiler that

I don't mean this to sound superior or critical. I found this article a little surprising because the authors seem to have an intuition about the compiler that I have never had. I've never expected potential stack overflows from unbounded recursion to be caught by the compiler. I've been delighted every now and then when a compiler gives me a warning when it can detect it. But I have always had stack overflows in my mental bucket of "stuff it's my responsibility to think about". Maybe this says something about how compilers have gotten smarter and more helpful since I first cut my teeth on them.

humanrebar 4y ago
The branding (or at least the zeitgeist) around Rust is that it is a "safe language", which implies that bugs are all logic bugs or somehow the fault of the programmer for doing something "unsafe" (hence the keyword). Anyone who thinks about it for a bit knows there are limits to guarantees of safety. It's notoriously impossible to prove that a program terminates appropriately, for instance. So I can understand some shock to run into the arcane all of a sudden. It must be especially shocking because it feels especially out of place.
- ecshafer 4y ago
  Rust is a safe language for a specific definition of safe regarding a class of memory errors.
  - pa7ch 4y ago
    Regarding multiple classes of memory errors I think. Many languages are memory safe from control flow takeover, rust extends this to data races corrupting data.
  - Georgelemental 4y ago
    Rust (without `unsafe`) protects you from Undefined Behavior. Behavior that is defined as crashing your program doesn't fall under that.
- bobbylarrybobby 4y ago
  Rust is pretty clear about “safe” meaning no memory unsafety in unsafe-free code. It certainly doesn't market that code without unsafe will be bug free (this would force the safe subset to no longer be Turing complete).
  - kibwen 4y ago
    It can still be surprising because in other languages it may be the case that a stack overflow can lead to memory unsafety. However, in Rust, a stack overflow leads to a guaranteed abort on major platforms, and a best-effort abort on niche platforms (the reason being that LLVM only provides stack probes for major platforms, so niche platforms have to make do with only guard pages).
    - ridiculous_fish 4y ago
      My understanding is that there is a decent amount of undefined behavior on stack overflow in Rust, due to its use of TLS. https://github.com/rust-lang/rust/issues/43146 https://github.com/rust-lang/rust/issues/43146
      - kibwen 4y ago
        I'm unclear how that would lead to undefined behavior on stack overflow. Stack overflow on Rust is specified to immediately abort the entire process (with some aforementioned wrinkles as to how achievable that is on certain targets), which necessarily nukes all TLS. The fact that the linked issue isn't tagged with "I-unsound" suggests that nobody has ever indicated that that might somehow be a soundness issue. If you think otherwise, I encourage you to bring it up so that any soundness issue can be properly tracked.
        swsieber 4y ago
        To push this point a little further: in Rust when there's not a clean way to prevent memory unsafety (or when you haven't requested to), it aborts the process. Out of array indexing, stack overflows, etc.
- hinkley 4y ago
  I have chosen this as a hill to die upon: Philosophically, the Halting Problem and the Pigeonhole Principle are very similar. They both talk about what is not possible with arbitrary input, but they have very little to say about practical input. You can't compress a random number generator. You can compress human communication, in any form, some of them famously. The existence of PP or HP are only boundary conditions, as entropy goes to infinity. Humans also call entropy 'chaos', which has a connotation that's doubly meaningful in information theory. Chaos obfuscates. It's only useful when obfuscation is the point (encryption). We run away from it as fast as we can, and the farther we get from in the more interesting opportunities open up. Languages that disallow certain patterns can solve the halting problem for pieces of the application, maybe even entire 'useful' applications.
  - humanrebar 4y ago
    For some categories of applications, or even components of applications, the inability to meet a deadline in handling an event is a safety violation in the category of "or else people could die". I mentioned the halting problem instead since it's close enough to those requirements for an HN comment. Rust has nothing special to say about that definition of safety, except that it thinks it has a better model for avoiding certain kinds of memory bugs that could contribute to those issues. Given that, I can see that "safe language" as a concept can be confusing and even leave engineers a little unprepared for unexpected categories of bugs. Anyway, I brought up the halting problem to illustrate. I'm sure there are other illustrations you can substitute regardless of your opinion on the halting problem.
kccqzy 4y ago
I have the same thought! But then again when I tried a simple example in clang the -Winfinite-recursion warning triggered. In the compiled code, clang compiled the function as just `ret`, which is alright because C++ standard says if a thread does not eventually do an atomic op or call certain functions then it's undefined behavior. Still wouldn't recommend anyone to rely on the compiler to catch this.
m463 4y ago
> But I have always had stack overflows in my mental bucket of "stuff it's my responsibility to think about". I remember working on kernel modules, and coming from userland being surprised to find kernel stacks of 16k or 24k are common. Anything that could help with that without having to do kernel debugging on weird errors would be great.
- rcxdude 4y ago
  Static stack analysis is a thing, though it's a thing which annoyingly lacks any real open-source implementation, and any good tooling whatsoever in my experience. It does limit what you can do though, since dynamic dispatch and recursion can quickly make it fail.
Arnavion 4y ago
It's actually very easy for rustc to raise a compiler error if you have recursion inside an async fn. It naturally falls out from the way async is implemented, because the resulting Future type would be infinitely sized. An `async fn` essentially compiles to a state machine where each `.await` is a transition to another state, and local variables and the future being `.await`'d are data of that state. If a future were to `.await` itself, that state data would contain the future itself, which means the future would need to be larger than itself, which is impossible. Similarly corecursion would simultaneously require that `FooFuture` contain `BarFuture and `BarFuture` contain `FooFuture`, which would require both to be bigger than the other, which is again impossible. The solution to the infinite size problem is to wrap the future in `Box<dyn Future>` before `.await`ing it, so that the state data is a fixed size (`Box<dyn Trait>` is always two pointers long, a data pointer and a vtable pointer). Since TFA was using the `#[async_trait]` macro which changes the function to return `Box<dyn Future>` (as the stable alternative to async-fn-in-traits / TAIT), it ended up suppressing the recursion error too.
deleted 4y ago
[deleted]
bilekas 4y ago
It's certainly a Rust thing, and definitely a part of the appeal. It doesn't remove the "stuff it's my responsibility to think about" but it certainly alleviates it. Your comment actually sounds like the first step to becoming a 'Rustacean'.
chubot 4y ago
What I also think is weird is that people think seg faults and crashes are "unsafe". In the terms of Cardelli, a stack overflow is a trapped runtime error -- and thus a SAFE behavior of a language runtime. What's not safe is when your program keeps running in an unknown state: - buffer overflows - who knows what it does after memory corruption? - integer overflows, especially in C where the behavior isn't defined. But in Rust's release mode, these runtime checks are omitted, so that's unsafe too. --- On the other hand, the behavior of null pointer exceptions and a stack overflows is SAFE. It protects you from undefined behavior at runtime. Crashing is a DEFINED behavior -- it's also what happens on divide-by-zero by default, and apparently when Rust's allocators run out of heap memory. This isn't a theoretical distinction, because diagnosing and fixing NPEs and stack overflows is easy compared to the other two. And crashes can lead to denial of service, but not remote code execution, which is WAY worse.
- Someone 4y ago
  > On the other hand, the behavior of null pointer exceptions and a stack overflows is SAFE. For many languages only if the OS (sometimes with a bit of help from the runtime/ compiler) protects you against them. Not all language specs require programs to abort when they occur, if only because checking for them on systems without a MMU is expensive.
- btilly 4y ago
  I think that you're the weird one here. The most common reason to GET a segmentation fault is memory corruption. Frequently from a buffer overflow. Which makes them the canary in the coal mine for bugs that often turn out to be exploitable security vulnerabilities. So if a program segfaults on me and I don't know exactly why, I should dial my paranoia up and expect that it is insecure. That an attacker who looked at it could use it to take over my computer instead of simply leaving me with a core dump.
  - astrange 4y ago
    > So if a program segfaults on me and I don't know exactly why, I should dial my paranoia up and expect that it is insecure. This is true in that the program contains at least one bug, but some memory errors are safer than others. A SIGBUS is safer than SIGSEGV (usually means you mmapped a file and got an I/O error reading it) and a read is safer than a write. This comes up if you're prioritizing crashes from fuzzing.

humanrebar 4y ago

The branding (or at least the zeitgeist) around Rust is that it is a "safe language", which implies that bugs are all logic bugs or somehow the fault of the programmer for doing something "unsafe" (hence the keyword). Anyone who thinks about it for a bit knows there are limits to guarantees of safety. It's notoriously impossible to prove that a program terminates appropriately, for instance. So I can understand some shock to run into the arcane all of a sudden. It must be especially shocking because it feels especially out of place.

ecshafer 4y ago
Rust is a safe language for a specific definition of safe regarding a class of memory errors.
- pa7ch 4y ago
  Regarding multiple classes of memory errors I think. Many languages are memory safe from control flow takeover, rust extends this to data races corrupting data.
- Georgelemental 4y ago
  Rust (without `unsafe`) protects you from Undefined Behavior. Behavior that is defined as crashing your program doesn't fall under that.
bobbylarrybobby 4y ago
Rust is pretty clear about “safe” meaning no memory unsafety in unsafe-free code. It certainly doesn't market that code without unsafe will be bug free (this would force the safe subset to no longer be Turing complete).
- kibwen 4y ago
  It can still be surprising because in other languages it may be the case that a stack overflow can lead to memory unsafety. However, in Rust, a stack overflow leads to a guaranteed abort on major platforms, and a best-effort abort on niche platforms (the reason being that LLVM only provides stack probes for major platforms, so niche platforms have to make do with only guard pages).
  - ridiculous_fish 4y ago
    My understanding is that there is a decent amount of undefined behavior on stack overflow in Rust, due to its use of TLS. https://github.com/rust-lang/rust/issues/43146 https://github.com/rust-lang/rust/issues/43146
    - kibwen 4y ago
      I'm unclear how that would lead to undefined behavior on stack overflow. Stack overflow on Rust is specified to immediately abort the entire process (with some aforementioned wrinkles as to how achievable that is on certain targets), which necessarily nukes all TLS. The fact that the linked issue isn't tagged with "I-unsound" suggests that nobody has ever indicated that that might somehow be a soundness issue. If you think otherwise, I encourage you to bring it up so that any soundness issue can be properly tracked.
      - swsieber 4y ago
        To push this point a little further: in Rust when there's not a clean way to prevent memory unsafety (or when you haven't requested to), it aborts the process. Out of array indexing, stack overflows, etc.
hinkley 4y ago
I have chosen this as a hill to die upon: Philosophically, the Halting Problem and the Pigeonhole Principle are very similar. They both talk about what is not possible with arbitrary input, but they have very little to say about practical input. You can't compress a random number generator. You can compress human communication, in any form, some of them famously. The existence of PP or HP are only boundary conditions, as entropy goes to infinity. Humans also call entropy 'chaos', which has a connotation that's doubly meaningful in information theory. Chaos obfuscates. It's only useful when obfuscation is the point (encryption). We run away from it as fast as we can, and the farther we get from in the more interesting opportunities open up. Languages that disallow certain patterns can solve the halting problem for pieces of the application, maybe even entire 'useful' applications.
- humanrebar 4y ago
  For some categories of applications, or even components of applications, the inability to meet a deadline in handling an event is a safety violation in the category of "or else people could die". I mentioned the halting problem instead since it's close enough to those requirements for an HN comment. Rust has nothing special to say about that definition of safety, except that it thinks it has a better model for avoiding certain kinds of memory bugs that could contribute to those issues. Given that, I can see that "safe language" as a concept can be confusing and even leave engineers a little unprepared for unexpected categories of bugs. Anyway, I brought up the halting problem to illustrate. I'm sure there are other illustrations you can substitute regardless of your opinion on the halting problem.

kccqzy 4y ago

I have the same thought! But then again when I tried a simple example in clang the -Winfinite-recursion warning triggered. In the compiled code, clang compiled the function as just `ret`, which is alright because C++ standard says if a thread does not eventually do an atomic op or call certain functions then it's undefined behavior. Still wouldn't recommend anyone to rely on the compiler to catch this.

m463 4y ago

> But I have always had stack overflows in my mental bucket of "stuff it's my responsibility to think about". I remember working on kernel modules, and coming from userland being surprised to find kernel stacks of 16k or 24k are common. Anything that could help with that without having to do kernel debugging on weird errors would be great.

rcxdude 4y ago
Static stack analysis is a thing, though it's a thing which annoyingly lacks any real open-source implementation, and any good tooling whatsoever in my experience. It does limit what you can do though, since dynamic dispatch and recursion can quickly make it fail.

Arnavion 4y ago

It's actually very easy for rustc to raise a compiler error if you have recursion inside an async fn. It naturally falls out from the way async is implemented, because the resulting Future type would be infinitely sized. An `async fn` essentially compiles to a state machine where each `.await` is a transition to another state, and local variables and the future being `.await`'d are data of that state. If a future were to `.await` itself, that state data would contain the future itself, which means the future would need to be larger than itself, which is impossible. Similarly corecursion would simultaneously require that `FooFuture` contain `BarFuture and `BarFuture` contain `FooFuture`, which would require both to be bigger than the other, which is again impossible. The solution to the infinite size problem is to wrap the future in `Box<dyn Future>` before `.await`ing it, so that the state data is a fixed size (`Box<dyn Trait>` is always two pointers long, a data pointer and a vtable pointer). Since TFA was using the `#[async_trait]` macro which changes the function to return `Box<dyn Future>` (as the stable alternative to async-fn-in-traits / TAIT), it ended up suppressing the recursion error too.

deleted 4y ago

[deleted]

bilekas 4y ago

It's certainly a Rust thing, and definitely a part of the appeal. It doesn't remove the "stuff it's my responsibility to think about" but it certainly alleviates it. Your comment actually sounds like the first step to becoming a 'Rustacean'.

chubot 4y ago

What I also think is weird is that people think seg faults and crashes are "unsafe". In the terms of Cardelli, a stack overflow is a trapped runtime error -- and thus a SAFE behavior of a language runtime. What's not safe is when your program keeps running in an unknown state: - buffer overflows - who knows what it does after memory corruption? - integer overflows, especially in C where the behavior isn't defined. But in Rust's release mode, these runtime checks are omitted, so that's unsafe too. --- On the other hand, the behavior of null pointer exceptions and a stack overflows is SAFE. It protects you from undefined behavior at runtime. Crashing is a DEFINED behavior -- it's also what happens on divide-by-zero by default, and apparently when Rust's allocators run out of heap memory. This isn't a theoretical distinction, because diagnosing and fixing NPEs and stack overflows is easy compared to the other two. And crashes can lead to denial of service, but not remote code execution, which is WAY worse.

Someone 4y ago
> On the other hand, the behavior of null pointer exceptions and a stack overflows is SAFE. For many languages only if the OS (sometimes with a bit of help from the runtime/ compiler) protects you against them. Not all language specs require programs to abort when they occur, if only because checking for them on systems without a MMU is expensive.
btilly 4y ago
I think that you're the weird one here. The most common reason to GET a segmentation fault is memory corruption. Frequently from a buffer overflow. Which makes them the canary in the coal mine for bugs that often turn out to be exploitable security vulnerabilities. So if a program segfaults on me and I don't know exactly why, I should dial my paranoia up and expect that it is insecure. That an attacker who looked at it could use it to take over my computer instead of simply leaving me with a core dump.
- astrange 4y ago
  > So if a program segfaults on me and I don't know exactly why, I should dial my paranoia up and expect that it is insecure. This is true in that the program contains at least one bug, but some memory errors are safer than others. A SIGBUS is safer than SIGSEGV (usually means you mmapped a file and got an I/O error reading it) and a read is safer than a write. This comes up if you're prioritizing crashes from fuzzing.