8 ms·

Achievement unlocked: rustc segfault

380 points by bcantrill 5y ago

akomtu 5y ago
Offtopic. Can we make a piece of software that automatically pays for discovered bugs with some form of cryptocurrency? The idea inspired by Knuth, who offered to pay twice more for each new found bug in TeX.
- scatters 5y ago
  Well sure, each and every smart contract has an automatic bug bounty attached.
- TheDong 5y ago
  > Can we make a piece of software that automatically pays for discovered bugs with some form of cryptocurrency? In the general case, no, not without having AGI. There are a subset of bugs that computers can verify (i.e. for a compiler, it might be asserted that "any input that causes a crash is a bug"), but the majority of bugs will require a human-like intelligence to read them and reason about them to determine that "yes, this is a bug". Given you have to involve a human in the loop to actually fix it, it seems like automating a bounty for it is fixing a problem that doesn't really exist. There's also vanishingly few projects that want to pay for literally any bug. The majority only want to pay for critical bugs or security vulnerabilities, both of which are purely human judgements.
- greiskul 5y ago
  It's a very bad idea to make any sort of completely automated system for dispensing money, cause you will have human adversaries that will try to exploit it.
hu3 5y ago
One of the reasons aircrafts like Boeing runs on safer languages like Ada. There's a pretty cool article about it: http://archive.adaic.com/projects/atwork/boeing.html http://archive.adaic.com/projects/atwork/boeing.html edit: To those stating it was C++ portion of the compiler code at fault. That's not the point. Ada offers greater protection against segfaults than Rust. Not to mention that part of the compiler still being written in C++ and not Rust doesn't bode well.
- johnisgood 5y ago
  This is going to invite all the people who know nothing about the language and the circumstances with regarding to Ariane 5. :)
  - hu3 5y ago
    Agreed. To those seeking more information about this, I always refer to this article because of the links it contains: http://www.rvs.uni-bielefeld.de/publications/Incidents/DOCS/Research/Rvs/Misc/Additional/Reports/ariane.html http://www.rvs.uni-bielefeld.de/publications/Incidents/DOCS/...
- __jem 5y ago
  I didn’t read that closely, but isn’t this is a segfault in the compiler, not a miscompilation?
  - titzer 5y ago
    Nothing inspires confidence like the compiler crashing. Given C/C++'s attitude towards UB, a crash just means you got lucky and the program really f'd up before doing something seriously nefarious like silent memory corruption or carrying on with nonsensical results....
    - kelnos 5y ago
      As a practical matter, though, I'd much rather the compiler segfault, than miscompile my code so it does something incorrect at runtime. But agreed that seeing a compiler segfault doesn't inspire confidence!
      - titzer 5y ago
        Me too!
  - gpm 5y ago
    Yes. A segfault in the portion of the compiler written in C++ no less, though triggered by invalid input from the portion of the compiler written in rust (due to a logic bug). Unless GPs point is that "rust code can still have bugs" (so can ADA) or "the rust compiler still has bugs" (I'm willing to bet so do all ADA compilers) I don't see how this supports his claim.
rurban 5y ago
Pretty easy to get llvm segfaults, I got that with pony also in development versions. The difference to rust is that this happens with rust compiled binaries also at run-time, in pony very rarely. https://github.com/rust-lang/rust/issues?q=is%3Aissue+SIGSEGV https://github.com/rust-lang/rust/issues?q=is%3Aissue+SIGSEG... => 87 + 291 vs https://github.com/ponylang/ponyc/issues?q=is%3Aissue+SIGSEGV https://github.com/ponylang/ponyc/issues?q=is%3Aissue+SIGSEG... => 11 + 50 Now compare that to a safe systems language, e.g. sbcl: https://bugs.launchpad.net/sbcl/+bugs?field.searchtext=SIGSEGV https://bugs.launchpad.net/sbcl/+bugs?field.searchtext=SIGSE... => 1 (invalid) or clisp: https://sourceforge.net/p/clisp/bugs/?q=%7B%22status%22%3A+%7B%22%24nin%22%3A+%5B%22closed%22%2C+%22closed-accepted%22%2C+%22closed-duplicate%22%2C+%22closed-fixed%22%2C+%22closed-invalid%22%2C+%22closed-out-of-date%22%2C+%22closed-postponed%22%2C+%22closed-rejected%22%2C+%22closed-remind%22%2C+%22closed-wont-fix%22%2C+%22closed-works-for-me%22%5D%7D%7D&limit=25&page=0&sort=ticket_num%20desc&filter=%7B%22_milestone%22%3A%5B%22segfault%22%5D%7D&columns-0.name=ticket_num&columns-0.sort_name=ticket_num&columns-0.label=Ticket%20Number&columns-0.active=on&columns-1.name=summary&columns-1.sort_name=summary&columns-1.label=Summary&columns-1.active=on&columns-2.name=_milestone&columns-2.sort_name=custom_fields._milestone&columns-2.label=Milestone&columns-2.active=on&columns-3.name=status&columns-3.sort_name=status&columns-3.label=Status&columns-3.active=on&columns-4.name=assigned_to&columns-4.sort_name=assigned_to_username&columns-4.label=Owner&columns-4.active=on&columns-5.name=reported_by&columns-5.sort_name=reported_by&columns-5.label=Creator&columns-5.active=on&columns-6.name=created_date&columns-6.sort_name=created_date&columns-6.label=Created&columns-6.active=on&columns-7.name=mod_date&columns-7.sort_name=mod_date&columns-7.label=Updated&columns-7.active=on&columns-8.name=labels&columns-8.sort_name=labels&columns-8.label=Labels&columns-8.active=&columns-9.name=_priority&columns-9.sort_name=_priority&columns-9.label=Priority&columns-9.active=on https://sourceforge.net/p/clisp/bugs/?q=%7B%22status%22%3A+%... => 15
- woodruffw 5y ago
  Not to besmirch Pony (I think it's a very interesting language), but isn't some of this a base sampling error? There are probably a few orders of magnitude more engineers writing Rust (and therefore running `rustc`), so we'd expect crashes in Rust to surface more frequently.
EdSchouten 5y ago
It’s amazing how in-line assembly is always such a big source of miscompilations/compiler crashes. Both with C/C++ and Rust. With GCC or Clang. It doesn’t matter.
- gpm 5y ago
  I think it makes sense as a place to expect bugs. It's a rarely used feature which interacts in a unique way with some pretty core parts of the compiler, in a platform dependent manner.
- mhh__ 5y ago
  Although in recent years the representation has become a lot more abstract (and people to correctly generate that abstract representation), inline assembler tends to have a lot of monotonous drivel to get right.
- kibwen 5y ago
  Amusingly, if I recall correctly, the author of this post was the author of Rust's original (never stable, now removed) inline assembly support, years and years ago. Certainly one of the best people in the world to have encountered this bug!
- deleted 5y ago
  [deleted]
monocasa 5y ago
Oh, lol, I guess I get that achievement too. Custom target spec json with incompatible MIPS cpu and abi options was what did it. That was a few years back, but post 1.0 at least.
kelnos 5y ago
Perhaps I missed it, but too bad OP didn't submit a fix to LLVM as well, or at least file a bug report there. Sure, rustc was emitting bad IR, but LLVM shouldn't crash! It should catch the issue and exit cleanly with an error message. Probably would have been easier to debug this issue if LLVM hadn't crashed in the first place, too. Either way, a really fun read. For some reason I enjoy reading debugging stories for bugs that I almost certainly wouldn't be able to solve myself.
- mshockwave 5y ago
  > It should catch the issue and exit cleanly with an error message. IIRC LLVM's IR verification is not enabled in release build. In other words, if you're using rustc with debug version of LLVM the error message should pop up. EDIT: "...LLVM's IR verification is not enabled in release build..." this is wrong, LLVM doesn't turn off verification based on its build mode. It is up to the user of LLVM, namely rustc in this case, to enable verification. For instance, you can verify IR after each optimization Pass (which is pretty expensive) by configuring `llvm::StandardInstrumentation` properly. Or you can verify the IR before codegen by switching up one of `llvm::TargetMachine`'s options. Clang always enables the latter (verification) by default but disables the former regardless of the optimization level or its build mode.
  - rob74 5y ago
    ...and it's probably not enabled because the Rust compiler is already slow enough as it is? But yeah, I guess it's a fair trade-off having 0.0001% of builds crash if it makes the other 99.9999% a bit faster...
  - EE84M3i 5y ago
    In either case dereferencing a null pointer would still be a bug, right? Or is it kind of "all bets are off" if you feed LLVM bad IR and don't enable verification?
    - comex 5y ago
      It is.
trollian 5y ago
Getting LLVM to SEGV is depressingly easy.
- VWWHFSfQ 5y ago
  GCC would never compile this code. Just saying...
- account42 5y ago
  LLVM, GCC, MSVC - all are complex enough that you run into internal compiler errors from time to time. Maybe a bit less for MSVC since you don't get to run random development snapshots, but still above 0.
VWWHFSfQ 5y ago
This doesn't seem like a bug in rustc at all. Maybe just by emitting bad IR. But this is actually bug in LLVM.
- DannyBee 5y ago
  why? They are using a build with assertions/etc disabled, so it segfaults rather than asserts. When you use a build that is set not to do that, it gives you an error :)
- woodruffw 5y ago
  I think it's reasonable to consider it a bug in both: it's a MIR lowering bug, one that gets promoted into a different bug by LLVM.
- LAC-Tech 5y ago
  If rustc uses a dependency that has a bug and this manifests itself as a bug in rustc, then IMO rustc still has a bug.
- tsimionescu 5y ago
  No, LLVM is configured by rustc to NOT to check its input (for performance reasons), so it's normal for it to crash if fed invalid input. The bug is 100% in rustc's court.
wyldfire 5y ago
Note that assertions or compiler segfaults or other similar issues can be automagically reduced to the critical input using tools like c-reduce/cvise. Sure, they're designed for C and C-like languages, but they're tolerable for rust, assembly languages, etc. Probably anything that has text with line separators would work reasonably well. Also for LLVM IR, there's llvm-reduce and -opt-bisect-limit. Not sure if they'd have helped here but they're generally useful tools for tracking down similar issues.
- ahmedalsudani 5y ago
  Wow those look like great tools! Thank you for sharing. There were GCC segfaults that started appearing on my machine a few months ago, and they magically disappeared after a recent update. Those tools are going to be a massive help next time that happens.
shp0ngle 5y ago
Yeah I did not understand a word of this.
- rekoil 5y ago
  There was a meme in there that I could enjoy at least!

380 points by bcantrill 5y ago

Offtopic. Can we make a piece of software that automatically pays for discovered bugs with some form of cryptocurrency? The idea inspired by Knuth, who offered to pay twice more for each new found bug in TeX.

scatters 5y ago
Well sure, each and every smart contract has an automatic bug bounty attached.
TheDong 5y ago
> Can we make a piece of software that automatically pays for discovered bugs with some form of cryptocurrency? In the general case, no, not without having AGI. There are a subset of bugs that computers can verify (i.e. for a compiler, it might be asserted that "any input that causes a crash is a bug"), but the majority of bugs will require a human-like intelligence to read them and reason about them to determine that "yes, this is a bug". Given you have to involve a human in the loop to actually fix it, it seems like automating a bounty for it is fixing a problem that doesn't really exist. There's also vanishingly few projects that want to pay for literally any bug. The majority only want to pay for critical bugs or security vulnerabilities, both of which are purely human judgements.
greiskul 5y ago
It's a very bad idea to make any sort of completely automated system for dispensing money, cause you will have human adversaries that will try to exploit it.

hu3 5y ago

One of the reasons aircrafts like Boeing runs on safer languages like Ada. There's a pretty cool article about it: http://archive.adaic.com/projects/atwork/boeing.html http://archive.adaic.com/projects/atwork/boeing.html edit: To those stating it was C++ portion of the compiler code at fault. That's not the point. Ada offers greater protection against segfaults than Rust. Not to mention that part of the compiler still being written in C++ and not Rust doesn't bode well.

johnisgood 5y ago
This is going to invite all the people who know nothing about the language and the circumstances with regarding to Ariane 5. :)
- hu3 5y ago
  Agreed. To those seeking more information about this, I always refer to this article because of the links it contains: http://www.rvs.uni-bielefeld.de/publications/Incidents/DOCS/Research/Rvs/Misc/Additional/Reports/ariane.html http://www.rvs.uni-bielefeld.de/publications/Incidents/DOCS/...
__jem 5y ago
I didn’t read that closely, but isn’t this is a segfault in the compiler, not a miscompilation?
- titzer 5y ago
  Nothing inspires confidence like the compiler crashing. Given C/C++'s attitude towards UB, a crash just means you got lucky and the program really f'd up before doing something seriously nefarious like silent memory corruption or carrying on with nonsensical results....
  - kelnos 5y ago
    As a practical matter, though, I'd much rather the compiler segfault, than miscompile my code so it does something incorrect at runtime. But agreed that seeing a compiler segfault doesn't inspire confidence!
    - titzer 5y ago
      Me too!
- gpm 5y ago
  Yes. A segfault in the portion of the compiler written in C++ no less, though triggered by invalid input from the portion of the compiler written in rust (due to a logic bug). Unless GPs point is that "rust code can still have bugs" (so can ADA) or "the rust compiler still has bugs" (I'm willing to bet so do all ADA compilers) I don't see how this supports his claim.

rurban 5y ago

Pretty easy to get llvm segfaults, I got that with pony also in development versions. The difference to rust is that this happens with rust compiled binaries also at run-time, in pony very rarely. https://github.com/rust-lang/rust/issues?q=is%3Aissue+SIGSEGV https://github.com/rust-lang/rust/issues?q=is%3Aissue+SIGSEG... => 87 + 291 vs https://github.com/ponylang/ponyc/issues?q=is%3Aissue+SIGSEGV https://github.com/ponylang/ponyc/issues?q=is%3Aissue+SIGSEG... => 11 + 50 Now compare that to a safe systems language, e.g. sbcl: https://bugs.launchpad.net/sbcl/+bugs?field.searchtext=SIGSEGV https://bugs.launchpad.net/sbcl/+bugs?field.searchtext=SIGSE... => 1 (invalid) or clisp: https://sourceforge.net/p/clisp/bugs/?q=%7B%22status%22%3A+%7B%22%24nin%22%3A+%5B%22closed%22%2C+%22closed-accepted%22%2C+%22closed-duplicate%22%2C+%22closed-fixed%22%2C+%22closed-invalid%22%2C+%22closed-out-of-date%22%2C+%22closed-postponed%22%2C+%22closed-rejected%22%2C+%22closed-remind%22%2C+%22closed-wont-fix%22%2C+%22closed-works-for-me%22%5D%7D%7D&limit=25&page=0&sort=ticket_num%20desc&filter=%7B%22_milestone%22%3A%5B%22segfault%22%5D%7D&columns-0.name=ticket_num&columns-0.sort_name=ticket_num&columns-0.label=Ticket%20Number&columns-0.active=on&columns-1.name=summary&columns-1.sort_name=summary&columns-1.label=Summary&columns-1.active=on&columns-2.name=_milestone&columns-2.sort_name=custom_fields._milestone&columns-2.label=Milestone&columns-2.active=on&columns-3.name=status&columns-3.sort_name=status&columns-3.label=Status&columns-3.active=on&columns-4.name=assigned_to&columns-4.sort_name=assigned_to_username&columns-4.label=Owner&columns-4.active=on&columns-5.name=reported_by&columns-5.sort_name=reported_by&columns-5.label=Creator&columns-5.active=on&columns-6.name=created_date&columns-6.sort_name=created_date&columns-6.label=Created&columns-6.active=on&columns-7.name=mod_date&columns-7.sort_name=mod_date&columns-7.label=Updated&columns-7.active=on&columns-8.name=labels&columns-8.sort_name=labels&columns-8.label=Labels&columns-8.active=&columns-9.name=_priority&columns-9.sort_name=_priority&columns-9.label=Priority&columns-9.active=on https://sourceforge.net/p/clisp/bugs/?q=%7B%22status%22%3A+%... => 15

woodruffw 5y ago
Not to besmirch Pony (I think it's a very interesting language), but isn't some of this a base sampling error? There are probably a few orders of magnitude more engineers writing Rust (and therefore running `rustc`), so we'd expect crashes in Rust to surface more frequently.

EdSchouten 5y ago

It’s amazing how in-line assembly is always such a big source of miscompilations/compiler crashes. Both with C/C++ and Rust. With GCC or Clang. It doesn’t matter.

gpm 5y ago
I think it makes sense as a place to expect bugs. It's a rarely used feature which interacts in a unique way with some pretty core parts of the compiler, in a platform dependent manner.
mhh__ 5y ago
Although in recent years the representation has become a lot more abstract (and people to correctly generate that abstract representation), inline assembler tends to have a lot of monotonous drivel to get right.
kibwen 5y ago
Amusingly, if I recall correctly, the author of this post was the author of Rust's original (never stable, now removed) inline assembly support, years and years ago. Certainly one of the best people in the world to have encountered this bug!
deleted 5y ago
[deleted]

monocasa 5y ago

Oh, lol, I guess I get that achievement too. Custom target spec json with incompatible MIPS cpu and abi options was what did it. That was a few years back, but post 1.0 at least.

kelnos 5y ago

Perhaps I missed it, but too bad OP didn't submit a fix to LLVM as well, or at least file a bug report there. Sure, rustc was emitting bad IR, but LLVM shouldn't crash! It should catch the issue and exit cleanly with an error message. Probably would have been easier to debug this issue if LLVM hadn't crashed in the first place, too. Either way, a really fun read. For some reason I enjoy reading debugging stories for bugs that I almost certainly wouldn't be able to solve myself.

mshockwave 5y ago
> It should catch the issue and exit cleanly with an error message. IIRC LLVM's IR verification is not enabled in release build. In other words, if you're using rustc with debug version of LLVM the error message should pop up. EDIT: "...LLVM's IR verification is not enabled in release build..." this is wrong, LLVM doesn't turn off verification based on its build mode. It is up to the user of LLVM, namely rustc in this case, to enable verification. For instance, you can verify IR after each optimization Pass (which is pretty expensive) by configuring `llvm::StandardInstrumentation` properly. Or you can verify the IR before codegen by switching up one of `llvm::TargetMachine`'s options. Clang always enables the latter (verification) by default but disables the former regardless of the optimization level or its build mode.
- rob74 5y ago
  ...and it's probably not enabled because the Rust compiler is already slow enough as it is? But yeah, I guess it's a fair trade-off having 0.0001% of builds crash if it makes the other 99.9999% a bit faster...
- EE84M3i 5y ago
  In either case dereferencing a null pointer would still be a bug, right? Or is it kind of "all bets are off" if you feed LLVM bad IR and don't enable verification?
  - comex 5y ago
    It is.

trollian 5y ago

Getting LLVM to SEGV is depressingly easy.

VWWHFSfQ 5y ago
GCC would never compile this code. Just saying...
account42 5y ago
LLVM, GCC, MSVC - all are complex enough that you run into internal compiler errors from time to time. Maybe a bit less for MSVC since you don't get to run random development snapshots, but still above 0.

VWWHFSfQ 5y ago

This doesn't seem like a bug in rustc at all. Maybe just by emitting bad IR. But this is actually bug in LLVM.

DannyBee 5y ago
why? They are using a build with assertions/etc disabled, so it segfaults rather than asserts. When you use a build that is set not to do that, it gives you an error :)
woodruffw 5y ago
I think it's reasonable to consider it a bug in both: it's a MIR lowering bug, one that gets promoted into a different bug by LLVM.
LAC-Tech 5y ago
If rustc uses a dependency that has a bug and this manifests itself as a bug in rustc, then IMO rustc still has a bug.
tsimionescu 5y ago
No, LLVM is configured by rustc to NOT to check its input (for performance reasons), so it's normal for it to crash if fed invalid input. The bug is 100% in rustc's court.

wyldfire 5y ago

Note that assertions or compiler segfaults or other similar issues can be automagically reduced to the critical input using tools like c-reduce/cvise. Sure, they're designed for C and C-like languages, but they're tolerable for rust, assembly languages, etc. Probably anything that has text with line separators would work reasonably well. Also for LLVM IR, there's llvm-reduce and -opt-bisect-limit. Not sure if they'd have helped here but they're generally useful tools for tracking down similar issues.

ahmedalsudani 5y ago
Wow those look like great tools! Thank you for sharing. There were GCC segfaults that started appearing on my machine a few months ago, and they magically disappeared after a recent update. Those tools are going to be a massive help next time that happens.

shp0ngle 5y ago

Yeah I did not understand a word of this.

rekoil 5y ago
There was a meme in there that I could enjoy at least!