5 ms·

No it isn't. It's just evidence that it's a trade-off you might want to make in order to achieve some other goal, specifically security. But if "security" isn'

No it isn't. It's just evidence that it's a trade-off you might want to make in order to achieve some other goal, specifically security. But if "security" isn't remotely a concern for a given project (like almost anything graphics / gaming related), this is not at all evidence for changing anything. It could be that Rust's optimizer eliminates the bounds checking so regularly as to be a moot point, but this isn't saying anything of the sort. It's saying that the cost, whatever it was, was judged to be worth paying for the improved security for these projects

bluGill 4y ago
I take it the opposite: C programmers out of abundance of caution of putting in bounds checks for code that will never be called with out of bound data. As such rust is eliminating code that is being manually written. If you don't write the bounds check in C, and rust for the equivalent determines that the bounds check isn't needed the code should be the same (to the assembly level). However if you write a bounds check in C the optimizer might not eliminate it.
- kllrnohj 4y ago
  Can you point to any such bounds check in C that an optimizer cannot eliminate but it can eliminate the equivalent one in Rust? I'm sure it's possible to construct such a thing, but I cannot imagine it ever being common enough to show up on any sort of head to head comparison.
  - Gankra 4y ago
    there's almost no bounds checking in rust code before the optimizer even looks at it because we use iterators and not goofy manually indexed for loops that are begging you to make a typo that crashes your code :)
    - kllrnohj 4y ago
      Yeah but idiomatic modern C++ is also using iterators and even before that there's no bounds checking to eliminate in the first place since operator[] is unchecked so the optimizer can't be struggling to eliminate it since it's not there. The question isn't "does Rust have bad bounds checking optimizations" but rather "what is this mythical heavily-bounds-checked C code that the compiler can't optimize away?"
      - Gankra 4y ago
        No the claim is always that Rust "must" be slower than C/C++ because it has pervasive bounds checking for array indexing. Then people insist on wanting to replace every x[i] in prod with x.get_unchecked(i) only to learn that, not only was that indexing not slowing the code down (the branch is perfectly predictable in a correct program!), but actually any difference is so in the noise that the random perturbation is worse (or that the asserts were actually adding extra facts for more profitable optimizations in llvm). There is definitely specific hot loops with weird access patterns where it can be high impact but those are the exception, not the rule, as the Android team demonstrated.
  - stouset 4y ago
    Trivially, anything using the Iterator trait. I don't know that I've ever actually manually indexed an array over years of using Rust.
    - berkut 4y ago
      In the HPC / offline graphics / simulation world, there's lots of things like sparse arrays / grids, where you index into compacted grids and iterators wouldn't be that practical in that scenario (i.e. one single item, although for things like filtering with surrounding cells they can still be useful sometimes), and bounds checks do make a bit of a difference there (it's definitely measureable above the noise threshold), and due to the random nature of the data, the branch predictors don't help avoid the overhead.
      - stouset 4y ago
        Sure, I know there's paradigms where manual indexing is important. GP asked for an example where bounds-checking could be eliminated in Rust, so I gave the first one I thought of.
    - kllrnohj 4y ago
      I don't think your answer is relevant to the question: > Can you point to any such bounds check in C
  - tialaramex 4y ago
    I would guess all the aliasing stuff will get you. In C it's very difficult for the compiler to know whether two pointers are aliased, if we change X maybe Y changes too (because actually X and Y were the same). In Rust if we can write to it then it isn't aliased, and if we can't write to it then nobody can change it, thus changing X definitely can't change Y and the emitted machine code is sometimes simpler as a result, doing what you naively expected rather than what the C needs to do just in case you're crazy and there is an alias. Now, in modern C you can say you don't have aliasing, but you're probably wrong and so there's a high risk when you do that you now get "impossible" bugs because you swore to the optimiser that if X changes, Y is unaffected, then created a situation where that wasn't true and now your program has no defined meaning, which is going to be tricky to debug. So, on the whole C programmers do not use this, indeed in places like the Linux kernel they even turn off the C standard's very minimal aliasing rules (which forbid aliasing objects of different types), they just don't trust themselves.
    - kllrnohj 4y ago
      But the compiler doesn't need to care in that case because it's not bounds checking those pointers in the first place in C. So that's not going to give you slow C code from bounds checking that the optimizer failed to eliminate. Like yeah there's aliasing changes, but in "idiomatic" C/C++ how is that getting you bounds checking that's not being optimized away fairly consistently?
      - tialaramex 4y ago
        Wait, previously you were talking about bounds checks which can't be optimised out, now you seem to be saying in C you wouldn't bother writing any bounds checks, which is a quite different claim.
        Sirened 4y ago
        I think what they're saying is that C compilers don't care about aliasing here because it's not actually bounds checking the pointers, it's just a numeric comparison between two random arguments. It's much easier for an optimizing compiler to eliminate a duplicated boolean check between two numbers this way because passing numbers from one function to the next has no aliasing concerns.
masklinn 4y ago
> But if "security" isn't remotely a concern for a given project (like almost anything graphics / gaming related) Gaming platforms have gotten a lot less lenient over time, and with pretty much every game these days having online components, "security isn't remotely a concern" has become a lot less true.
- berkut 4y ago
  Sure, but then there's things like HPC / offline graphics / simulation (VFX/CG), where performance is the end-all concern (or memory efficiency sometimes at the expense of CPU time), and security isn't a concern at all there, with lots of things like random index lookups into sparse arrays / grids, etc. I know for a fact that bound checks do make a bit of a difference there, as the data's random, so the branch predictors are close to useless in that situation...
  - yosefk 4y ago
    Everybody who works with Maya, Flash (now Adobe Animate) etc. knows that they crash all the time, and often corrupt files so you back them up every hour or so. Carmack insists, in a gaming context, to run heavyweight, high-false-positive-rate static analyzers because users don't like crashes. When C++ dies (which in 20 years it will, and I wasn't that hopeful 20 years ago), people will look in the same bewilderment at excuses made for its insane behavior as they look now at mid-20th-century arguments against high-level languages (and assemblers before them - like Mel said, "you never know where it will put things so you'd have to use separate constants.")
    - Taywee 4y ago
      > which in 20 years it will, and I wasn't that hopeful 20 years ago That's too optimistic. C++ would easily die in 20 years if it didn't already have 30+ years of still-active legacy that can't easily be converted or rewritten. I've recently even had to start new projects in C++ because platforms I depend on demand it or because I have to interface with existing code and libraries that still only exist as C++. I'm not a fan of the language by any means, but I'll eat my shoe if it's "dead" in 20 years for anything except maybe greenfield development.
estebank 4y ago
It's not just about bounds checks. I am way more agressive about borrowing fields of other types, particularly mutable borrows in ways that I wouldn't attempt in C to protect myself from future code from breaking invariants I'd have to rely on. This means I can write the hyper optimized version of an algorithm on any language, but I'm more likely to even attempt it in Rust.
- masklinn 4y ago
  An other bit I first saw observed by Armin Ronacher and which is indeed quite common is that Rust does not require you to be defensive (and pay the price) around protected resources e.g. you can hand out references to rc’d types or mutex’d, and you know it’s safe, if a bit constraining. And so you save the overhead of the extra refcounting or the re-entrant locks (though that would be unsound anyway), and you can safely use anything which works off of references.
chlorion 4y ago
There were (or are?) some older call of duty games being sold on steam that had unpatched RCE vulnerabilities in them. Simply joining a server ran by a malicious host can result in the players system being totally compromised. These games would go on sale once a year or whatever and attract new players, and people would post warnings in the steam forums and whatnot to try to stop people from being effected by the issue, but I am sure some people either didn't listen or didn't notice the warnings. I haven't looked into the issue in a few years at this point, but it's very possible that the games are still unpatched and being listed in the store to this day. Anything that connects to the internet needs to be strongly concerned about security!
- kaba0 4y ago
  That’s why managed languages in itself are not enough. Security on desktop OSs are a joke, with perhaps mac being a bit ahead than the rest. But especially desktop linux.. every random bash script could encrypt your documents, or leak out your browser cache, do whatever it wants..
- HideousKojima 4y ago
  Similarly there was an RCE discovered in Dark Souls III about a year ago. And even though modders had s fix for it before the details of the exploit were even publicly revealed, it took the devs 8 months or so to finally fix it.

bluGill 4y ago

I take it the opposite: C programmers out of abundance of caution of putting in bounds checks for code that will never be called with out of bound data. As such rust is eliminating code that is being manually written. If you don't write the bounds check in C, and rust for the equivalent determines that the bounds check isn't needed the code should be the same (to the assembly level). However if you write a bounds check in C the optimizer might not eliminate it.

kllrnohj 4y ago
Can you point to any such bounds check in C that an optimizer cannot eliminate but it can eliminate the equivalent one in Rust? I'm sure it's possible to construct such a thing, but I cannot imagine it ever being common enough to show up on any sort of head to head comparison.
- Gankra 4y ago
  there's almost no bounds checking in rust code before the optimizer even looks at it because we use iterators and not goofy manually indexed for loops that are begging you to make a typo that crashes your code :)
  - kllrnohj 4y ago
    Yeah but idiomatic modern C++ is also using iterators and even before that there's no bounds checking to eliminate in the first place since operator[] is unchecked so the optimizer can't be struggling to eliminate it since it's not there. The question isn't "does Rust have bad bounds checking optimizations" but rather "what is this mythical heavily-bounds-checked C code that the compiler can't optimize away?"
    - Gankra 4y ago
      No the claim is always that Rust "must" be slower than C/C++ because it has pervasive bounds checking for array indexing. Then people insist on wanting to replace every x[i] in prod with x.get_unchecked(i) only to learn that, not only was that indexing not slowing the code down (the branch is perfectly predictable in a correct program!), but actually any difference is so in the noise that the random perturbation is worse (or that the asserts were actually adding extra facts for more profitable optimizations in llvm). There is definitely specific hot loops with weird access patterns where it can be high impact but those are the exception, not the rule, as the Android team demonstrated.
- stouset 4y ago
  Trivially, anything using the Iterator trait. I don't know that I've ever actually manually indexed an array over years of using Rust.
  - berkut 4y ago
    In the HPC / offline graphics / simulation world, there's lots of things like sparse arrays / grids, where you index into compacted grids and iterators wouldn't be that practical in that scenario (i.e. one single item, although for things like filtering with surrounding cells they can still be useful sometimes), and bounds checks do make a bit of a difference there (it's definitely measureable above the noise threshold), and due to the random nature of the data, the branch predictors don't help avoid the overhead.
    - stouset 4y ago
      Sure, I know there's paradigms where manual indexing is important. GP asked for an example where bounds-checking could be eliminated in Rust, so I gave the first one I thought of.
  - kllrnohj 4y ago
    I don't think your answer is relevant to the question: > Can you point to any such bounds check in C
- tialaramex 4y ago
  I would guess all the aliasing stuff will get you. In C it's very difficult for the compiler to know whether two pointers are aliased, if we change X maybe Y changes too (because actually X and Y were the same). In Rust if we can write to it then it isn't aliased, and if we can't write to it then nobody can change it, thus changing X definitely can't change Y and the emitted machine code is sometimes simpler as a result, doing what you naively expected rather than what the C needs to do just in case you're crazy and there is an alias. Now, in modern C you can say you don't have aliasing, but you're probably wrong and so there's a high risk when you do that you now get "impossible" bugs because you swore to the optimiser that if X changes, Y is unaffected, then created a situation where that wasn't true and now your program has no defined meaning, which is going to be tricky to debug. So, on the whole C programmers do not use this, indeed in places like the Linux kernel they even turn off the C standard's very minimal aliasing rules (which forbid aliasing objects of different types), they just don't trust themselves.
  - kllrnohj 4y ago
    But the compiler doesn't need to care in that case because it's not bounds checking those pointers in the first place in C. So that's not going to give you slow C code from bounds checking that the optimizer failed to eliminate. Like yeah there's aliasing changes, but in "idiomatic" C/C++ how is that getting you bounds checking that's not being optimized away fairly consistently?
    - tialaramex 4y ago
      Wait, previously you were talking about bounds checks which can't be optimised out, now you seem to be saying in C you wouldn't bother writing any bounds checks, which is a quite different claim.
      - Sirened 4y ago
        I think what they're saying is that C compilers don't care about aliasing here because it's not actually bounds checking the pointers, it's just a numeric comparison between two random arguments. It's much easier for an optimizing compiler to eliminate a duplicated boolean check between two numbers this way because passing numbers from one function to the next has no aliasing concerns.

masklinn 4y ago

> But if "security" isn't remotely a concern for a given project (like almost anything graphics / gaming related) Gaming platforms have gotten a lot less lenient over time, and with pretty much every game these days having online components, "security isn't remotely a concern" has become a lot less true.

berkut 4y ago
Sure, but then there's things like HPC / offline graphics / simulation (VFX/CG), where performance is the end-all concern (or memory efficiency sometimes at the expense of CPU time), and security isn't a concern at all there, with lots of things like random index lookups into sparse arrays / grids, etc. I know for a fact that bound checks do make a bit of a difference there, as the data's random, so the branch predictors are close to useless in that situation...
- yosefk 4y ago
  Everybody who works with Maya, Flash (now Adobe Animate) etc. knows that they crash all the time, and often corrupt files so you back them up every hour or so. Carmack insists, in a gaming context, to run heavyweight, high-false-positive-rate static analyzers because users don't like crashes. When C++ dies (which in 20 years it will, and I wasn't that hopeful 20 years ago), people will look in the same bewilderment at excuses made for its insane behavior as they look now at mid-20th-century arguments against high-level languages (and assemblers before them - like Mel said, "you never know where it will put things so you'd have to use separate constants.")
  - Taywee 4y ago
    > which in 20 years it will, and I wasn't that hopeful 20 years ago That's too optimistic. C++ would easily die in 20 years if it didn't already have 30+ years of still-active legacy that can't easily be converted or rewritten. I've recently even had to start new projects in C++ because platforms I depend on demand it or because I have to interface with existing code and libraries that still only exist as C++. I'm not a fan of the language by any means, but I'll eat my shoe if it's "dead" in 20 years for anything except maybe greenfield development.

estebank 4y ago

It's not just about bounds checks. I am way more agressive about borrowing fields of other types, particularly mutable borrows in ways that I wouldn't attempt in C to protect myself from future code from breaking invariants I'd have to rely on. This means I can write the hyper optimized version of an algorithm on any language, but I'm more likely to even attempt it in Rust.

masklinn 4y ago
An other bit I first saw observed by Armin Ronacher and which is indeed quite common is that Rust does not require you to be defensive (and pay the price) around protected resources e.g. you can hand out references to rc’d types or mutex’d, and you know it’s safe, if a bit constraining. And so you save the overhead of the extra refcounting or the re-entrant locks (though that would be unsound anyway), and you can safely use anything which works off of references.

chlorion 4y ago

There were (or are?) some older call of duty games being sold on steam that had unpatched RCE vulnerabilities in them. Simply joining a server ran by a malicious host can result in the players system being totally compromised. These games would go on sale once a year or whatever and attract new players, and people would post warnings in the steam forums and whatnot to try to stop people from being effected by the issue, but I am sure some people either didn't listen or didn't notice the warnings. I haven't looked into the issue in a few years at this point, but it's very possible that the games are still unpatched and being listed in the store to this day. Anything that connects to the internet needs to be strongly concerned about security!

kaba0 4y ago
That’s why managed languages in itself are not enough. Security on desktop OSs are a joke, with perhaps mac being a bit ahead than the rest. But especially desktop linux.. every random bash script could encrypt your documents, or leak out your browser cache, do whatever it wants..
HideousKojima 4y ago
Similarly there was an RCE discovered in Dark Souls III about a year ago. And even though modders had s fix for it before the details of the exploit were even publicly revealed, it took the devs 8 months or so to finally fix it.