5 ms·

I saddens me that smart people like Herb and Bjarne have good ideas on how to make C++ a much safer language but the actual output of the standard's committee i

I saddens me that smart people like Herb and Bjarne have good ideas on how to make C++ a much safer language but the actual output of the standard's committee is so far behind. Herb mentions std::span as as safety improvement but the [] operator doesn't do bounds checks and the .at() method isn't even there yet! Shameless plug but I discuss this issue on my blog: https://btmc.substack.com/p/memory-unsafety-is-an-attitude-problem https://btmc.substack.com/p/memory-unsafety-is-an-attitude-p... Herb and Bjarne have the right attitude, sadly it doesn't seem enough C++ devs or people in the standards committee do. Same applies to C.

pjmlp 3y ago
Here is Dennis Ritchie proposal for fat pointers in C. https://www.bell-labs.com/usr/dmr/www/vararray.pdf https://www.bell-labs.com/usr/dmr/www/vararray.pdf It is a culture thing, eventually the authors don't have any last words to say, if they let the community rule the language design, and their voice is equally one vote.
- sirwhinesalot 3y ago
  > This is a version of a paper published in Journal of C Language Translation, vol 2 number 2, September 1990 This says it all really. Nothing more needs to be said. Unfortunate.
  - akoboldfrying 3y ago
    I haven't read the linked paper, but both CPU speed and RAM available have increased about 100x since 1990, and nobody then had uttered the words "threat model". Some approaches that are sensible now were reasonable to overlook in 1990 for being too heavy.
    - sirwhinesalot 3y ago
      Yes but the same story kept repeating over the years. C89 had a good excuse. C99 was iffy with the VLA stuff instead of proper slices. What excuse did C11 have?
    - pjmlp 3y ago
      Check when Morris worm came out. And by the way, "A consequence of this principle is that every occurrence of every subscript of every subscripted variable was on every occasion checked at run time against both the upper and the lower declared bounds of the array. Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interests of efficiency on production runs. Unanimously, they urged us not to--they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous. I note with fear and horror that even in 1980 language designers and users have not learned this lesson. In any respectable branch of engineering, failure to observe such elementary precautions would have long been against the law." -- C.A.R Hoare's "The 1980 ACM Turing Award Lecture"
      - akoboldfrying 3y ago
        The Morris worm affected around 2000 VAX machines a couple of years previously, and was the first ever such incident on that scale. In other words, almost nobody in 1990 had been affected by a computer security incident. It didn't make sense in 1990 to prioritise this security threat over efficiency concerns. Insisting on memory safety back then would be like insisting on code being accompanied by checkable formal proofs of correctness now: It's a technique that can be applied right now and that does improve safety, but it comes at such a cost that the tradeoff only makes sense for a handful of niche applications (aerospace, automotive, medical devices).
        pjmlp 3y ago
        Yeah, that is why we didn't had to buy anti-virus software, duh.
        akoboldfrying 3y ago
        Viruses in 1990 propagated by people running .EXE files they copied from somewhere, or booting floppy disks they found somewhere. Tell me how bounds checks on array accesses would have prevented that.
GrumpySloth 3y ago
At $work, the standard solution to ASAN reporting use-after-free issues is to... not run ASAN builds. The fact that builds in CI exhibit random inexplicable crashes regularly every week doesn’t seem to make anyone have any second thoughts. A colleague once claimed that there is nothing wrong with out-of-bounds accesses as long as we don’t crash. The same bunch is also religiously opposed to using debuggers and regularly break debugger builds by removing debug information from builds at multiple levels, blocking ptrace debugging in system images through sysctl... This is all so toxic.
- sirwhinesalot 3y ago
  My personal opinion is that if companies would get real punishments for all of these goddamn security breaches, and the CTO's head was on the line, you'd see a shift in attitude real quick.
  - bluetomcat 3y ago
    Not all pieces of software are created equal. A desktop CAD application that doesn't do any networking and doesn't manipulate sensitive user data isn't worthy of binary exploitation. If there is adequate security at the system OS layer, at worst it will corrupt a user's file. Infrastructure network code that runs on millions of servers worldwide is a completely different story. Being able to send a sequence of bytes that unlocks funny side-effects of a network service can be weaponised on a mass scale.
    - sirwhinesalot 3y ago
      Those are increasingly rare. Nowadays you have all these apps requiring subscriptions and expecting users to login and what not. But I agree it depends heavily on exactly what application are we talking about. Is it running on server? Definitely needs to be security conscious. Is it a library that might at some point be used by an application running on a server? Needs to be more hardened than a nuclear bunker.
      - 3y ago
gpderetta 3y ago
Even if not mandated by the standard, concrete standard library implementations do provide bound checking on span (and vector, and optional, etc.), but, even when meant for production use, are disabled by default. And I don't see a big push in the community to enable them. I think the committee is just an expression of the community on this front.
- account42 3y ago
  > Even if not mandated by the standard, concrete standard library implementations do provide bound checking on span (and vector, and optional, etc.), but, even when meant for production use, are disabled by default. That's a choice though. You can enable these in your production builds if you want (with libstdc++ at least) and some Linux distributions have chosen to do just that. The thing though is that these checks are NOT free and the overhead is not justified for all use cases so forcing them on everyone is not appropriate for C++.
  - gpderetta 3y ago
    > The thing though is that these checks are NOT free and the overhead is not justified for all use cases so forcing them on everyone is not appropriate for C++. Well, that's why they should be a flag. The question is whether it should be enabled by default or not.
    - fanf2 3y ago
      It should be enabled by default, and if you want to index without bounds checking you should have to write something like a.unsafe_at(i)
jasode 3y ago
> but the actual output of the standard's committee is so far behind. That criticism misunderstands the actual way the C++ committee works. They're not a supreme legislative group that can dictate what a new C++ should be and everybody else is required to just obey. In contrast, Apple can dictate what the next version Swift will do. Microsoft can do the same with C#. Instead, what happens in C++ is somebody or company makes a proposal with some concrete working code for others to evaluate and vote "yes" on. So the reality means one of the teams from MSVC, gcc, clang, Intel C++, etc have to take the lead with a Safe C++ alternative that convinces everybody else to implement. To Herb Sutter's credit, he did make a "C++2" implementation: https://github.com/hsutter/cppfront https://github.com/hsutter/cppfront But his side project at Microsoft didn't gain traction with gcc, clang, etc and everybody else in the industry. So at this point, the C++ committee will be perceived as "so far behind" ... because there's nothing for them to vote on. Similar situation happened with "breaking ABI compatibility". Google wants to break ABI but others didn't.
- pjmlp 3y ago
  And that is why while C++26 is being discussed, C++20 modules are still full of warts, working on Visual C++ (kind of), and not really anywhere else, as ISO C++ has long stop being about standardizing stuff with actual field experience.
  - jasode 3y ago
    >And that is why [...] C++20 modules are still full of warts, working on Visual C++ (kind of), and not really anywhere else, as ISO C++ has long stop being about standardizing stuff with actual field experience. Yes, but your complaint about flawed C++ standards or incomplete implementations is orthogonal to what I was writing about. I'm just explaining that the C++ committee doesn't have the power to impose changes that some people think it does. Basically, I'm saying "the sky is blue" type of statement. The C++ committee is a reflection of what the members' compiler teams want to collectively do. The committee doesn't have unilateral supreme power to dictate standards to compiler teams not interested in it. Instead, they collect feedback from people sharing proposals in papers and put things to a vote. The compilers' teams have the real power and not the committee. (What I've stated in this paragraph should be uncontroversial facts but the downvoters disagree so I'd love to hear them explain exactly what power and agency they think the C++ committee actually has.) If one understand the above situation, then the following different situations shouldn't have any mystery as to cause & effect: - How did the std::chrono get added to the standard library? Because Howard Hinnant made a proposal with working code, and others liked it enough to vote it in. - Why is there's no _standard_ cross-platform GUI and audio library in C++? Why is there not standardized Safe Syntax C++ like Rust? Because nobody has made a proposal that convinced enough of the other members to agree to a x-platform GUI and audio framework. - Why does C++ committee adds features and prioritizes things I don't care about? Because the <$feature$> you cared about wasn't proposed for them to discuss and convince others enough to vote it in. But yes I do understand the "warts" complaint you're talking about. It's frustrating that the std:regex with bad performance got approved. In similar examples, N Josuttis in multiple conference videos has complained about the problems with ranges and views. He says it was wrong for the C++ committee to approve it. (footnote: The original author of the proposal tried to explain his technical rationale: https://old.reddit.com/r/cpp/comments/zq2xdi/are_there_likely_to_be_any_changes_to_views_to/ https://old.reddit.com/r/cpp/comments/zq2xdi/are_there_likel... ) To reiterate, I'm not trying to explain away bad language standards. New features that will have flaws will continue to happen in the future whether it's created by a singular corporation like Apple(Swift) or a cooperative group like the C++ committee. I'm just explaining why some "wishlist desirable C++ feature" isn't going to be in the standard if there's no proposal that convinces the other members to vote it in. EDIT to reply: >When we complain about the "committee" [...] the things they choose to propose and vote for. The C++ committee members are not static but the webpage has list of names : https://isocpp.org/wiki/faq/wg21 https://isocpp.org/wiki/faq/wg21 Clicking on various PnnnnR.pdf proposals that motivated each feature in the conformance table shows most authors are not from the actual committee members: https://en.cppreference.com/w/cpp/compiler_support https://en.cppreference.com/w/cpp/compiler_support Using the above workflow to address your complaint about std::span and at(), I found this comment from the original author Tristan Brindle who proposed it and why he thinks the committee voted no: 2019-10-18T22:55:30z https://old.reddit.com/r/cpp/comments/djqdu2/why_is_stdspan_is_missing_at/f48hhb5/ https://old.reddit.com/r/cpp/comments/djqdu2/why_is_stdspan_...
humanrebar 3y ago
> ...the actual output of the standard's committee is so far behind The committee just publishes documents. It is actually far ahead of C++ implementations. The committee would probably move faster if there were more attention (and funds and volunteer work) spent on advancing C++ implementations. This especially seems true for safety and security concerns as they tend to have more tangible problems to solve than the other kinds of standards proposals.
- sirwhinesalot 3y ago
  You're not far ahead if you're running in the wrong direction. The standard is prioritizing the wrong things. It's normal that implementations are struggling when they need to implement something as complicated as C++ modules for example. There's no excuse for the .at() method being missing from std::span. On the C side of things the problem is more egregious, it took over 30 years to standardize typeof after every compiler ever had already implemented something of the sort. GCC's __attribute__((cleanup)) should have definitely been standardized ages ago with so many libraries and codebases relying on it. What does the C standard give us instead? _Generic. It's just silly at this point.
  - gpderetta 3y ago
    > There's no excuse for the .at() method being missing from std::span. The issue is that there are two camps. One believes that precondition failures should not be recoverable and should abort the application and thus think that 'at' is an abomination. The other believes that throwing exceptions on the face of precondition failure is appropriate. Hence what goes into the standard depends on how many people on each camp are present at each committee vote. This is also one of the reasons why the contracts proposal is not yet in the standard. On a more practical side, .at does not help in any way to bound check the hundreds of billions of existing lines of C++.
    - sirwhinesalot 3y ago
      std::span came out in C++20, by that logic neither did it help in any way... Personally operator [] should abort by default because otherwise it is redundant with .at()
stefanos82 3y ago
> Herb mentions std::span as as safety improvement but the [] operator doesn't do bounds checks and the .at() method isn't even there yet! You mean this implementation? https://en.cppreference.com/w/cpp/container/span/at https://en.cppreference.com/w/cpp/container/span/at To quote: "Returns a reference to the element at specified location pos, with bounds checking. If pos is not within the range of the span, an exception of type std::out_of_range is thrown."
- pjmlp 3y ago
  You missed the Std column => (C++26)
AlbertoGP 3y ago
> Shameless plug but I discuss this issue on my blog: > “First make them care, then make it easy for them to do the right thing.” I would say first make it easy to do the right thing, making them care will be an Eternal September.
- sirwhinesalot 3y ago
  Sadly even if you make it easy to do the right thing, without the right attitude to match it matters little. Some people still concatenate unsanitized input with raw SQL strings despite the abundance of libraries that make creating safe queries easier. At this point I think heads need to roll before people take the problem seriously.
  - AlbertoGP 3y ago
    I wasn’t clear: we do agree that attitude is the crucial piece, I just disagreed on the order in which it should be done. I think that implementing in compilers the mechanisms for doing the right thing can be done first, and relatively quickly. Which now that I think more about it, would need the right attitude on the part of the compiler vendors and standards committee.
pptr 3y ago
Being able to use a foreach loops is a decent improvement. And since you now have a proper access API, you could on theory enable bounds checks via compiler flag, even for operator[].

pjmlp 3y ago

Here is Dennis Ritchie proposal for fat pointers in C. https://www.bell-labs.com/usr/dmr/www/vararray.pdf https://www.bell-labs.com/usr/dmr/www/vararray.pdf It is a culture thing, eventually the authors don't have any last words to say, if they let the community rule the language design, and their voice is equally one vote.

sirwhinesalot 3y ago
> This is a version of a paper published in Journal of C Language Translation, vol 2 number 2, September 1990 This says it all really. Nothing more needs to be said. Unfortunate.
- akoboldfrying 3y ago
  I haven't read the linked paper, but both CPU speed and RAM available have increased about 100x since 1990, and nobody then had uttered the words "threat model". Some approaches that are sensible now were reasonable to overlook in 1990 for being too heavy.
  - sirwhinesalot 3y ago
    Yes but the same story kept repeating over the years. C89 had a good excuse. C99 was iffy with the VLA stuff instead of proper slices. What excuse did C11 have?
  - pjmlp 3y ago
    Check when Morris worm came out. And by the way, "A consequence of this principle is that every occurrence of every subscript of every subscripted variable was on every occasion checked at run time against both the upper and the lower declared bounds of the array. Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interests of efficiency on production runs. Unanimously, they urged us not to--they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous. I note with fear and horror that even in 1980 language designers and users have not learned this lesson. In any respectable branch of engineering, failure to observe such elementary precautions would have long been against the law." -- C.A.R Hoare's "The 1980 ACM Turing Award Lecture"
    - akoboldfrying 3y ago
      The Morris worm affected around 2000 VAX machines a couple of years previously, and was the first ever such incident on that scale. In other words, almost nobody in 1990 had been affected by a computer security incident. It didn't make sense in 1990 to prioritise this security threat over efficiency concerns. Insisting on memory safety back then would be like insisting on code being accompanied by checkable formal proofs of correctness now: It's a technique that can be applied right now and that does improve safety, but it comes at such a cost that the tradeoff only makes sense for a handful of niche applications (aerospace, automotive, medical devices).
      - pjmlp 3y ago
        Yeah, that is why we didn't had to buy anti-virus software, duh.
        akoboldfrying 3y ago
        Viruses in 1990 propagated by people running .EXE files they copied from somewhere, or booting floppy disks they found somewhere. Tell me how bounds checks on array accesses would have prevented that.

GrumpySloth 3y ago

At $work, the standard solution to ASAN reporting use-after-free issues is to... not run ASAN builds. The fact that builds in CI exhibit random inexplicable crashes regularly every week doesn’t seem to make anyone have any second thoughts. A colleague once claimed that there is nothing wrong with out-of-bounds accesses as long as we don’t crash. The same bunch is also religiously opposed to using debuggers and regularly break debugger builds by removing debug information from builds at multiple levels, blocking ptrace debugging in system images through sysctl... This is all so toxic.

sirwhinesalot 3y ago
My personal opinion is that if companies would get real punishments for all of these goddamn security breaches, and the CTO's head was on the line, you'd see a shift in attitude real quick.
- bluetomcat 3y ago
  Not all pieces of software are created equal. A desktop CAD application that doesn't do any networking and doesn't manipulate sensitive user data isn't worthy of binary exploitation. If there is adequate security at the system OS layer, at worst it will corrupt a user's file. Infrastructure network code that runs on millions of servers worldwide is a completely different story. Being able to send a sequence of bytes that unlocks funny side-effects of a network service can be weaponised on a mass scale.
  - sirwhinesalot 3y ago
    Those are increasingly rare. Nowadays you have all these apps requiring subscriptions and expecting users to login and what not. But I agree it depends heavily on exactly what application are we talking about. Is it running on server? Definitely needs to be security conscious. Is it a library that might at some point be used by an application running on a server? Needs to be more hardened than a nuclear bunker.
    - 3y ago

gpderetta 3y ago

Even if not mandated by the standard, concrete standard library implementations do provide bound checking on span (and vector, and optional, etc.), but, even when meant for production use, are disabled by default. And I don't see a big push in the community to enable them. I think the committee is just an expression of the community on this front.

account42 3y ago
> Even if not mandated by the standard, concrete standard library implementations do provide bound checking on span (and vector, and optional, etc.), but, even when meant for production use, are disabled by default. That's a choice though. You can enable these in your production builds if you want (with libstdc++ at least) and some Linux distributions have chosen to do just that. The thing though is that these checks are NOT free and the overhead is not justified for all use cases so forcing them on everyone is not appropriate for C++.
- gpderetta 3y ago
  > The thing though is that these checks are NOT free and the overhead is not justified for all use cases so forcing them on everyone is not appropriate for C++. Well, that's why they should be a flag. The question is whether it should be enabled by default or not.
  - fanf2 3y ago
    It should be enabled by default, and if you want to index without bounds checking you should have to write something like a.unsafe_at(i)

jasode 3y ago

> but the actual output of the standard's committee is so far behind. That criticism misunderstands the actual way the C++ committee works. They're not a supreme legislative group that can dictate what a new C++ should be and everybody else is required to just obey. In contrast, Apple can dictate what the next version Swift will do. Microsoft can do the same with C#. Instead, what happens in C++ is somebody or company makes a proposal with some concrete working code for others to evaluate and vote "yes" on. So the reality means one of the teams from MSVC, gcc, clang, Intel C++, etc have to take the lead with a Safe C++ alternative that convinces everybody else to implement. To Herb Sutter's credit, he did make a "C++2" implementation: https://github.com/hsutter/cppfront https://github.com/hsutter/cppfront But his side project at Microsoft didn't gain traction with gcc, clang, etc and everybody else in the industry. So at this point, the C++ committee will be perceived as "so far behind" ... because there's nothing for them to vote on. Similar situation happened with "breaking ABI compatibility". Google wants to break ABI but others didn't.

pjmlp 3y ago
And that is why while C++26 is being discussed, C++20 modules are still full of warts, working on Visual C++ (kind of), and not really anywhere else, as ISO C++ has long stop being about standardizing stuff with actual field experience.
- jasode 3y ago
  >And that is why [...] C++20 modules are still full of warts, working on Visual C++ (kind of), and not really anywhere else, as ISO C++ has long stop being about standardizing stuff with actual field experience. Yes, but your complaint about flawed C++ standards or incomplete implementations is orthogonal to what I was writing about. I'm just explaining that the C++ committee doesn't have the power to impose changes that some people think it does. Basically, I'm saying "the sky is blue" type of statement. The C++ committee is a reflection of what the members' compiler teams want to collectively do. The committee doesn't have unilateral supreme power to dictate standards to compiler teams not interested in it. Instead, they collect feedback from people sharing proposals in papers and put things to a vote. The compilers' teams have the real power and not the committee. (What I've stated in this paragraph should be uncontroversial facts but the downvoters disagree so I'd love to hear them explain exactly what power and agency they think the C++ committee actually has.) If one understand the above situation, then the following different situations shouldn't have any mystery as to cause & effect: - How did the std::chrono get added to the standard library? Because Howard Hinnant made a proposal with working code, and others liked it enough to vote it in. - Why is there's no _standard_ cross-platform GUI and audio library in C++? Why is there not standardized Safe Syntax C++ like Rust? Because nobody has made a proposal that convinced enough of the other members to agree to a x-platform GUI and audio framework. - Why does C++ committee adds features and prioritizes things I don't care about? Because the <$feature$> you cared about wasn't proposed for them to discuss and convince others enough to vote it in. But yes I do understand the "warts" complaint you're talking about. It's frustrating that the std:regex with bad performance got approved. In similar examples, N Josuttis in multiple conference videos has complained about the problems with ranges and views. He says it was wrong for the C++ committee to approve it. (footnote: The original author of the proposal tried to explain his technical rationale: https://old.reddit.com/r/cpp/comments/zq2xdi/are_there_likely_to_be_any_changes_to_views_to/ https://old.reddit.com/r/cpp/comments/zq2xdi/are_there_likel... ) To reiterate, I'm not trying to explain away bad language standards. New features that will have flaws will continue to happen in the future whether it's created by a singular corporation like Apple(Swift) or a cooperative group like the C++ committee. I'm just explaining why some "wishlist desirable C++ feature" isn't going to be in the standard if there's no proposal that convinces the other members to vote it in. EDIT to reply: >When we complain about the "committee" [...] the things they choose to propose and vote for. The C++ committee members are not static but the webpage has list of names : https://isocpp.org/wiki/faq/wg21 https://isocpp.org/wiki/faq/wg21 Clicking on various PnnnnR.pdf proposals that motivated each feature in the conformance table shows most authors are not from the actual committee members: https://en.cppreference.com/w/cpp/compiler_support https://en.cppreference.com/w/cpp/compiler_support Using the above workflow to address your complaint about std::span and at(), I found this comment from the original author Tristan Brindle who proposed it and why he thinks the committee voted no: 2019-10-18T22:55:30z https://old.reddit.com/r/cpp/comments/djqdu2/why_is_stdspan_is_missing_at/f48hhb5/ https://old.reddit.com/r/cpp/comments/djqdu2/why_is_stdspan_...

humanrebar 3y ago

> ...the actual output of the standard's committee is so far behind The committee just publishes documents. It is actually far ahead of C++ implementations. The committee would probably move faster if there were more attention (and funds and volunteer work) spent on advancing C++ implementations. This especially seems true for safety and security concerns as they tend to have more tangible problems to solve than the other kinds of standards proposals.

sirwhinesalot 3y ago
You're not far ahead if you're running in the wrong direction. The standard is prioritizing the wrong things. It's normal that implementations are struggling when they need to implement something as complicated as C++ modules for example. There's no excuse for the .at() method being missing from std::span. On the C side of things the problem is more egregious, it took over 30 years to standardize typeof after every compiler ever had already implemented something of the sort. GCC's __attribute__((cleanup)) should have definitely been standardized ages ago with so many libraries and codebases relying on it. What does the C standard give us instead? _Generic. It's just silly at this point.
- gpderetta 3y ago
  > There's no excuse for the .at() method being missing from std::span. The issue is that there are two camps. One believes that precondition failures should not be recoverable and should abort the application and thus think that 'at' is an abomination. The other believes that throwing exceptions on the face of precondition failure is appropriate. Hence what goes into the standard depends on how many people on each camp are present at each committee vote. This is also one of the reasons why the contracts proposal is not yet in the standard. On a more practical side, .at does not help in any way to bound check the hundreds of billions of existing lines of C++.
  - sirwhinesalot 3y ago
    std::span came out in C++20, by that logic neither did it help in any way... Personally operator [] should abort by default because otherwise it is redundant with .at()

stefanos82 3y ago

> Herb mentions std::span as as safety improvement but the [] operator doesn't do bounds checks and the .at() method isn't even there yet! You mean this implementation? https://en.cppreference.com/w/cpp/container/span/at https://en.cppreference.com/w/cpp/container/span/at To quote: "Returns a reference to the element at specified location pos, with bounds checking. If pos is not within the range of the span, an exception of type std::out_of_range is thrown."

pjmlp 3y ago
You missed the Std column => (C++26)

AlbertoGP 3y ago

> Shameless plug but I discuss this issue on my blog: > “First make them care, then make it easy for them to do the right thing.” I would say first make it easy to do the right thing, making them care will be an Eternal September.

sirwhinesalot 3y ago
Sadly even if you make it easy to do the right thing, without the right attitude to match it matters little. Some people still concatenate unsanitized input with raw SQL strings despite the abundance of libraries that make creating safe queries easier. At this point I think heads need to roll before people take the problem seriously.
- AlbertoGP 3y ago
  I wasn’t clear: we do agree that attitude is the crucial piece, I just disagreed on the order in which it should be done. I think that implementing in compilers the mechanisms for doing the right thing can be done first, and relatively quickly. Which now that I think more about it, would need the right attitude on the part of the compiler vendors and standards committee.

pptr 3y ago

Being able to use a foreach loops is a decent improvement. And since you now have a proper access API, you could on theory enable bounds checks via compiler flag, even for operator[].