5 ms·

> The profiles technology isn't very good. Can you be very specific about why? Here's the argument for why profiles might work: with all of the profiles enabl

> The profiles technology isn't very good. Can you be very specific about why? Here's the argument for why profiles might work: with all of the profiles enabled, you are only allowed to use the safe subset of C++ and all of the unsafe stuff is hidden behind APIs whose implementations don't have those profiles enabled. Those projects that enable all profiles by default effectively get Swift-like or Rust-like protection. Like, you could force all array operations to use C++ stdlib primitives, enable full hardening of the stdlib, and then have bounds safety. And you could force all lifetime operations to use C++ stdlib refcounting primitives, and then have lifetime safety in a Swift-like way (i.e. eager refcounting everywhere). I can imagine how this falls over but then it might just be a matter of engineering to make it not fall over. (I'm playing devils advocate a bit since I prefer Fil-C++.)

seanbax 1y ago
I illustrate why it won't work with a number of examples here: https://www.circle-lang.org/draft-profiles.html https://www.circle-lang.org/draft-profiles.html To address your points: 1. The safe subset of C++ is too small to do anything with. 2. The Standard Library is not written in the safe subset. My favorite example from the above paper is the problem of std::sort -- the compiler has no idea if both operands are iterators into the same allocation. The function is fundamentally unsafe. Which C++ profile do you turn on to make that safe? Does it ban use of std::sort? Does it ban use of all <algorithms>, all of which work on pointers/iterators that are susceptible to use-after-free UB? The whole Standard Library is unsafe. I proposed a rigorously safe std2, and that was rejected. And now you propose a safe std2 (using refcounting primitives)--why would that fare better? What does Profiles actually propose? No change in existing code. The compiler simply finds all UB. Right.
IAmLiterallyAB 1y ago
My limited understanding is. There is no safe subset (That's what was just discontinued, profiles are the alternative.) And C++ code simply doesn't have the necessary info to make safety decisions. Sean explains it better than I can https://www.circle-lang.org/draft-profiles.html https://www.circle-lang.org/draft-profiles.html
- jmull 1y ago
  The analysis you link to is insufficient. E.g., the first case is "Inferring aliasing". He presents some examples and states, "The compiler cannot infer a function’s aliasing requirements from its declaration or even from its definition." But why not? The aliasing requirements come directly from vector. If the compiler has those then determining the aliasing requirements of those functions is straightforward. Now, maybe there is some argument that a C++ compiler cannot determine the aliasing requirements of vector, but if that's the claim, then the paper should make it, and back it up. The paper continues in the same vein in the next section, as if the lifetime requirements of map and min cannot be known or cannot bubble up through the functions that call them. As written, the paper says almost nothing about the feasibility of static analysis of C++ to achieve safety goals for C++.
  - seanbax 1y ago
    The Profiles authors are the ones claiming this uses local analysis only: https://news.ycombinator.com/item?id=41942126 https://news.ycombinator.com/item?id=41942126 They are clear that Profiles infers everything from function types and not function bodies. Obviously that won't work, but that's what they say.
    - jmull 1y ago
      In that post (I think your own?) it says, "Local analysis only. It's not looking in function definitions." But local analysis means analysis of function definitions. At least it does to me. I can't think of what else it could mean. I think there must be some aspect of people talking past each other here, using the same words to mean different things. Further, I don't think local analysis of the code comprising a function means throwing away the results of that analysis rather than passing it up the line to the analysis of callers of the function. E.g., local analysis of std::sort would establish its aliasing limitations, which would be available to analysis of the body of "f1" from the example in the paper (the results of which, in turn, would be available to callers of f1). Now, I don't know if that's actually feasible/workable without the "heavy" annotation that C++ profiles wants to forbid. That's the key question to me.
AlotOfReading 1y ago
There'd be less opposition if profiles worked that way. The real goal is to define a subset that excludes 95% of the unsafe stuff, as opposed to providing hard guarantees.
coffeeaddict1 1y ago
> And you could force all lifetime operations to use C++ stdlib refcounting primitives, and then have lifetime safety in a Swift-like way (i.e. eager refcounting everywhere) That's going to be a non-starter for 99% of serious C++ projects there. The performance hit is going to be way too large. For bounds checking, sure I think the performance penalty is so small that it can be done.
- pizlonator 1y ago
  That would have been my first guess but WebKit's experience doing exactly this is the opposite. See https://www.youtube.com/watch?v=RLw13wLM5Ko https://www.youtube.com/watch?v=RLw13wLM5Ko Note that they also allowed other kinds of pointers so long as their use could be statically verified using very simple rules.
- gmueckl 1y ago
  You have to realize that the number of locations in code where reference counter adjustment is actually meaningful is rather small and there are simple rules to keep the excess thrash from reference counting pointer wrappers to a minimum. The main one, as mentioned in the talk the sibling comment called out, is that it is OK to pass a raw pointer or reference to a function while holding on to a reference count for as long as that other function runs (and doesn't leak the pointer through a side effect). This rule catches a lot of pointless counter arithmetic through excessive pointer wrapper copying.
  - silon42 1y ago
    Maybe C++ should copy some Swift, before attempting to challenge Rust.
    - pjmlp 1y ago
      It has already multiple times, Managed C++, C++/CLI, C++/CX, C++ Builder, Unreal C++ But those aren't extensions or approaches WG21 cares about having. The C++11 GC design didn't even took those experiences into consideration, thus it got zero adoption, and was dropped on C++20.
steveklabnik 1y ago
> with all of the profiles enabled, you are only allowed to use the safe subset of C++ and all of the unsafe stuff is hidden behind APIs whose implementations don't have those profiles enabled. This is not the goal of profiles. It’s to be “good enough.” Guaranteed safety isn’t in the cards.
- pizlonator 1y ago
  > This is not the goal of profiles. It’s to be “good enough.” Guaranteed safety isn’t in the cards. - Rust isn’t totally guaranteed safe since folks can and do use unsafe code. - Exact same situation in Swift - Go has escape hatches like it you race, but not only. So most “safe” things are really “safe enough” for some definition of “enough”.
  - AlotOfReading 1y ago
    The language standard assumes that everyone collectively agrees to standard semantics implying certain things. If users don't follow the rules and write something without semantics (undefined behavior), the entire program is meaningless as opposed to just the bit around the violation. You know this, so I emphasize it here because it's entirely incompatible with the view that "good enough" is a meaningful concept to discuss from the PoV of the standard. Rust does a pretty good job formalizing what the safety guarantees are and when you can assume them. Other languages don't, but they also don't support safety concepts that C++ nominally does like safety critical systems. "Good enough" can be perfectly fine for a web service like Go while being grossly inadequate for HPC or safety critical.
  - steveklabnik 1y ago
    You’re misunderstanding what I’m saying. Safe Rust guarantees memory safety. Profiles do not. This is regardless of the ability of the unchecked versions, on both sides, to introduce issues. Profiles do not, even for code that is 100% using profiles, guarantee safety.
    - pizlonator 1y ago
tialaramex 1y ago
Yes I can be specific. Firstly, you need composition. Rust's safety composes. The safe Rust library for farm animals from Geoff, the safe Rust library for cooking recipes by Alice and the safe Rust library for web server by Bert together with my safe program code adds up to my safe Rust farm foods web site. By having N profiles, where N is intended to be at least five and might grow arbitrarily and be user extensible, C++ guarantees it cannot deliver composition this way. Maybe they can define some sort of composition and maybe everybody will ship software which conforms to that definition and so eventually they get composition, that's not there today, so it's just a giant unknown at best. Secondly, of the profiles described so far, most of them are just solving parts of the single overarching problem Rust addresses, for the serial case. So if they ship that, which already involves some amount of new work yet to be finished, you need all of those profiles to get to only partial memory safety. Which comes to the third part. Once you start down this path, as they found, you realise you actually want a borrowck. You won't call it that of course, because that would be embarrassing. But you'll need to track reference lifetimes and you'll need annotation and you end up building most of the stuff you insisted you didn't want. For now, you can handwave, this is an unsolved static analysis problem. Well, not so much unsolved as you know the solution and you don't like it. Your idea to do the reference counting everywhere is not something WG21 has looked at, I think the perf cost is sufficiently bad that they won't even glance at it. They're also not going to ship a GC. Finally though, C++ is a concurrent language. It has a whole memory model which doesn't even make sense if you aren't thinking about concurrency. But to deliver concurrent memory safety without Fil-C's overheads you would want... well, Rust's Send and Sync traits, which sure enough have eerie twins in the Safe C++ proposal. No attempt to solve this is even hinted at in the current profiles proposal, and they would need to work one out and if it's not Send + Sync again they'd need to prove it is correct.
- pizlonator 1y ago
  I think the point is that folks will incrementally move their code towards having all profiles enabled, and that's sort of fundamental if the goal is to give folks with C++ codebases an incremental path to safety. So I don't buy your first and second points. > Which comes to the third part. Once you start down this path, as they found, you realise you actually want a borrowck. That's a bold statement. It might be true for some very loose definition of "borrow checker". See the super simple static analysis that WebKit uses (that presentation is now linked in at least two places on this HN discussion, so I won't link it again). > Your idea to do the reference counting everywhere is not something WG21 has looked at, I think the perf cost is sufficiently bad that they won't even glance at it. They're also not going to ship a GC. The point isn't to have ref counting on every pointer at the language level, but rather: if your prevent folks from calling `delete` directly (as one of the profiles does) then you're effectively forcing folks to use smart pointers. Reference counting that happens by smart pointers is something that they would ship. We know this because it's already happened. I imagine this would really mean that some references are ref counted (if you use shared_ptr or similar) while other references use some other policy. > Finally though, C++ is a concurrent language. It has a whole memory model which doesn't even make sense if you aren't thinking about concurrency. But to deliver concurrent memory safety without Fil-C's overheads you would want... well, Rust's Send and Sync traits Yeah, this might be an area where they leave a hole. Like, you might have reference counting that is only partially thread safe: - The refcount of any object is atomic. - The smart pointer itself is racy. So, racing on pointers can pop the protections. If they got that far, then that wouldn't be so bad. The marginal safety advantage of Rust would be very slim at that point.
Rusky 1y ago
If that is what profiles were actually doing, it would probably make sense. But it's not what profiles are doing. Instead, for example, the lifetime safety profile (https://github.com/isocpp/CppCoreGuidelines/blob/master/docs/Lifetime.pdf https://github.com/isocpp/CppCoreGuidelines/blob/master/docs...) is a Rust-like compile time borrow checker that relies on annotations like [[clang::lifetimebound]], yet they also repeatedly insist that profiles will not require this kind of annotation (see the papers linked from https://www.circle-lang.org/draft-profiles.html#abstract https://www.circle-lang.org/draft-profiles.html#abstract). Their messaging is just not consistent with the concrete proposals they have described, let alone actually implemented.
- pjmlp 1y ago
  Additionally they ignore field experience, I can tell that on VC++ the lifetime checker only has worked in small examples, as I was really into trying it out. Microsoft even has blog posts admitting that only with SAL like annotations it can be improved, while keeping the usual C++ semantics. Yet WG21 has ignored this field experience.
dminik 1y ago
As far as I'm concerned, there are two main issues with profiles: 1. They're either unimplementable or useless (too many false positives and false negatives). I think this is pretty evident based on the fact that profiles have been proposed for a while and that no real implementation exists. Worse, out of all of the open source projects and for profit companies, noone has been able to implement any sort of static analysis that would even begin to approach the guarantees Rust makes. 2. The language doesn't give you any tools to actually write safe code. Ok, let's say that someone actually implements safety profiles. And it highlights your usage of a standard library type. What do you do? Safe C++ didn't require a new standard library just because. The current stdlib is riddled with safety issues that can't really be fixed and would not be fixed because of backwards compatibility. You're stuck. And so you turn the safety profile off.

seanbax 1y ago

I illustrate why it won't work with a number of examples here: https://www.circle-lang.org/draft-profiles.html https://www.circle-lang.org/draft-profiles.html To address your points: 1. The safe subset of C++ is too small to do anything with. 2. The Standard Library is not written in the safe subset. My favorite example from the above paper is the problem of std::sort -- the compiler has no idea if both operands are iterators into the same allocation. The function is fundamentally unsafe. Which C++ profile do you turn on to make that safe? Does it ban use of std::sort? Does it ban use of all <algorithms>, all of which work on pointers/iterators that are susceptible to use-after-free UB? The whole Standard Library is unsafe. I proposed a rigorously safe std2, and that was rejected. And now you propose a safe std2 (using refcounting primitives)--why would that fare better? What does Profiles actually propose? No change in existing code. The compiler simply finds all UB. Right.

IAmLiterallyAB 1y ago

My limited understanding is. There is no safe subset (That's what was just discontinued, profiles are the alternative.) And C++ code simply doesn't have the necessary info to make safety decisions. Sean explains it better than I can https://www.circle-lang.org/draft-profiles.html https://www.circle-lang.org/draft-profiles.html

jmull 1y ago
The analysis you link to is insufficient. E.g., the first case is "Inferring aliasing". He presents some examples and states, "The compiler cannot infer a function’s aliasing requirements from its declaration or even from its definition." But why not? The aliasing requirements come directly from vector. If the compiler has those then determining the aliasing requirements of those functions is straightforward. Now, maybe there is some argument that a C++ compiler cannot determine the aliasing requirements of vector, but if that's the claim, then the paper should make it, and back it up. The paper continues in the same vein in the next section, as if the lifetime requirements of map and min cannot be known or cannot bubble up through the functions that call them. As written, the paper says almost nothing about the feasibility of static analysis of C++ to achieve safety goals for C++.
- seanbax 1y ago
  The Profiles authors are the ones claiming this uses local analysis only: https://news.ycombinator.com/item?id=41942126 https://news.ycombinator.com/item?id=41942126 They are clear that Profiles infers everything from function types and not function bodies. Obviously that won't work, but that's what they say.
  - jmull 1y ago
    In that post (I think your own?) it says, "Local analysis only. It's not looking in function definitions." But local analysis means analysis of function definitions. At least it does to me. I can't think of what else it could mean. I think there must be some aspect of people talking past each other here, using the same words to mean different things. Further, I don't think local analysis of the code comprising a function means throwing away the results of that analysis rather than passing it up the line to the analysis of callers of the function. E.g., local analysis of std::sort would establish its aliasing limitations, which would be available to analysis of the body of "f1" from the example in the paper (the results of which, in turn, would be available to callers of f1). Now, I don't know if that's actually feasible/workable without the "heavy" annotation that C++ profiles wants to forbid. That's the key question to me.

AlotOfReading 1y ago

There'd be less opposition if profiles worked that way. The real goal is to define a subset that excludes 95% of the unsafe stuff, as opposed to providing hard guarantees.

coffeeaddict1 1y ago

> And you could force all lifetime operations to use C++ stdlib refcounting primitives, and then have lifetime safety in a Swift-like way (i.e. eager refcounting everywhere) That's going to be a non-starter for 99% of serious C++ projects there. The performance hit is going to be way too large. For bounds checking, sure I think the performance penalty is so small that it can be done.

pizlonator 1y ago
That would have been my first guess but WebKit's experience doing exactly this is the opposite. See https://www.youtube.com/watch?v=RLw13wLM5Ko https://www.youtube.com/watch?v=RLw13wLM5Ko Note that they also allowed other kinds of pointers so long as their use could be statically verified using very simple rules.
gmueckl 1y ago
You have to realize that the number of locations in code where reference counter adjustment is actually meaningful is rather small and there are simple rules to keep the excess thrash from reference counting pointer wrappers to a minimum. The main one, as mentioned in the talk the sibling comment called out, is that it is OK to pass a raw pointer or reference to a function while holding on to a reference count for as long as that other function runs (and doesn't leak the pointer through a side effect). This rule catches a lot of pointless counter arithmetic through excessive pointer wrapper copying.
- silon42 1y ago
  Maybe C++ should copy some Swift, before attempting to challenge Rust.
  - pjmlp 1y ago
    It has already multiple times, Managed C++, C++/CLI, C++/CX, C++ Builder, Unreal C++ But those aren't extensions or approaches WG21 cares about having. The C++11 GC design didn't even took those experiences into consideration, thus it got zero adoption, and was dropped on C++20.

steveklabnik 1y ago

> with all of the profiles enabled, you are only allowed to use the safe subset of C++ and all of the unsafe stuff is hidden behind APIs whose implementations don't have those profiles enabled. This is not the goal of profiles. It’s to be “good enough.” Guaranteed safety isn’t in the cards.

pizlonator 1y ago
> This is not the goal of profiles. It’s to be “good enough.” Guaranteed safety isn’t in the cards. - Rust isn’t totally guaranteed safe since folks can and do use unsafe code. - Exact same situation in Swift - Go has escape hatches like it you race, but not only. So most “safe” things are really “safe enough” for some definition of “enough”.
- AlotOfReading 1y ago
  The language standard assumes that everyone collectively agrees to standard semantics implying certain things. If users don't follow the rules and write something without semantics (undefined behavior), the entire program is meaningless as opposed to just the bit around the violation. You know this, so I emphasize it here because it's entirely incompatible with the view that "good enough" is a meaningful concept to discuss from the PoV of the standard. Rust does a pretty good job formalizing what the safety guarantees are and when you can assume them. Other languages don't, but they also don't support safety concepts that C++ nominally does like safety critical systems. "Good enough" can be perfectly fine for a web service like Go while being grossly inadequate for HPC or safety critical.
- steveklabnik 1y ago
  You’re misunderstanding what I’m saying. Safe Rust guarantees memory safety. Profiles do not. This is regardless of the ability of the unchecked versions, on both sides, to introduce issues. Profiles do not, even for code that is 100% using profiles, guarantee safety.
  - pizlonator 1y ago

tialaramex 1y ago

Yes I can be specific. Firstly, you need composition. Rust's safety composes. The safe Rust library for farm animals from Geoff, the safe Rust library for cooking recipes by Alice and the safe Rust library for web server by Bert together with my safe program code adds up to my safe Rust farm foods web site. By having N profiles, where N is intended to be at least five and might grow arbitrarily and be user extensible, C++ guarantees it cannot deliver composition this way. Maybe they can define some sort of composition and maybe everybody will ship software which conforms to that definition and so eventually they get composition, that's not there today, so it's just a giant unknown at best. Secondly, of the profiles described so far, most of them are just solving parts of the single overarching problem Rust addresses, for the serial case. So if they ship that, which already involves some amount of new work yet to be finished, you need all of those profiles to get to only partial memory safety. Which comes to the third part. Once you start down this path, as they found, you realise you actually want a borrowck. You won't call it that of course, because that would be embarrassing. But you'll need to track reference lifetimes and you'll need annotation and you end up building most of the stuff you insisted you didn't want. For now, you can handwave, this is an unsolved static analysis problem. Well, not so much unsolved as you know the solution and you don't like it. Your idea to do the reference counting everywhere is not something WG21 has looked at, I think the perf cost is sufficiently bad that they won't even glance at it. They're also not going to ship a GC. Finally though, C++ is a concurrent language. It has a whole memory model which doesn't even make sense if you aren't thinking about concurrency. But to deliver concurrent memory safety without Fil-C's overheads you would want... well, Rust's Send and Sync traits, which sure enough have eerie twins in the Safe C++ proposal. No attempt to solve this is even hinted at in the current profiles proposal, and they would need to work one out and if it's not Send + Sync again they'd need to prove it is correct.

pizlonator 1y ago
I think the point is that folks will incrementally move their code towards having all profiles enabled, and that's sort of fundamental if the goal is to give folks with C++ codebases an incremental path to safety. So I don't buy your first and second points. > Which comes to the third part. Once you start down this path, as they found, you realise you actually want a borrowck. That's a bold statement. It might be true for some very loose definition of "borrow checker". See the super simple static analysis that WebKit uses (that presentation is now linked in at least two places on this HN discussion, so I won't link it again). > Your idea to do the reference counting everywhere is not something WG21 has looked at, I think the perf cost is sufficiently bad that they won't even glance at it. They're also not going to ship a GC. The point isn't to have ref counting on every pointer at the language level, but rather: if your prevent folks from calling `delete` directly (as one of the profiles does) then you're effectively forcing folks to use smart pointers. Reference counting that happens by smart pointers is something that they would ship. We know this because it's already happened. I imagine this would really mean that some references are ref counted (if you use shared_ptr or similar) while other references use some other policy. > Finally though, C++ is a concurrent language. It has a whole memory model which doesn't even make sense if you aren't thinking about concurrency. But to deliver concurrent memory safety without Fil-C's overheads you would want... well, Rust's Send and Sync traits Yeah, this might be an area where they leave a hole. Like, you might have reference counting that is only partially thread safe: - The refcount of any object is atomic. - The smart pointer itself is racy. So, racing on pointers can pop the protections. If they got that far, then that wouldn't be so bad. The marginal safety advantage of Rust would be very slim at that point.

Rusky 1y ago

If that is what profiles were actually doing, it would probably make sense. But it's not what profiles are doing. Instead, for example, the lifetime safety profile (https://github.com/isocpp/CppCoreGuidelines/blob/master/docs/Lifetime.pdf https://github.com/isocpp/CppCoreGuidelines/blob/master/docs...) is a Rust-like compile time borrow checker that relies on annotations like [[clang::lifetimebound]], yet they also repeatedly insist that profiles will not require this kind of annotation (see the papers linked from https://www.circle-lang.org/draft-profiles.html#abstract https://www.circle-lang.org/draft-profiles.html#abstract). Their messaging is just not consistent with the concrete proposals they have described, let alone actually implemented.

pjmlp 1y ago
Additionally they ignore field experience, I can tell that on VC++ the lifetime checker only has worked in small examples, as I was really into trying it out. Microsoft even has blog posts admitting that only with SAL like annotations it can be improved, while keeping the usual C++ semantics. Yet WG21 has ignored this field experience.

dminik 1y ago

As far as I'm concerned, there are two main issues with profiles: 1. They're either unimplementable or useless (too many false positives and false negatives). I think this is pretty evident based on the fact that profiles have been proposed for a while and that no real implementation exists. Worse, out of all of the open source projects and for profit companies, noone has been able to implement any sort of static analysis that would even begin to approach the guarantees Rust makes. 2. The language doesn't give you any tools to actually write safe code. Ok, let's say that someone actually implements safety profiles. And it highlights your usage of a standard library type. What do you do? Safe C++ didn't require a new standard library just because. The current stdlib is riddled with safety issues that can't really be fixed and would not be fixed because of backwards compatibility. You're stuck. And so you turn the safety profile off.