11 ms·

Dropping support for old C++ standards

130 points by learnedSloth 4y ago

VHRanger 4y ago
Titus Winters leading the google abseil library eventually came to the conclusion that the only sane way to manage a large scale C++ system is to "live at head" [1] -- that is, libraries should live at the head production version of their dependencies. This is patchworked around in more easygoing languages with dependency management systems, docker containers, etc. etc. but if you can enforce living at head from the start it makes everyone's life easier. https://abseil.io/about/philosophy#we-recommend-that-you-choose-to-live-at-head https://abseil.io/about/philosophy#we-recommend-that-you-cho...
- fbdab103 4y ago
  Doesn't that require you have a Google sized army of engineers and tooling to stay in sync?
  - jcelerier 4y ago
    I do it pretty much on my own for https://ossia.io https://ossia.io ; it takes me roughly a day every other month to update my mac, linux and windows SDKs to the new LLVM / Qt / FFMPEG / {... other large dependency I use ...}. Definitely not the end of the world. Said SDKs & build scripts are available here: https://github.com/ossia/sdk https://github.com/ossia/sdk for anyone interested (I'll be honest though: the scripts are a mess!)
    - dataflow 4y ago
      1 day for... ~500 kLOC, it seems. Which is pretty good I think, but larger codebases would have a lot more to deal with -- not just more dependencies, but also more breakages per dependency.
  - mastax 4y ago
    Or you just checkout the version that does what you need and leave it alone.
    - deleted 4y ago
      [deleted]
    - UncleMeat 4y ago
      google3 has a one version rule. You can’t just check out the version of a dependency that does what you want.
- bluGill 4y ago
  That works for google web where they can upgrade everything at will. When you work in embedded upgrades can cost millions of dollars since someone needs to go to remote locations to run the upgrade.
- bb88 4y ago
  That's fine as long as HEAD doesn't break things. I can no longer count the number of times we had an issue with a "supposedly" minor release that ended up breaking major things in our stack. Most of them were things that could have been detected using unit tests or some kind of basic regression testing. If you have a 1000 dependency packages, and at any point in time 0.1% of them are broken, then odds are you will always have something broken.
  - lbrandy 4y ago
    We should be clear... in these large organizations... HEAD is always broken. But it has the advantage of being broken for everyone, tested by everyone, fixed by everyone, and thus fixed for everyone. And this usually makes it far better than the alternatives. Having 1000 dependencies with versions pinned means you are living alone and will run into fewer issues, but when they do come, they will be absolute nightmares that no one else is dealing with and no one can help with. And one day you'll have to do the game of begging someone else to upgrade their version of a downstream thing to fix the issue, and they won't, so you'll try to get the other group to backport the fix in their thing to the version you can't upgrade off. And they won't. etc. etc. Full versioning is the worst of all approaches, IMO, for large complex interconnected codebases (especially ones that are many-to-many from libraries to output binaries) but it absolutely is sometimes the only viable one (for example, the entire open-source-ecosystem is a giant(er) version of this problem, and in that space, versioning is the only thing that I can imagine working).
    - galangalalgol 4y ago
      Supply chain attacks are worsened if everyone lives at the head. Staying far enough behind that some brave (and hopefully small) project discovers the compromise of a repo for some dependency five layers deep before you re-pin to a new version is probably the best mitigation short of some permissions based model like Austral is working on.
      - VHRanger 4y ago
        Supply chain attacks only matter for libraries that can make their own network call, or libraries that directly touch unsanitized web input however?
- leroy-is-here 4y ago
  is "living at the head" constantly pulling each update or version-controlled source of every library?
- dataflow 4y ago
  Note that "large scale" is pulling a heavy weight here. Also, living at the HEAD of your language standard is quite a bit different from living at the HEAD of other dependencies.
  - jcelerier 4y ago
    It's not just the language standard, they build and deploy the latest clang HEAD every week or so
    - dataflow 4y ago
      Yes I realize. I didn't claim otherwise. The original post was about dropping support for older C++ standards. I was pointing out that Abseil's proposition is quite a bit more heavy handed than Boost's.
- pabs3 4y ago
  Does that include the head of GCC/LLVM?
  - nlewycky 4y ago
    It does. Google mirrors every commit to LLVM in their monorepo, builds and tests the whole monorepo with a fresh Clang nightly, and (ideally) one of those Clang nightlies is released as the new stable compiler for all users of the monorepo every week. This helps keep Google at HEAD and helps keeps LLVM upstream stable.
    - fooker 4y ago
      I thought llvm lived in /third_party, and internal projects usually target specific LLVM versions, not the current open source trunk.
      - nlewycky 4y ago
        As far as I know, there was only ever one version of LLVM at a time in the monorepo. It's possible things changed after I left (either the compiler team or Google). Each upstream commit didn't land directly into the monorepo, instead there was a long lived branch, and on the compiler team there was a buildcop rotation responsible for doing an integrate from that branch into //third_party/llvm. This included running the tests (and fixing any problems) for any other software that depends on LLVM as well as building an unstable crosstool and doing some basic smoke tests on that. Taking that crosstool through testing and to stable crosstool was the responsibility of a different buildcop rotation, using a special compiler team tool for testing the testing crosstool nightly, then to release to stable we used the ordinary presubmits, but for all projects at once, making its testing as similar as possible to any normal code change.
        pabs3 4y ago
        How does Google deal with other projects' embedded code copies? ie some other project embeds a random old snapshot of LLVM, when you import that project into the monorepo do you end up with two copies of LLVM, or do you strip the old copy and port the codebase to the current LLVM in the monorepo?
- noobermin 4y ago
  Who is "everyone" here? It sounds like "everyone" is just the developers.
- friendzis 4y ago
  This somehow reminds me of the argument around git-flow. It's a decent reasoning, however, the whole idea is based on having a single, bleeding edge version. Basically a SaaS. A lot of companies/products do not work that way. Some have physical products out there that have to be updated, some have on-premises deployments, some sell user software of which there are multiple versions under support. Each of these live versions have to have their own source branches and dependency trees. A single `:latest` can render future bugfixes unbuildable.
- saagarjha 4y ago
  Google concludes the way they’ve always done things is the only way that scales, news at 11. Staying at HEAD can be quite nice in some respects, but the conclusion there only applies to Google, and only then the Google that was actively created to be amenable to the results you’re seeing.
- Narann 4y ago
  I'm not sure to understand: What "living at the head" means?
  - humanrebar 4y ago
    There are various ways to implement this, but to simplify the explanation, assume that versioning always works via lockfiles [1]. Lockfiles record all of the versions of all of the projects that contributed to some experiment or release. They're common in various language-specific ecosystems like gem, pip, cargo, and so on. Assuming you have these lockfiles, you have the typical option, which would be to make a lockfile for each released entity, record it for later reproduction, and update it at least every release. The "live at head" approach would be to instead have a main shared lockfile for every project in the company in a recorded sequence. All projects pick a version of that lockfile to release from. Practically speaking, all projects probably just take the latest version (the head) of that lockfile and everyone works hard to make sure that lockfile always works for everyone. The main advantage here is pretty straightforward combinatorial math. Maintaining and validating unique combinations of dependencies for every release in a codebase is NP-silly, whereas sharing one set of dependencies across as many applications as possible isn't easy but it has a much nicer cost curve. In theory at least, but a lot of large organizations claim practice backs the theory up as well. [1] Versioning doesn't have to work this way. Putting all code into one big source repository (vendoring) has the same effect.
mappu 4y ago
I'm surprised to see "GCC as shipped by RHEL 7" used as an argument downthread. If you need to use an older GCC then you can equally well use an older Boost.
- AnthonyMouse 4y ago
  You can often have multiple dependencies. Library A uses Boost and requires an older version of GCC. Library B uses Boost and requires the newer version of Boost. You want to use libraries A and B in the same project, what now?
  - dataflow 4y ago
    > what now? Build each one as a separate shared library and wrap each one with a C interface?
    - gpderetta 4y ago
      You can expose a C++ interface, as long as all the types used are ABI stable (i.e std library types bit not boost types). I have used many libraries that use boost internally (often in a custom namespace) but do not expose it on the API
  - deleted 4y ago
    [deleted]
  - deleted 4y ago
    [deleted]
  - Waterluvian 4y ago
    Like Nodejs, just have both, dedupe when possible, static compilation, still fight over getting both libraries to co-operate with different Boosts, rage a bit, curse thee, thy name is dependency hell!
  - jamesfinlayson 4y ago
    I've never worked in a big C++ code base or had that issue but could you library B in a namespace?
xvilka 4y ago
Should drop anything below C++14. Everyone who is not happy could always stay on older versions.
- doodlesdev 4y ago
  Why not go for C++17 already? C++14 seems pretty arbitrary, AFAIK C++17 is already supported by GCC, clang and MSVC fully [0]. It's also now the default dialect in a few compilers including GCC. Meanwhile C++20 isn't really well supported at all in the moment [1] so it makes sense waiting more for it, even though features such as Concepts and Modules should simplify library code a lot and make some tasks much more trivial. [0]: https://en.cppreference.com/w/cpp/compiler_support/17 https://en.cppreference.com/w/cpp/compiler_support/17 [1]: https://en.cppreference.com/w/cpp/compiler_support/20 https://en.cppreference.com/w/cpp/compiler_support/20
  - pclmulqdq 4y ago
    The argument I can see is that 14 is the minimum feature set that every engineer knows how to use, and before 14 you are missing some of the "core" features of the language that are widely used today. By contrast, some things in 17 are still foreign to some engineers, although there are no real "breaking" changes from 14 to 17 (in terms of the model of how to think about the language). 20 and 23 are a bit of a mess in terms of support.
    - Gigachad 4y ago
      6 years is more than enough time to get ready for the update. If you can’t work out how to use c++ 17 then that isn’t the problem of library devs.
      - pjmlp 4y ago
        Last time I read about it, PlayStation SDK was on C++14, and they aren't the only platform in such state.
jokoon 4y ago
I really like what Herb Sutter is doing with cpp2/cppfront: it's a new language that translates to C++, by defaulting to C++ good practices and "avoiding 95% of C++ pitfalls". Please watch its presentation on it. It's designed to interact with C++. Backward compatibility is good to have, but C++ needs for alternatives that allow it to drop support for old things, because the language needs to evolve and backward compatibility is preventing it, and it turns into very very long compile time. And even if backward compatibility is out in C++, it would break in the language, not in the ABI, so old C++ and new C++ could still easily cohabit together, quite like C has always lived with C++ for a long time now. I like C++, but nobody denies that C++ carries a lot of weight for being disliked.
- skywal_l 4y ago
  Have a look at Circle from Sean Baxter [0]. It's pretty impressive. [0]: https://github.com/seanbaxter/circle/blob/master/new-circle/README.md https://github.com/seanbaxter/circle/blob/master/new-circle/...
- mzs 4y ago
  this? https://youtu.be/ELeZAKCN4tY https://youtu.be/ELeZAKCN4tY
  - jokoon 4y ago
    yes
maldev 4y ago
So I mainly do Kernel or super low level systems code. And i've never had an issue targeting the latest C++ standard. Given I don't use the C++ STL and only use the language features. The backwards compatability is amazing and i've never had a single issue in my career with swapping forward. So good on them. I also do Windows work though, so maybe it's different on Linux.
NoZZz 4y ago
Boo boooo.

130 points by learnedSloth 4y ago

Titus Winters leading the google abseil library eventually came to the conclusion that the only sane way to manage a large scale C++ system is to "live at head" [1] -- that is, libraries should live at the head production version of their dependencies. This is patchworked around in more easygoing languages with dependency management systems, docker containers, etc. etc. but if you can enforce living at head from the start it makes everyone's life easier. https://abseil.io/about/philosophy#we-recommend-that-you-choose-to-live-at-head https://abseil.io/about/philosophy#we-recommend-that-you-cho...

fbdab103 4y ago
Doesn't that require you have a Google sized army of engineers and tooling to stay in sync?
- jcelerier 4y ago
  I do it pretty much on my own for https://ossia.io https://ossia.io ; it takes me roughly a day every other month to update my mac, linux and windows SDKs to the new LLVM / Qt / FFMPEG / {... other large dependency I use ...}. Definitely not the end of the world. Said SDKs & build scripts are available here: https://github.com/ossia/sdk https://github.com/ossia/sdk for anyone interested (I'll be honest though: the scripts are a mess!)
  - dataflow 4y ago
    1 day for... ~500 kLOC, it seems. Which is pretty good I think, but larger codebases would have a lot more to deal with -- not just more dependencies, but also more breakages per dependency.
- mastax 4y ago
  Or you just checkout the version that does what you need and leave it alone.
  - deleted 4y ago
    [deleted]
  - UncleMeat 4y ago
    google3 has a one version rule. You can’t just check out the version of a dependency that does what you want.
bluGill 4y ago
That works for google web where they can upgrade everything at will. When you work in embedded upgrades can cost millions of dollars since someone needs to go to remote locations to run the upgrade.
bb88 4y ago
That's fine as long as HEAD doesn't break things. I can no longer count the number of times we had an issue with a "supposedly" minor release that ended up breaking major things in our stack. Most of them were things that could have been detected using unit tests or some kind of basic regression testing. If you have a 1000 dependency packages, and at any point in time 0.1% of them are broken, then odds are you will always have something broken.
- lbrandy 4y ago
  We should be clear... in these large organizations... HEAD is always broken. But it has the advantage of being broken for everyone, tested by everyone, fixed by everyone, and thus fixed for everyone. And this usually makes it far better than the alternatives. Having 1000 dependencies with versions pinned means you are living alone and will run into fewer issues, but when they do come, they will be absolute nightmares that no one else is dealing with and no one can help with. And one day you'll have to do the game of begging someone else to upgrade their version of a downstream thing to fix the issue, and they won't, so you'll try to get the other group to backport the fix in their thing to the version you can't upgrade off. And they won't. etc. etc. Full versioning is the worst of all approaches, IMO, for large complex interconnected codebases (especially ones that are many-to-many from libraries to output binaries) but it absolutely is sometimes the only viable one (for example, the entire open-source-ecosystem is a giant(er) version of this problem, and in that space, versioning is the only thing that I can imagine working).
  - galangalalgol 4y ago
    Supply chain attacks are worsened if everyone lives at the head. Staying far enough behind that some brave (and hopefully small) project discovers the compromise of a repo for some dependency five layers deep before you re-pin to a new version is probably the best mitigation short of some permissions based model like Austral is working on.
    - VHRanger 4y ago
      Supply chain attacks only matter for libraries that can make their own network call, or libraries that directly touch unsanitized web input however?
leroy-is-here 4y ago
is "living at the head" constantly pulling each update or version-controlled source of every library?
dataflow 4y ago
Note that "large scale" is pulling a heavy weight here. Also, living at the HEAD of your language standard is quite a bit different from living at the HEAD of other dependencies.
- jcelerier 4y ago
  It's not just the language standard, they build and deploy the latest clang HEAD every week or so
  - dataflow 4y ago
    Yes I realize. I didn't claim otherwise. The original post was about dropping support for older C++ standards. I was pointing out that Abseil's proposition is quite a bit more heavy handed than Boost's.
pabs3 4y ago
Does that include the head of GCC/LLVM?
- nlewycky 4y ago
  It does. Google mirrors every commit to LLVM in their monorepo, builds and tests the whole monorepo with a fresh Clang nightly, and (ideally) one of those Clang nightlies is released as the new stable compiler for all users of the monorepo every week. This helps keep Google at HEAD and helps keeps LLVM upstream stable.
  - fooker 4y ago
    I thought llvm lived in /third_party, and internal projects usually target specific LLVM versions, not the current open source trunk.
    - nlewycky 4y ago
      As far as I know, there was only ever one version of LLVM at a time in the monorepo. It's possible things changed after I left (either the compiler team or Google). Each upstream commit didn't land directly into the monorepo, instead there was a long lived branch, and on the compiler team there was a buildcop rotation responsible for doing an integrate from that branch into //third_party/llvm. This included running the tests (and fixing any problems) for any other software that depends on LLVM as well as building an unstable crosstool and doing some basic smoke tests on that. Taking that crosstool through testing and to stable crosstool was the responsibility of a different buildcop rotation, using a special compiler team tool for testing the testing crosstool nightly, then to release to stable we used the ordinary presubmits, but for all projects at once, making its testing as similar as possible to any normal code change.
      - pabs3 4y ago
        How does Google deal with other projects' embedded code copies? ie some other project embeds a random old snapshot of LLVM, when you import that project into the monorepo do you end up with two copies of LLVM, or do you strip the old copy and port the codebase to the current LLVM in the monorepo?
noobermin 4y ago
Who is "everyone" here? It sounds like "everyone" is just the developers.
friendzis 4y ago
This somehow reminds me of the argument around git-flow. It's a decent reasoning, however, the whole idea is based on having a single, bleeding edge version. Basically a SaaS. A lot of companies/products do not work that way. Some have physical products out there that have to be updated, some have on-premises deployments, some sell user software of which there are multiple versions under support. Each of these live versions have to have their own source branches and dependency trees. A single `:latest` can render future bugfixes unbuildable.
saagarjha 4y ago
Google concludes the way they’ve always done things is the only way that scales, news at 11. Staying at HEAD can be quite nice in some respects, but the conclusion there only applies to Google, and only then the Google that was actively created to be amenable to the results you’re seeing.
Narann 4y ago
I'm not sure to understand: What "living at the head" means?
- humanrebar 4y ago
  There are various ways to implement this, but to simplify the explanation, assume that versioning always works via lockfiles [1]. Lockfiles record all of the versions of all of the projects that contributed to some experiment or release. They're common in various language-specific ecosystems like gem, pip, cargo, and so on. Assuming you have these lockfiles, you have the typical option, which would be to make a lockfile for each released entity, record it for later reproduction, and update it at least every release. The "live at head" approach would be to instead have a main shared lockfile for every project in the company in a recorded sequence. All projects pick a version of that lockfile to release from. Practically speaking, all projects probably just take the latest version (the head) of that lockfile and everyone works hard to make sure that lockfile always works for everyone. The main advantage here is pretty straightforward combinatorial math. Maintaining and validating unique combinations of dependencies for every release in a codebase is NP-silly, whereas sharing one set of dependencies across as many applications as possible isn't easy but it has a much nicer cost curve. In theory at least, but a lot of large organizations claim practice backs the theory up as well. [1] Versioning doesn't have to work this way. Putting all code into one big source repository (vendoring) has the same effect.

mappu 4y ago

I'm surprised to see "GCC as shipped by RHEL 7" used as an argument downthread. If you need to use an older GCC then you can equally well use an older Boost.

AnthonyMouse 4y ago
You can often have multiple dependencies. Library A uses Boost and requires an older version of GCC. Library B uses Boost and requires the newer version of Boost. You want to use libraries A and B in the same project, what now?
- dataflow 4y ago
  > what now? Build each one as a separate shared library and wrap each one with a C interface?
  - gpderetta 4y ago
    You can expose a C++ interface, as long as all the types used are ABI stable (i.e std library types bit not boost types). I have used many libraries that use boost internally (often in a custom namespace) but do not expose it on the API
- deleted 4y ago
  [deleted]
- deleted 4y ago
  [deleted]
- Waterluvian 4y ago
  Like Nodejs, just have both, dedupe when possible, static compilation, still fight over getting both libraries to co-operate with different Boosts, rage a bit, curse thee, thy name is dependency hell!
- jamesfinlayson 4y ago
  I've never worked in a big C++ code base or had that issue but could you library B in a namespace?

xvilka 4y ago

Should drop anything below C++14. Everyone who is not happy could always stay on older versions.

doodlesdev 4y ago
Why not go for C++17 already? C++14 seems pretty arbitrary, AFAIK C++17 is already supported by GCC, clang and MSVC fully [0]. It's also now the default dialect in a few compilers including GCC. Meanwhile C++20 isn't really well supported at all in the moment [1] so it makes sense waiting more for it, even though features such as Concepts and Modules should simplify library code a lot and make some tasks much more trivial. [0]: https://en.cppreference.com/w/cpp/compiler_support/17 https://en.cppreference.com/w/cpp/compiler_support/17 [1]: https://en.cppreference.com/w/cpp/compiler_support/20 https://en.cppreference.com/w/cpp/compiler_support/20
- pclmulqdq 4y ago
  The argument I can see is that 14 is the minimum feature set that every engineer knows how to use, and before 14 you are missing some of the "core" features of the language that are widely used today. By contrast, some things in 17 are still foreign to some engineers, although there are no real "breaking" changes from 14 to 17 (in terms of the model of how to think about the language). 20 and 23 are a bit of a mess in terms of support.
  - Gigachad 4y ago
    6 years is more than enough time to get ready for the update. If you can’t work out how to use c++ 17 then that isn’t the problem of library devs.
    - pjmlp 4y ago
      Last time I read about it, PlayStation SDK was on C++14, and they aren't the only platform in such state.

jokoon 4y ago

I really like what Herb Sutter is doing with cpp2/cppfront: it's a new language that translates to C++, by defaulting to C++ good practices and "avoiding 95% of C++ pitfalls". Please watch its presentation on it. It's designed to interact with C++. Backward compatibility is good to have, but C++ needs for alternatives that allow it to drop support for old things, because the language needs to evolve and backward compatibility is preventing it, and it turns into very very long compile time. And even if backward compatibility is out in C++, it would break in the language, not in the ABI, so old C++ and new C++ could still easily cohabit together, quite like C has always lived with C++ for a long time now. I like C++, but nobody denies that C++ carries a lot of weight for being disliked.

skywal_l 4y ago
Have a look at Circle from Sean Baxter [0]. It's pretty impressive. [0]: https://github.com/seanbaxter/circle/blob/master/new-circle/README.md https://github.com/seanbaxter/circle/blob/master/new-circle/...
mzs 4y ago
this? https://youtu.be/ELeZAKCN4tY https://youtu.be/ELeZAKCN4tY
- jokoon 4y ago
  yes

maldev 4y ago

So I mainly do Kernel or super low level systems code. And i've never had an issue targeting the latest C++ standard. Given I don't use the C++ STL and only use the language features. The backwards compatability is amazing and i've never had a single issue in my career with swapping forward. So good on them. I also do Windows work though, so maybe it's different on Linux.

NoZZz 4y ago

Boo boooo.