12 ms·
Newer C++ features can create a lot of system yak shaving
- HelloNurse 9y agoRunning freshly compiled experimental or very recent software on very old servers that cannot be upgraded piecemeal seems very unusual: if you don't want to destabilize the servers with updates, you shouldn't want to destabilize them with unproven new software either. It's also strange to be unwilling to fork in source control, if necessary, a maintenance version for legacy servers and a current version for current C++ and current servers. New software should probably run on more current servers and access the old RHEL 6 servers through stable network services (shared folders, DBMS, etc.) instead of expecting them to run bleeding edge software.
- AHTERIX5000 9y agoRHEL 6.0, the version released something like 10 years ago? Yeah, that is going to be painful and maybe the way out isn't rebuilding everything manually but upgrading the system ;-) I only had to deal with RHEL6.x that had gcc 4.8 installed via packaging, I think Red Hat had a solution for this.
- cowsandmilk 9y ago> I think Red Hat had a solution for this. Yes, Red Hat has a solution in the form of the Developer Tool Set, which will give you gcc 7.2.1[0] [0] https://access.redhat.com/documentation/en-us/red_hat_developer_toolset/7/html/7.0_release_notes/dts7.0_release https://access.redhat.com/documentation/en-us/red_hat_develo...
- aldanor 9y agoDitto, just wanted to say the same thing. On RH / CentOS, devtoolset exists exactly for the reason OP is worried about.
- wycy 9y agoCan someone elucidate why one might not want to just update from RHEL 6--especially on what I gather is a personal machine in the OP--rather than go through this?
- noselasd 9y agoBecause the personal machine might be used to develop for a production machine running RHEL 6, and that cannot be upgraded because reasons(no time, software doesn't support RHEL 7 yet, $precious_driver doesn't support RHEL 7, + a lot of other reasons that makes sense to an OPS person but not a developer). Binaries created on RHEL 7 does not run on RHEL 6 - even when linking statically appears to work, you run the risk of running into the unforseen. You could develop on RHEL 7 and build for production on RHEL 6, but this causes you enough grief and trobleshooting nesting up in all kinds of minor differences to not make it worth it. Best you can do if you cannot upgrade from RHEL 6 is to install devtoolset(https://www.softwarecollections.org/en/scls/rhscl/devtoolset-7/ https://www.softwarecollections.org/en/scls/rhscl/devtoolset... - these are official packages crerated by Red Hat) to get a newer compiler.
- thawkins 9y agoLoad fedora 27, and put all your legacy projects into a rhel6 docker container, with your workspace volume mapped into a "projects" folder, that way you also get to play your code with multiple rhel variants.
- gnufx 9y agoThe only major difficulty in my experience doing EPEL/Fedora builds from the same source has normally been the prohibition on using SCLs in EPEL, which at least has been relaxed recently to allow devtoolset on EPEL. You can build for EPEL6 just fine on RHEL7, assuming you have the privilege to run mock. (EPEL packages are currently built on Fedora 27.)
- joelthelion 9y agoEither update RHEL6 or keep using the version of C++ that is supported on your OS. It's really that simple...
- wolfgke 9y ago> Either update RHEL6 or keep using the version of C++ that is supported on your OS. It's really that simple... Or switch to Windows: Here you can use about any version of C++ you want (just e.g. install the respective version of Visual Studio that supports your desired version of C++). No need to pay attention which version of C++ is supported on which Windows version. :-)
- jjnoakes 9y agoYou still have to pay attention to what version of C++ your dependencies compiled with, unless you are compiling them all. And on Linux you can install newer compilers too. The problems are still there; the root cause is C++ having no ABI, not Linux vs Windows.
- aorth 9y ago> The only way out of this is to tear off the bandage and rebuild all of those with the new compiler and side-load them as well. This industry is just one big test of how long you can shave yaks (aka pay technical debt) before you quit and move to the forest. It's rewarding... until it's not. :)
- jjuhl 9y agoOP could just use devtoolset-7 on RHEL6 (https://access.redhat.com/documentation/en-us/red_hat_developer_toolset/7/html/7.0_release_notes/dts7.0_release https://access.redhat.com/documentation/en-us/red_hat_develo...) and get a modern (supported) compiler that way.
- inetknght 9y agoHaving used devtoolset-3, -4, -6, and -7, I can confirm that it's awesome
- Hello71 9y agoI don't think this fixes the problem of mixing C++ ABIs though.
- CUViper 9y agoThe devtoolset compilers are explicitly configured to keep compatibility with the base system's ABI. edit: well, OK, the official answer is more nuanced: https://access.redhat.com/documentation/en-us/red_hat_developer_toolset/7/html/user_guide/sect-GCC-CPP#sect-GCC-CPP-Compatibility https://access.redhat.com/documentation/en-us/red_hat_develo...
- ihnorton 9y agoOnly assuming you have root or a sufficiently cooperative IT department. In situations where this is a problem, both assumptions are often untrue (managed HPC).
- hermitdev 9y agoBeen there, done that. Several jobs ago. At a hedge fund, corporate mandated OS at the time was RHEL5. C++11 had been finalized for about 3 years at this point, latest GCC at the time had nearly full support (only glaring gap I remember at the time was regex was "there" but it was a stubbed-out interface, it didn't actually work). We'd even had lecturers in the office of the likes of Scott Meyers & Herb Sutter on C++11, but could we use it? Hell, no. Shit, some of us had even been interviewed by Scott Meyers for his Effective Modern C++ book. Still couldn't officially use C++11, so, given the frustration, we went rogue. It was a pain, but I built the latest stable GCC from scratch and all of the libs we needed against it (boost, unixodbc, mqseries, intel tbb, tibrv). It took a while (I think GCC took like 4 hours to build at the time, even with a parallelized build), but once set up, it was liberating.
- bipson 9y agoSo it's not the modern C++ features that cause the yak-shaving, but wanting to use these features on a RHEL6 instance with a compiler that does not support these modern features. I'm specifically stating this (obvious observation), because the headline might lead someone to believe that the features are at fault. The features are great and should considerably reduce yak shaving. Going a step further, we've all been there. I once spent weeks building STLPort and boost, so I could avoid re-writing and extending an existing project against a C++ compiler from 2006, and more importantly, the WinCE 6.0 API (with weird timers and everything - ugh), and use a kind-of portable, recent version of C++, with extended features. I deeply questioned every single professional decision I had made the prior years. This had nothing to do with yak-shaving, but with how much platform-specific code you would end up prior to C++11, and how much simple, every-day functionality you had to write again and again. C++11 and later are a huge leap in this regard IMO, and for everyone stuck with an old compiler: Consider using boost.
- fndrplayer13 9y agoI think this is a great summary. C++11 is a huge leap forward if you can start fresh and/or only target C++11 compatible systems. Personally, I've also been down the road of trying to get older systems (also RHEL6) to run C++11 code via a hand-built compiler and it was an exercise in absolute frustration. And yes, Boost was the answer to some of this. Another thing you can try if you're in a well-abstracted code-base is to write wrappers around some of this functionality. Eg, if C++11 or greater is available, use the STL functionality, else use Boost and/or whatever legacy thing we hand-built. The obviously crappy part of this is that your functionality can/will diverge on multiple platforms.
- eXpl0it3r 9y agoIt's a real click-bait title. Using incompatible versions of anything will sooner than later cause issues. It's not related to C++ or C++ features at all, it's simply that in this case the version of the compiler supporting newer C++ features, wasn't compatible with the rest of the system. However, ABI incompatibility is a real pain in C++ and one of the main reasons why I don't judge anyone building C libraries instead. While RHEL6 is just one example, you'll run into this issue all the time when developing on Windows. Here, either your C++ compiled libraries match 100% with the compiler used for to compile your application or it will simply not work. In 99.99% of the time you get a symbol issue with std::string you can be certain it's runtime library issue/compiler incompatibility.
- Kenji 9y agoI have to admit this was a real bummer. I know better than to try to upgrade anything that important on a Red Hat type box. There's just no sense in attempting to diverge from the base system. You either keep it or throw it out and go to the next major release... and I'm not ready for that. That leaves me with one option: leave it in place and then "side load" a new compiler and/or libraries, and then switch to using it for all of my local stuff. Just use docker?? That is literally what docker is for. Get your favourite flavour of debian or whatever, with a modern GCC and fire up that docker container. EDIT: Oh wow, docker is only available in RHEL7 and higher. Damn.
- Volcacius 9y agoHad the same issue with RHEL6, solved (more or less) using Gentoo Prefix. It comes with its own libc and the bootstrapping process was pretty straightforward.
- orbifold 9y agoHPC has the same problem (ancient bespoke Linux distributions with strange tooling) one way to solve it is spack https://github.com/spack/spack https://github.com/spack/spack. The ops person where I work is so much in love with it that he tried to get it to work on Mac OS, 2 days later and several patches it took only 8 hours to compile all nescessary transitive dependencies. In my opinion the only good way to solve this is to ignore the distribution package manager and keep track of all dependencies explicitly. Most build systems for C++ make that relatively easy. To choose the compiler / standard library you can then use http://modules.sourceforge.net http://modules.sourceforge.net
- gnufx 9y agoThe RHEL6 compute node image I used to maintain just used rpms (maybe via SCL). I'm inclined to think the trades-off win against Spack and Guix, assuming you don't have the world telling you to use proprietary compilers and libraries that can't be handled that way. The mess I'm used to seeing on HPC systems with combinatorial builds, with everything done through a confusion of environment modules, bothers me and typically confuses users, especially when there's a system package that provides the same thing. (At least look at TACC's Lmod instead of the canonical modules implementation.)
- aphexairlines 9y ago> When it comes to C++ programs and libraries, you really do not want to try to mix and match across compilers. It really does not like it, and it will make you suffer. Hence the Amazon build tooling for C and C++. I wish they would open source it, write about it, make it into an AWS product, or release it in any other form.
- 0xFFC 9y agoCan you provide a little bit more information about Amazon tools for C/C++?
- aphexairlines 9y agoThere's an overview of Brazil and Apollo here: https://gist.github.com/terabyte/15a2d3d407285b8b5a0a7964dd6283b0 https://gist.github.com/terabyte/15a2d3d407285b8b5a0a7964dd6... And a bit more about Apollo as part of the AWS CodeDeploy announcement: https://www.allthingsdistributed.com/2014/11/apollo-amazon-deployment-engine.html https://www.allthingsdistributed.com/2014/11/apollo-amazon-d...
- oblio 9y agoI don't want to be mean, but people like to make fun of Java. In this regard life is much, much easier. Yes WORA is only partially true, but it is still true. Life is so much easier it's not even funny. Except for the most performance critical bits I'd never write anything in C++... Oh, and embedded. I'm sorry you guys still have to go through all this pain, I know it's a really hard problem with all the legacy.
- banachtarski 9y agoNah we feel sorry for you guys haha
- chris_wot 9y agoI’ve recently been reading up on how Rust does memory management and I’ve been wondering... whilst not a really new feature, I wonder if you could program via unique_ptrs in C++ and use RAII to handle concurrency in much the same way that Rust deals with it.
- banachtarski 9y agoThat's what many people do but not necessarily unique_ptrs ALL the time (too may heap allocations make the CPU a sad panda)
- snarfy 9y agoC++ really needs a standard ABI. None of these newer features really matter much without it. You still can't make libraries without compiling your code 10 different times with a handful of compilers and flags and shipping all of the resulting binaries.
- lallysingh 9y agoThere is one. Apologies for the ugly link, it goes to a pdf and I'm on a phone: https://www.google.com/url?sa=t&source=web&rct=j&url=https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf&ved=2ahUKEwiR4IimoZ7aAhVM2IMKHSlbD3gQFjAAegQIBxAB&usg=AOvVaw0MVQdLNlkOBigLu5VDb82x https://www.google.com/url?sa=t&source=web&rct=j&url=https:/... 10 times? Do you mean 10 platforms vs 1 on something like Java?
- vinkelhake 9y agoI don't think C++ desperately needs a standard ABI, but ignoring that for a moment... The System V ABI doesn't really do anything for the people who asks for a standard C++ ABI. It's not enough to mandate how objects are laid out. People want the ability to define a library function that uses standard types - for example std::string - in the interface. To build a library with libstdc++ and link it with an application that has been built with libc++, there would (among other things) have to be a common string type that they agree on. Herb Sutter has done some work in this area, but I don't know the current status of it. https://isocpp.org/files/papers/n4028.pdf https://isocpp.org/files/papers/n4028.pdf
- jcelerier 9y ago> To build a library with libstdc++ and link it with an application that has been built with libc++, there would (among other things) have to be a common string type that they agree on. well, then there would be a single standard library implementation. I think that would be the way to go : just put libc++ out of the std namespace and in a custom namespace, `st2` for instance, and use its types from everywhere. MSVC, GCC, whatever.
- 9y ago
- ksherlock 9y agoLast year I was playing with Red Hat's Open Shift. Remember when -std=c++0x was cutting edge? RHEL does. I ended up compiling (and statically linking) my c++11 code with Windows/WSL/Ubuntu/g++ and copying it over.
- dralley 9y agoAs many others have pointed out, there is an officially supported solution to this problem.... https://access.redhat.com/documentation/en-us/red_hat_developer_toolset/7/html/7.0_release_notes/dts7.0_release https://access.redhat.com/documentation/en-us/red_hat_develo...
- rosshemsley 9y agoI actually thought this would be about yak shaving due to the huge number of ways to achieve everything in modern C++xy. My internal monologues when writing modern C++ look a lot like this... So I have to return from this function... it uses a lot of resources, so I don't want to copy it. But then there is RVO... Although is this actually NRVO? What was that rule again? I guess I can rely on move semantics? But only Scott Meyers himself understands those and he promptly exited the C++ community after figuring them out... And anyway, what if I want to change the ownership explicitly? Let's just use a unique_ptr. Oh but actually I mostly want to share it when I have returned, and that means two blocks of memory will be allocated per pointer instead of the optimal "single" allocation with the reference count at the beginning. Although maybe that's only in libc++? oh, and what if there is an error, should it return nullptr in that case?..
- vinkelhake 9y agoI realize that you're mostly ranting, but on the off chance that you're serious: check out std::make_shared. It's a helper function that can allocate storage for the object and shared_ptr's control block in one go. It comes with its own caveats (what doesn't in C++?), but it has the answer to your concern about allocations.
- ovao 9y agoTo add: according to cppreference, all known STL implementations leverage this optimization. This used to be specific to Microsoft's STL implementation, but apparently all the major STL vendors are now on-board with it.
- gpderetta 9y agomake_shared traces its roots back into the original boost implementation.
- rosshemsley 9y agoYep I know it :) My joke was "Oh, I should return a unique_ptr<>, but then I will be losing out on the make_shared optimization" ;)
- mark-r 9y agoI've heard the term "yak shaving" before but the meaning escapes me - can anybody fill me in?
- lisper 9y agoIt's the result of an internal monologue that goes something like, "I want to do X, and the best way to do it is to use this spiffy new feature Y, but in order to access that I need to upgrade P and Q and R, and while I'm at it I may as well also take care of A and B and C..." and at the end of this process you find yourself shaving a yak, having completely forgotten about what it was you originally set out to do.
- majewsky 9y ago> Any apparently useless activity which, by allowing you to overcome intermediate difficulties, allows you to solve a larger problem. Source: https://en.wiktionary.org/wiki/yak_shaving https://en.wiktionary.org/wiki/yak_shaving Like when you have to spend hours fiddling with $package_manager because you want to build a $hipster_language app.
- cfv 9y agoCarlin Vieri, a googler, coined the term while at MIT. " 1) Any apparently useless activity which, by allowing you to overcome intermediate difficulties, allows you to solve a larger problem. 2) A less useful activity done consciously or subconsciously to procrastinate about a larger but more useful task. "
- PeterisP 9y agoWatch this gif https://imgur.com/gallery/rQIb4Vw https://imgur.com/gallery/rQIb4Vw , and at the scene of car with the engine taken out imagine shaving a yak. But the original story, as far as I know, comes from Seth Godin http://sethgodin.typepad.com/seths_blog/2005/03/dont_shave_that.html http://sethgodin.typepad.com/seths_blog/2005/03/dont_shave_t...
- mark-r 9y agoAh, Malcolm in the Middle - how I miss it. I think that was about the time I stopped watching TV.
- yuniq 9y agoI just wish 'auto' in Cpp can be changed to 'var'. Switching between C#/JS/C++, I always end up typing var instead of auto :).
- lisper 9y ago#define var auto Not that I necessarily recommend doing this.
- deleted 9y ago[deleted]
- YSFEJ4SWJUVU6 9y agoThat's very unlikely to happen, ever. (Too many "int var;" out there.)
- hermitdev 9y agoYes. Also, the name auto was chosen for a very specific reason: it was already a keyword, albeit one that was rarely used. It was deemed better to re-purpose the existing keyword that introduce a new one. New non-contextual keywords are a big deal.
- jerrre 9y agoconst var would become a nice contradiction...
- banachtarski 9y agoWhy are you using var in JS.
- vernie 9y agoWhat are you supposed to be using?
- plorkyeran 9y ago
- wallstprog 9y agoThe author is correct that trying to get modern toolchains on older systems (like RH6) is a real pain. But sometimes there's no real choice -- some applications value reliability above all else, and tend to be deployed on RH where its relative stability is a feature, not a bug. I've written about this (http://btorpey.github.io/blog/2015/01/02/building-clang/ http://btorpey.github.io/blog/2015/01/02/building-clang/), specifically in the context of getting clang running on RH6, which required first getting a C++11-compatible gcc running on RH6. For those who are in the same boat, you may find the article helpful.
- lallysingh 9y agoI hate to be /that guy/, but newer systems can run nix, which takes care of this stuff for you. I dunno about RHEL 6.
- superbatfish 9y agoFWIW, I highly recommend using conda as a package manager for your own pure C++ packages. You can compile everything with the compilers they ship, and the only external dependency from your own system will be glibc. Yes, it means you're using your own "mini-me environment", but you can share that environment across all of your C++ projects. As long as you build with: LDFLAGS="-Wl,-rpath,${PREFIX}/lib -L${PREFIX}/lib" ... then everything you compile ends up in the self-contained environment. Furthermore, distributing your build products to another user (or another machine) is as simple as: tar -czf mystuff.tar.gz ${PREFIX} Then your friends can take your tarball and unzip it in any directory of their choosing. The whole prefix (environment) is self contained (except glibc, which must be at least as new as the version on your build machine). It just works.
- heavenlyhash 9y agoDo you have any links to docs or blog posts about how you'd go about doing that with Conda and ${PREFIX} managing compilers and libs? I'd love to see what a fully worked example looks like.
- RayDonnelly 9y agoOfficial blog posts: https://www.anaconda.com/blog/developer-blog/improved-security-performance-in-anaconda-distribution-5/ https://www.anaconda.com/blog/developer-blog/improved-securi... https://www.anaconda.com/blog/developer-blog/announcing-the-release-of-anaconda-distribution-5-0/ https://www.anaconda.com/blog/developer-blog/announcing-the-... https://www.anaconda.com/blog/developer-blog/utilizing-the-new-compilers-in-anaconda-distribution-5/ https://www.anaconda.com/blog/developer-blog/utilizing-the-n... I would recommend using conda-build to build packages, since it provides a lot of tooling to ensure things work well and to detect problems. You can still create software by taring up the final ${PREFIX} if you wish, but if it's open source stuff then the best thing to do is to try to get the conda build recipes accepted by conda-forge. The recipes that make the Anaconda Distribution can be found at: https://github.com/AnacondaRecipes/aggregate https://github.com/AnacondaRecipes/aggregate
- RayDonnelly 9y ago
- gok 9y ago> you find out that existing C++ libraries on the box are no longer happy being linked into your program, like, oh, say, protobuf, gmock or gtest Or even worse, they link happily then fail weirdly at run time because all the STL structure layouts have changed. This is hugely exacerbated by Linux distributions perpetuating the fiction that C++ is an appropriate language for system library APIs. So now there is a generation of dependency-oriented developers who haven’t realized that “apt-get install libfoo-cxx-dev” is an enormous headache waiting to happen.
- hellofunk 9y ago> scoped_ptr which worked just like the one at a certain company. It gives you a few neat things, like the ability to just forward-declare a class inside your .h file and not include the whole thing right there. But all pointers do this in C++? Why would you need a smart pointer to do this for you?
- ben0x539 9y agoI think the starting point is having a by-value member of that class, which has convenient automatic destruction semantics, but requires the actual class definition in scope. To lift that requirement, you move to a pointer field, and if you go with a scoped_ptr rather than a raw pointer, you don't lose the automatic destruction.
- hermitdev 9y agoscoped_ptr has the same issue due to the implicit destructor needing to know how to destroy the thing being held. The trick is you need to explicitly define your destructor (and it's implementation needs to have full knowledge of what it's pointing to). It doesn't have to do anything - ideally, it should be empty. This is known as the PIMPL idiom, or Cheshire Cat.
- ben0x539 9y agowow somehow i was sure i could get out of that just by having the destructor of the containing class in a separate compilation unit, that's unfortunate.
- hermitdev 9y ago> A couple of days ago, that's exactly what I did. It turns out that it's not a huge deal to compile gcc with a prefix which will put it off to the side and then just drop the entire mess into that directory. You can then just do /path/to/your/new/g++ and you'll have the new compiler working for you. It'll gobble up the new syntax and will be happy. I've been there, and it's not fun. And, been down the path of options the author enumerates after this statement. Short of it is, on Linux (and I'm sure it's likely the same on BSDs & Mac), if you're not using the system supplied compiler, you're in for a world of hurt. You will have to build everything you depend on from scratch with your custom compiler. That said, it's possible. Just not fun, or user-friendly.
- ixtli 9y agoAfter reading this I'm left with the feeling that the author is complaining more about "enterprise" linux distros that never ever change (hello, many centos users who are still running distros that require python2.7!) than about changes to the C++ standard. And for what it's worth, I agree that I'm always frustrated by such things, but it's not clear this is something we should blame on attempts to keep C++ modern.
- makecheck 9y agoEver since my first Linux experience, where /usr actually contained way more useful tools by default than other systems, this has been a double-edged sword. All the convenience of autoconf and default behaviors, etc. goes far out the window when /usr/lib is ever so slightly too old. It’s kind of a curse. The virtualenv concept makes more sense: set a default on day 1 but assume you’ll want a new foundation periodically. When the foundations themselves are versioned, you can evolve your environment to some degree without having a completely chaotic mix of the latest updates.
- ris 9y agoYeah, this whole thing ain't a problem with Nix. The whole idea of only a single file being able to exist at /usr/lib/libstdc++.so seems quite backwards and ancient to me now.
- w_t_payne 9y agoI am not sure what the OP is complaining about. Isn't this just another day in the office for a C++ dev?