6 ms·

Python 3 broke backwards compatibility and everyone stayed on 2 as a result. Why is breaking backwards compatibility "for the hell of it" in C++ not going to h

Python 3 broke backwards compatibility and everyone stayed on 2 as a result. Why is breaking backwards compatibility "for the hell of it" in C++ not going to have the same problem? Codebases stay in stasis, stuck on older compilers, for far less drastic reasons. And I've found that modernizing a codebase is a lot easier if you can compile it to test your incremental changes. Talking about a magic wand that makes source-to-source changes on a 100MLOC codebase trivial is pure fantasy - requiring you to ignore that even in the best version control systems, the resulting ~10MLOC merge is going to cause a lot of merge conflicts (some of which will be incorrectly resolved) and history clutter. To say nothing of all the poor bastards who are satisfied with the occasional tarball per release. Who are still doing better than the people who can't be bothered to backup the code they wrote directly on the server it will run on. And nevermind the fact that at least some of your downstream customers are still using some old compiler that they're stuck with - the only option for some legacy embedded platform they can't eliminate yet, made by that company that went out of business. And maybe that kind of cost would be worth it - for the right reasons, given the right motivations. But I feel like I'm looking at a laundry list of mostly minor, tiny issues, that are a "solved problem" in the form of existing warnings. I can crank them up to errors as I modernize a codebase. You can enable them by default in your build systems and IDEs. You can mandate them across your organization, and in your contracts. You can incrementally apply them per library as you get around to doing more maintenance on them.

makecheck 11y ago
Had Python 3 stuck to relatively simple changes like converting to the print() function, making a few libraries more consistent, etc. then it would be closer to the C++ compatibility-breaking changes proposed here. Instead, Python 3 went far beyond things that were easily convertible and it isn't surprising that adoption was greatly slowed. C++ has atrocious defaults, where you have to add a bunch of keywords and do all these things just to make the simplest classes do the "right" thing. It's long overdue for some new behaviors. Adding things like "override" to existing code has been an absolute joy. It has already prevented errors during maintenance in projects at work. It is well worth a few days of ugly revision control logs. There are lots of ways to smooth a backward-incompatible transition for C++ on a larger scale. For instance, they could enable it on a file-by-file basis by having some new declaration like "using c++17;" (or better still, require only the old, out-of-date files to add this declaration in order to continue compiling).
- alextgordon 11y ago
  The print function was one of the worst offenders. It is a pointless vanity feature (Guido prefers the new syntax), hardly worth the economic cost of upgrading all Python code in existence to adopt it.
  - marvy 11y ago
    Actually, he prefers the old syntax. The new syntax is there because he got swayed by practical considerations. For instance, back in July 2000, there was this: https://www.python.org/dev/peps/pep-0214/ https://www.python.org/dev/peps/pep-0214/ Where he basically said "Maybe print should be a function, but... nah". Two years later, he says he regrets it: http://legacy.python.org/doc/essays/ppt/regrets/PythonRegrets.pdf http://legacy.python.org/doc/essays/ppt/regrets/PythonRegret... And the benefits are explained here: https://www.python.org/dev/peps/pep-3105/ https://www.python.org/dev/peps/pep-3105/
    - lisper 11y ago
      Could have had the best of both worlds by giving the print function a different name from the print statement (e.g. printf for the function). Then the print statement could have been deprecated gracefully. (And then the print function could have been renamed print if anyone really cared at that point.)
      - eru 11y ago
        Or alternatively they could have remove the mandatory parentheses around function calls, and kept the old syntax and made print a function. (Sorry, trolling. ML-style function call syntax would clash a lot with other parts of Python.)
        marvy 11y ago
        I don't know ML, but I really do wish Python had this feature. The only problem I see is that this makes it hard to refer to a function without calling it (see below). How does ML solve this? def g(): return 5 assert g() == 5 assert g != 5 Without parenthesis, how can the language distinguish the last two lines? How does ML do it? Would the same trick work in Python? I know Ruby has optional parenthesis, but functions aren't quite as first class there as they are in Python, which is why the have Proc and whatnot.
DennisMoore 11y ago
> But I feel like I'm looking at a laundry list of mostly minor, tiny issues, that are a "solved problem" in the form of existing warnings. I'm not aware of warnings for uninitialized variables in constructors. Can you clue me in how to get these on in Visual Studio? I'm using a third party tool that can detect uninitialized scalars only and would love to have the extra scanning.
- MaulingMonkey 11y ago
  Sadly Visual Studio's compiler has no warning I'm aware of. Using /sdl will forcibly initialize all your members however - I guess Microsoft's approach to the problem is that fixing it for you is better than giving you ignore-able warnings. There are third party tools - such as cppcheck - which should catch uninitialized members however. I generally also compile with clang, which catches a lot of additional stuff for me. And here's a random John Carmack link noting some of the static analysis tools out there: http://www.gamasutra.com/view/news/128836/InDepth_Static_Code_Analysis.php http://www.gamasutra.com/view/news/128836/InDepth_Static_Cod... I've been playing around with clang's Address Sanitizer all day as well, catching buffer overflows and underflows left and right in a janky old codebase I was suddenly thrust into recently...
  - DennisMoore 11y ago
    We are (slowly...) doing the same thing across our codebase. The tool we use -- PVS-Studio -- just introduced checking for uninit'd scalars so we are grinding those down and doing eyeball checks in the ctors, but I was hoping for a more reliable method for the latter portion. Thanks for /SDL! This looks like it may be very promising: http://blogs.microsoft.com/cybertrust/2012/06/06/warnings-sdl-and-improving-uninitialized-variable-detection/ http://blogs.microsoft.com/cybertrust/2012/06/06/warnings-sd... At the very least that would help remove random behavior at the cost of speed (if I understand correctly). I'll have to sleep on /SDL and do some perf testing come Monday. I had really wanted to use /analyze (https://msdn.microsoft.com/en-us/library/d3bbz7tz%28v=VS.100%29.aspx https://msdn.microsoft.com/en-us/library/d3bbz7tz%28v=VS.100...) but it crashes! D: Unfortunately, I'm rooted in Windows SDK and I don't think clang supports that. The Microsoft Application Verifier has some run-time memory checks, so I'm not totally unarmed. I looked at PC-lint doesn't have complete support for C++11 yet and I'm worried about the rate of false positives. I had not heard of cppcheck, so I'll have to give that go since it looks free. Thanks for the info!
conistonwater 11y ago
> a magic wand that makes source-to-source changes on a 100MLOC codebase trivial is pure fantasy Actually, I have a slightly unrelated question: what are the issues that affect 100MLOC codebases that aren't so important for smaller projects? This is something people sometimes bring up, but I'm not too sure what exactly they mean by it.
- perlgeek 11y ago
  It's basically impossible to review and touch the whole code base if it's 100MLOC. Unless there is a very good reason that convinces the business sponsors that it's OK to spend a few man years on it. Switching to a shinier version of the programming language usually isn't.
  - forrestthewoods 11y ago
    Are those projects likely to switch to c++11 or c++14 which dint have breaking changes? It seems unlikely. In which case why not break things?
    - adrianN 11y ago
      Those kind of projects tend to use an ancient proprietary compiler that is no longer maintained.
  - plonh 11y ago
    Google brags about constantly doing large scale refactoring on heir codebase.
- MaulingMonkey 11y ago
  The sheer size of the thing is a big problem. Getting ~100KLOC of C++ compiling on a "new" compiler has been something that's taken me a few days before. Not too unreasonable. 100MLOC is literally 1000x the size - which extrapolating from "a few days" would mean it'd take me 8-14 years to fix up on my own were I to do it by hand. Or more realistically, the pace of change/growth on a codebase of that size probably outstrips my ability to convert it. So now even a simple straightforward task such as fixing up technically nonstandard C++ syntax to standard C++ that multiple compilers will happily accept, is a task requiring either automated code modifications (scary!) or a large coordinated team effort. If 100 people working together are, optimistically, 100x as efficient as me alone - i.e. there's no overhead in terms of coordination and planning - it will still take them around a month to do the job via brute force.
Too 11y ago
The problem with python is that they broke both source-compatibility and library-compatibility at the same time. If c++ breaks source compatibility but still allows calling external C-libraries or older c++ libraries nobody would even blink.
humanrebar 11y ago
> I feel like I'm looking at a laundry list of mostly minor, tiny issues, that are a "solved problem" in the form of existing warnings. I can crank them up to errors as I modernize a codebase. So the perspective you might be missing is that there absolutely are broken or useless parts of the C++ standard. There has to be a way to fix those if we want: 1. Continued improvements to the C++ language 2. Compilers that aren't needlessly complex to implement and support 3. Fewer weird edge cases and therefore fewer bugs when people miss them. 4. A language that can be taught to you instead of drilled into you. A case in point is trigraphs. They are an almost entirely useless feature that requires an extra compiler pass. They cause real bugs that confuse and surprise all but the most grizzled C++ devs (myMap["crazyUserName!!!??!?!"]). It was like moving heaven and Earth to get them slated for removal in C++17. It is not acceptable that things like this are this hard to get deprecated and removed. There needs to be an accepted, uncontroversial way to identify cruft, mark it deprecated, and eventually remove it from the language. So... why the minor, tiny issues? Because we're not ready for the big ones yet. Let's get some minor issues under our belts and then we'll figure out how to deal with bigger ones.
- mreiland 11y ago
  > It is not acceptable that things like this are this hard to get deprecated and removed. Not only is it acceptable, it's preferable. What's not acceptable is when it's IMPOSSIBLE to get things deprecated and removed.
- MaulingMonkey 11y ago
  > So the perspective you might be missing is that there absolutely are broken or useless parts of the C++ standard. I'm not missing it. I feel C++ is a broken enough language that I generally prefer to use others. That said, C++ is my day job, so I get to feel the burn and wish it weren't so broken. As a result, I'm all for getting rid of useless parts of the C++ language. But backwards compatibility is not useless! I'm willing to sacrifice minor, "theoretical" backwards compatibility for relatively minor reasons - I don't think I've seen any code broken by >> being interpreted differently in C++11 in templates, for example. I only know of one case where trigraphs were used intentionally - and compilers have been disabling them by default and warning about them for quite some time now (to the point where I don't really care if the C++ standard is fixed, because C++ in the wild is fixed.) But getting rid of NULL and 0 as pointers? Breaking overrides missing the override keyword? Every C++ codebase I know will break massively. It'll make C++ification of a C codebase even more of a massive pain in the ass. Pass. Make them enabled warnings by default instead. Enable warnings as errors when you can. People are already doing this for you: http://codepad.org/R5XyUby2 http://codepad.org/R5XyUby2. -------------------------------- RE 1: Continued improvements to the C++ language are occuring. C++11, C++14, and C++17 are all introducing new stuff, and providing better options for securing your code. This does not require breaking backwards compatability. Unfortunately this goes against points 2, 3, and 4, for reasons that have nothing to do with what's been listed here. RE 2: I don't see anything in this list that would help significantly there. If anything, mandating more diagnostics about override keywords will complicate the compiler further. RE 3: So lets add some sequence points that make parameter evaluation order defined. Lets get rid of those type aliasing rules that every large codebase runs afoul of. Lets do some high impact stuff that will catch some really nasty edge cases that, realistically, will fix more codebases than it breaks. ...but none of that seem to be on this list. RE 4: A lost cause that's getting worse. Adding a bunch of caveats that "oh, except in C++2x you can't do this thing that's in decades worth of C++ tutorials, examples, and codebases - including most of what you've seen online" isn't going to make the teaching problem any easier. It's going to make it worse. ---------------------------------- "So... why the minor, tiny issues? Because we're not ready for the big ones yet. Let's get some minor issues under our belts and then we'll figure out how to deal with bigger ones." We've already got lots of experience under our belts with fixing minor issues. auto, >>, template exports, throw() statements - all the way back to the very first standard, which eliminated implicit conversions from int -> enum, void* -> t* , and for loop scoping - with all the breaking of C or prestandard C++ that this entailed. What will it take for us to be "ready" to deal with things like parameter evaluation order being unspecified - and thus at the mercy of your compiler version, flags, and phase of the moon? Fixing that would break no code that wasn't already broken, and eliminate a massive source of unspecified weirdness that I still struggle to explain to other C++ developers who have been coding for years. I still have to carefully explain the difference between operator precedence (how a() - b() - c() is parsed), and sequence points (which define the evaluation order of a(), b(), and c() - or more accurately don't). They point to && and ||, and I explain how they - along with "," and "?:" - are special cased by the C++ standard to define additional sequence points that just happen to match the operator precedence rules - unless you overload them, so never do that - and by the end of it, I've either blown their minds or they still don't quite believe me. Showing them how std::cout << i++ << i++ << i++ << "\n"; changes behavior depending on compiler flags tends to help - although that's cheating as there's additional unspecified (or undefined?) behavior for modifying the same integral multiple times without an intervening sequence point...

makecheck 11y ago

Had Python 3 stuck to relatively simple changes like converting to the print() function, making a few libraries more consistent, etc. then it would be closer to the C++ compatibility-breaking changes proposed here. Instead, Python 3 went far beyond things that were easily convertible and it isn't surprising that adoption was greatly slowed. C++ has atrocious defaults, where you have to add a bunch of keywords and do all these things just to make the simplest classes do the "right" thing. It's long overdue for some new behaviors. Adding things like "override" to existing code has been an absolute joy. It has already prevented errors during maintenance in projects at work. It is well worth a few days of ugly revision control logs. There are lots of ways to smooth a backward-incompatible transition for C++ on a larger scale. For instance, they could enable it on a file-by-file basis by having some new declaration like "using c++17;" (or better still, require only the old, out-of-date files to add this declaration in order to continue compiling).

alextgordon 11y ago
The print function was one of the worst offenders. It is a pointless vanity feature (Guido prefers the new syntax), hardly worth the economic cost of upgrading all Python code in existence to adopt it.
- marvy 11y ago
  Actually, he prefers the old syntax. The new syntax is there because he got swayed by practical considerations. For instance, back in July 2000, there was this: https://www.python.org/dev/peps/pep-0214/ https://www.python.org/dev/peps/pep-0214/ Where he basically said "Maybe print should be a function, but... nah". Two years later, he says he regrets it: http://legacy.python.org/doc/essays/ppt/regrets/PythonRegrets.pdf http://legacy.python.org/doc/essays/ppt/regrets/PythonRegret... And the benefits are explained here: https://www.python.org/dev/peps/pep-3105/ https://www.python.org/dev/peps/pep-3105/
  - lisper 11y ago
    Could have had the best of both worlds by giving the print function a different name from the print statement (e.g. printf for the function). Then the print statement could have been deprecated gracefully. (And then the print function could have been renamed print if anyone really cared at that point.)
    - eru 11y ago
      Or alternatively they could have remove the mandatory parentheses around function calls, and kept the old syntax and made print a function. (Sorry, trolling. ML-style function call syntax would clash a lot with other parts of Python.)
      - marvy 11y ago
        I don't know ML, but I really do wish Python had this feature. The only problem I see is that this makes it hard to refer to a function without calling it (see below). How does ML solve this? def g(): return 5 assert g() == 5 assert g != 5 Without parenthesis, how can the language distinguish the last two lines? How does ML do it? Would the same trick work in Python? I know Ruby has optional parenthesis, but functions aren't quite as first class there as they are in Python, which is why the have Proc and whatnot.

DennisMoore 11y ago

> But I feel like I'm looking at a laundry list of mostly minor, tiny issues, that are a "solved problem" in the form of existing warnings. I'm not aware of warnings for uninitialized variables in constructors. Can you clue me in how to get these on in Visual Studio? I'm using a third party tool that can detect uninitialized scalars only and would love to have the extra scanning.

MaulingMonkey 11y ago
Sadly Visual Studio's compiler has no warning I'm aware of. Using /sdl will forcibly initialize all your members however - I guess Microsoft's approach to the problem is that fixing it for you is better than giving you ignore-able warnings. There are third party tools - such as cppcheck - which should catch uninitialized members however. I generally also compile with clang, which catches a lot of additional stuff for me. And here's a random John Carmack link noting some of the static analysis tools out there: http://www.gamasutra.com/view/news/128836/InDepth_Static_Code_Analysis.php http://www.gamasutra.com/view/news/128836/InDepth_Static_Cod... I've been playing around with clang's Address Sanitizer all day as well, catching buffer overflows and underflows left and right in a janky old codebase I was suddenly thrust into recently...
- DennisMoore 11y ago
  We are (slowly...) doing the same thing across our codebase. The tool we use -- PVS-Studio -- just introduced checking for uninit'd scalars so we are grinding those down and doing eyeball checks in the ctors, but I was hoping for a more reliable method for the latter portion. Thanks for /SDL! This looks like it may be very promising: http://blogs.microsoft.com/cybertrust/2012/06/06/warnings-sdl-and-improving-uninitialized-variable-detection/ http://blogs.microsoft.com/cybertrust/2012/06/06/warnings-sd... At the very least that would help remove random behavior at the cost of speed (if I understand correctly). I'll have to sleep on /SDL and do some perf testing come Monday. I had really wanted to use /analyze (https://msdn.microsoft.com/en-us/library/d3bbz7tz%28v=VS.100%29.aspx https://msdn.microsoft.com/en-us/library/d3bbz7tz%28v=VS.100...) but it crashes! D: Unfortunately, I'm rooted in Windows SDK and I don't think clang supports that. The Microsoft Application Verifier has some run-time memory checks, so I'm not totally unarmed. I looked at PC-lint doesn't have complete support for C++11 yet and I'm worried about the rate of false positives. I had not heard of cppcheck, so I'll have to give that go since it looks free. Thanks for the info!

conistonwater 11y ago

> a magic wand that makes source-to-source changes on a 100MLOC codebase trivial is pure fantasy Actually, I have a slightly unrelated question: what are the issues that affect 100MLOC codebases that aren't so important for smaller projects? This is something people sometimes bring up, but I'm not too sure what exactly they mean by it.

perlgeek 11y ago
It's basically impossible to review and touch the whole code base if it's 100MLOC. Unless there is a very good reason that convinces the business sponsors that it's OK to spend a few man years on it. Switching to a shinier version of the programming language usually isn't.
- forrestthewoods 11y ago
  Are those projects likely to switch to c++11 or c++14 which dint have breaking changes? It seems unlikely. In which case why not break things?
  - adrianN 11y ago
    Those kind of projects tend to use an ancient proprietary compiler that is no longer maintained.
- plonh 11y ago
  Google brags about constantly doing large scale refactoring on heir codebase.
MaulingMonkey 11y ago
The sheer size of the thing is a big problem. Getting ~100KLOC of C++ compiling on a "new" compiler has been something that's taken me a few days before. Not too unreasonable. 100MLOC is literally 1000x the size - which extrapolating from "a few days" would mean it'd take me 8-14 years to fix up on my own were I to do it by hand. Or more realistically, the pace of change/growth on a codebase of that size probably outstrips my ability to convert it. So now even a simple straightforward task such as fixing up technically nonstandard C++ syntax to standard C++ that multiple compilers will happily accept, is a task requiring either automated code modifications (scary!) or a large coordinated team effort. If 100 people working together are, optimistically, 100x as efficient as me alone - i.e. there's no overhead in terms of coordination and planning - it will still take them around a month to do the job via brute force.

Too 11y ago

The problem with python is that they broke both source-compatibility and library-compatibility at the same time. If c++ breaks source compatibility but still allows calling external C-libraries or older c++ libraries nobody would even blink.

humanrebar 11y ago

> I feel like I'm looking at a laundry list of mostly minor, tiny issues, that are a "solved problem" in the form of existing warnings. I can crank them up to errors as I modernize a codebase. So the perspective you might be missing is that there absolutely are broken or useless parts of the C++ standard. There has to be a way to fix those if we want: 1. Continued improvements to the C++ language 2. Compilers that aren't needlessly complex to implement and support 3. Fewer weird edge cases and therefore fewer bugs when people miss them. 4. A language that can be taught to you instead of drilled into you. A case in point is trigraphs. They are an almost entirely useless feature that requires an extra compiler pass. They cause real bugs that confuse and surprise all but the most grizzled C++ devs (myMap["crazyUserName!!!??!?!"]). It was like moving heaven and Earth to get them slated for removal in C++17. It is not acceptable that things like this are this hard to get deprecated and removed. There needs to be an accepted, uncontroversial way to identify cruft, mark it deprecated, and eventually remove it from the language. So... why the minor, tiny issues? Because we're not ready for the big ones yet. Let's get some minor issues under our belts and then we'll figure out how to deal with bigger ones.

mreiland 11y ago
> It is not acceptable that things like this are this hard to get deprecated and removed. Not only is it acceptable, it's preferable. What's not acceptable is when it's IMPOSSIBLE to get things deprecated and removed.
MaulingMonkey 11y ago
> So the perspective you might be missing is that there absolutely are broken or useless parts of the C++ standard. I'm not missing it. I feel C++ is a broken enough language that I generally prefer to use others. That said, C++ is my day job, so I get to feel the burn and wish it weren't so broken. As a result, I'm all for getting rid of useless parts of the C++ language. But backwards compatibility is not useless! I'm willing to sacrifice minor, "theoretical" backwards compatibility for relatively minor reasons - I don't think I've seen any code broken by >> being interpreted differently in C++11 in templates, for example. I only know of one case where trigraphs were used intentionally - and compilers have been disabling them by default and warning about them for quite some time now (to the point where I don't really care if the C++ standard is fixed, because C++ in the wild is fixed.) But getting rid of NULL and 0 as pointers? Breaking overrides missing the override keyword? Every C++ codebase I know will break massively. It'll make C++ification of a C codebase even more of a massive pain in the ass. Pass. Make them enabled warnings by default instead. Enable warnings as errors when you can. People are already doing this for you: http://codepad.org/R5XyUby2 http://codepad.org/R5XyUby2. -------------------------------- RE 1: Continued improvements to the C++ language are occuring. C++11, C++14, and C++17 are all introducing new stuff, and providing better options for securing your code. This does not require breaking backwards compatability. Unfortunately this goes against points 2, 3, and 4, for reasons that have nothing to do with what's been listed here. RE 2: I don't see anything in this list that would help significantly there. If anything, mandating more diagnostics about override keywords will complicate the compiler further. RE 3: So lets add some sequence points that make parameter evaluation order defined. Lets get rid of those type aliasing rules that every large codebase runs afoul of. Lets do some high impact stuff that will catch some really nasty edge cases that, realistically, will fix more codebases than it breaks. ...but none of that seem to be on this list. RE 4: A lost cause that's getting worse. Adding a bunch of caveats that "oh, except in C++2x you can't do this thing that's in decades worth of C++ tutorials, examples, and codebases - including most of what you've seen online" isn't going to make the teaching problem any easier. It's going to make it worse. ---------------------------------- "So... why the minor, tiny issues? Because we're not ready for the big ones yet. Let's get some minor issues under our belts and then we'll figure out how to deal with bigger ones." We've already got lots of experience under our belts with fixing minor issues. auto, >>, template exports, throw() statements - all the way back to the very first standard, which eliminated implicit conversions from int -> enum, void* -> t* , and for loop scoping - with all the breaking of C or prestandard C++ that this entailed. What will it take for us to be "ready" to deal with things like parameter evaluation order being unspecified - and thus at the mercy of your compiler version, flags, and phase of the moon? Fixing that would break no code that wasn't already broken, and eliminate a massive source of unspecified weirdness that I still struggle to explain to other C++ developers who have been coding for years. I still have to carefully explain the difference between operator precedence (how a() - b() - c() is parsed), and sequence points (which define the evaluation order of a(), b(), and c() - or more accurately don't). They point to && and ||, and I explain how they - along with "," and "?:" - are special cased by the C++ standard to define additional sequence points that just happen to match the operator precedence rules - unless you overload them, so never do that - and by the end of it, I've either blown their minds or they still don't quite believe me. Showing them how std::cout << i++ << i++ << i++ << "\n"; changes behavior depending on compiler flags tends to help - although that's cheating as there's additional unspecified (or undefined?) behavior for modifying the same integral multiple times without an intervening sequence point...