9 ms·
To save C, we must save ABI
- zabzonk 5y agoI'd just like to clarify that it's specific _compilers_ that produce code that complies with an ABI specification (or don't). Neither the C nor the C++ language standards specify an ABI format.
- mhh__ 5y agoC++ is increasingly being covered by an ABI in all but name i.e. there are types that do have standard layouts.
- zabzonk 5y ago> there are types that do have standard layouts Out of interest (I'm not tracking the C++ standard much these days) such as? For example I know about the order of struct members and bit fields, but that has always been the case in C, and so in C++.
- mhh__ 5y agohttps://en.cppreference.com/w/cpp/types/is_standard_layout https://en.cppreference.com/w/cpp/types/is_standard_layout Bitfields are profoundly non-standard.
- steerablesafe 5y agoBitfields are standard, what are you talking about?
- mathstuf 5y agoHow they live inside of a structure is far into implementation-defined land. No ABI (afaik) says where bits live if they do not fill an octet. In practice, the compilers agree with themselves over time, but that is "code as spec" which is everyone's favorite thing for interop.
- steerablesafe 5y agoIt's well defined in the system V ABI, isn't it? Compilers do follow that. Otherwise no layout is defined by the C and C++ standards, even outside of bitfields. edit: It's under "Bit-Fields" in [1]. I'm not entirely happy with the wording, but it's there. [1] https://raw.githubusercontent.com/wiki/hjl-tools/x86-psABI/x86-64-psABI-1.0.pdf#subsection.3.1.2 https://raw.githubusercontent.com/wiki/hjl-tools/x86-psABI/x...
- colonwqbang 5y agoABIs are not only a quirk of each compiler, rather the ABI can be covered by its own standards document. See for instance the SysV Amd64 ABI specification. One should not have the idea that ABIs are magical or impossible to specify, if it is desired to do so.
- tomohawk 5y agoThis is a well reasoned proposal to introduce a standard, zero cost way to introduce ABI indirection into C, which would enable much more rapid adoption of safer and more performant implementations.
- andi999 5y agoI always thought that C ABI stability allows you to freely use vendor dlls even legacy dlls for legacy hardware. Won't all of that break if you change the ABI?
- ngcc_hk 5y agoIs C a binary standard or is it is source level standard? And if there is a fixed relationship what should it be? Any scope? Any historical legacy issue? Btw the sample break is between two long long and one int128, why they have to the same I wonder.
- rurban 5y agoThe binary standard is defined in the gABI. They are independent. I want to fix something in it, so I need to fix C++, C, and the gABI. Three separate processes, each needing about 3 years, though C and C++ are aligned now.
- JonChesterfield 5y agoProbably worth checking that the compiler alias implementations do sane things when the function has a different calling convention to the default (due to some attribute on it)
- sirwhinesalot 5y agoI sincerely hope this gets adopted. The C ABI is critically important and it needs a mechanism like this.
- b112 5y ago[flagged]
- zozbot234 5y agoHow does this _Alias feature differ from simply defining the local "aliased" name as a static const pointer-to-function (at file-wide scope)? Why wouldn't that work already?
- JonChesterfield 5y agoSyntactically distinct. &foo has different type for function vs pointer to function. Also makes life difficult for the optimiser; I think solvably as long as the new pointer is indeed static.
- sirwhinesalot 5y agoNot guaranteed to be zero-cost I guess?
- zozbot234 5y agoWhy wouldn't it be zero cost, if _Alias can be? (Note, the fact that the function pointer would be declared as "static const" at file scope is relevant in replicating the _Alias featureset.)
- sirwhinesalot 5y agoThere is an extra degree of indirection the optimizer has to get rid of. There is no extra pointer in the _Alias version. Whether this minor difference is relevant I'll leave to better informed people.
- johannes1234321 5y agoThis is successfully used by glib and others by using linker symbol versioning.
- mananaysiempre 5y agoConceptually, yes. Practically, is there a reason why ELF symbol versioning needs to be this complicated beast requiring non-trivial loader support instead of, well, the trivial name hack being proposed here (and already used, among other places, in Musl for time64 support)? I’m honestly curious, I’ve read Drepper’s description[1] multiple times and I still can’t figure out the “why” of the whole thing. [1] https://www.akkadia.org/drepper/symbol-versioning https://www.akkadia.org/drepper/symbol-versioning ETA: Hmm, maybe semantic interposition / the global namespace of ELF dynamic linking is at fault here? As in, if a shared object declares an import of printf and a glibc version, it should get printf@@GLIBC_whatever if the import actually ends up being from glibc but plain printf otherwise? Not sure that’s a good interpretation of what versioning should mean in this context but it’s a possible one that would force this unpleasantness.
- gpderetta 5y agoI think the equivalent of alias solution are feature macros used to enable and disable different interfaces, like it was done ages ago on linux to switch from 32 bit off_t to 64.
- mwcampbell 5y agoMicrosoft's current policy of not adding new symbols to an existing CRT DLL is justified IMO. Any possible way of messing up a Windows machine will happen on some poor user's machine somewhere, including downgrading libraries. So if Microsoft is able to make their systems more robust in the face of misguided users (edit: and misguided third-party developers), that's good.
- sillysaurusx 5y agoI recently tried to get MSI Afterburner to run on a pretty standard new-ish windows 11 gaming pc. Two hours later, I gave up. The mysterious side-by-side manifest error wasn’t solvable. I installed all the redists, and even installed a neat redist installer I randomly found on GitHub which brute-force-installs every possible redist known to humanity. Didn’t matter. The side by side error was just insurmountable. Now, as a former gamedev who grew up on windows, I have a soft spot for Microsoft. I love me some visual studio. Yum yum, that IDE was cool a decade before Jetbrains showed everyone why IDEs are cool. But it’s been… eight years now since I switched to macOS, and more generally exited from the MS development ecosystem. I don’t know, man (or woman or they). Microsoft’s current policies are hopelessly complicated compared to unix. I’ve been programming for almost two decades. If I can’t analyze and fix an error that clearly shows exactly which manifest entry is missing, well… I think we’ll just agree to disagree that Microsoft is making nice decisions about ABIs and related ideas. Here’s a wild idea: just let a program run, and trap errors into an error log. Jetbrains does it. Their early access IDE happily lurches along even with dozens of Frankensteinesque errors coming up. Refusing to start because of an ABI mismatch (or indeed, dying at the first error) was one of the most unfortunate decisions in OS design.
- secondcoming 5y agoStrange, Windows is excellent when it comes to allowing old applications to run on newer releases.
- matheusmoreira 5y agoThat's no longer the case. There's a ton of software that simply doesn't run anymore. Last time I used Windows years ago, there were at least 10 old games in my Steam library that wouldn't even start. I wouldn't be surprised if at some point Wine becomes better at running these applications than Windows itself.
- ninkendo 5y agoThis is such an exhausting article to read. Useless introduction that assumes I'm familiar with their previous articles, so many long segues and needless anecdotes, it seemingly never gets to the point. I got about halfway through and gave up looking for where the actual point is. Could someone explain what they're actually driving at? > Just a type change! Shouldn’t change the assembly too much, right? What reasonable person would think that? You're changing something from 64 bits wide to 128 bits wide. Of course the compiled code from the 64-bit version is looking at a single 64-bit register, with the 128-bit version looking at multiple registers. Why is this unexpected? Who would ever expect that you could just change the width of the inputs and return type of a function and expect it not to break the ABI? > Okay, so in C we can break ABI just by having the wrong types on a function and not matching it up with a declaration Of course. Why wouldn't you think that? > What if I told you this exact problem could happen, even if the header’s code read extern long long do_stuff(long long value);, and the implementation file had the right declaration and looked fine too? Alright, I'm almost intrigued enough to keep reading. I hope they get to the point soon though. > [pages and pages about linux package maintainers and red herrings about python and C++] > [going on and on about "the problem" without having ever made it explicit] At this point I give up trying to find the author's point. What needs to be saved about the ABI specifically? I haven't found the problem they're talking about and I've given up trying to skim for it. I certainly am not going to read this whole thing verbatim because they're just too much time-wasting cruft here.
- JonChesterfield 5y agoStandardise alias in order to unstick library evolution
- ninkendo 5y agoWhat is currently not possible, which alias will fix? That’s what I couldn’t get out of the article. If the answer is “the ability to change the types of library functions without changing their name” (which is what his first few examples were showing to be the “problem”), then of course you can’t do that, and I’m not convinced we should waste any time or effort trying to get that to work. If you want to change the types accepted/returned by a function, make a new function with a new name. Of course, if there’s something I’m missing here I’d love to know… the article has done a terrible job outlining it if that’s the case.
- jcranmer 5y agoIf I'm understanding this article correctly, the basic proposal is that if you enable symbol aliasing, you can enable ABI breakages without any pain whatsoever. However, they seem to be ignoring one very important issue with ABI breakages: libraries tend to pass types around. The actual most painful part of ABI breakage is if you have, say, library A that's compiled against old-ABI and library B that's compiled against new-ABI and library A passes ABI-varying types to library B. Like, say, std::string (which changed ABI as a result of C++11). gcc managed to make it work--but the process was so painful (and required adding new features to C++ to do so!), that they have said that they do not want to have to ever do that again. It's relatively easy to completely change the functionality of a library function and use this sort of magic to always select the right function. But when you broach changing something like intmax_t and time_t, you have to be cognizant that the issue is different libraries (neither of whom are the standard library) who aren't prepared for the type to be different communicating with the same type.
- noobermin 5y agoNot sure about the article (skimmed it honestly) but the referenced video does very much talk about it, and their proposal is fuck abi stability.
- stncls 5y agoYou are right. The article does acknowledge this, though. https://thephd.dev/to-save-c-we-must-save-abi-fixing-c-function-abi#what-i%E2%80%99ve-proposed-here-does-not-fix-all-scenarios-some-of-them https://thephd.dev/to-save-c-we-must-save-abi-fixing-c-funct... The author then argues that it is at least a step forward: > For those of us in large ecosystems who have to write plugins or play nice with other applications and system libraries, we’re generally the last to get the benefits. But, at least we’ll finally have the chance to have that discussion with our communities, rather than just being outright denied the opportunity before Day 0.
- veltas 5y ago"To Save C" C is not going to 'die' over these issues. And frankly "new features" in the upcoming C standard will not make or break C, unless you consider C to only be the ISO standard efforts.
- JonChesterfield 5y agoI get less confident in my C working with each standard release. The march towards pointer provenance combined with optimisation passes written for C++ means the pointer hackery I use C for has a finite lifespan.
- kazinator 5y agoThere is no "The C ABI"; it is determined entirely by implementors. ISO C does not define the term "ABI", nor does it define anything like it under a different name. Some platforms have a documented ABI across the board which covers all their APIs together with the principal compiler family. Others, like Windows, have an ABI for some core system interfaces. Development tools can have different ABIs in their ecosystem. The rest of the article seems to be the complaint that there are problems linking code from different compilers on Microsoft Windows*, which is completely unsurprising in the light of my above remark. The fix is: don't mix code from different compilers on Windows without adding the calling convention declaration specifiers (an language extension) and ensuring these are in the header files that are mutually used.
- layer8 5y ago> The fix is: don't mix code from different compilers It’s more than that: individual compilers can’t evolve (for example change the size of intmax_t), and therefore certain desired ISO C features couldn’t be implemented, because that would break ABI compatibility with code compiled by earlier versions, which would prevent users from upgrading their compiler until all their dependencies have been upgraded, which (a) may never happen and/or (b) may be a newer version with other compatibility breaks. It may be okay in a everything-is-open-source-recompile-the-world setting, but in general it’s absolutely not practical.
- kazinator 5y ago> It may be okay in a everything-is-open-source-recompile-the-world setting That is mostly a myth, except maybe in some embedded situations. > individual compilers can’t evolve (for example change the size of intmax_t) Compilers can provide __my_intmax_t which is decoupled from the standard intmax_t. If there are system libraries and third party libraries using intmax_t, you don't want to change it; that isn't really a valid form of evolution. Long before 64 bit systems were common, compilers had local types like __int64_t. Programs could detect (or assume) their presence, and typedef them to something nicer. GCC has __int128_t on platforms where intmax_t is 64 bits.
- jokoon 5y agoI watched the video on C++ ABI. It's really awful to hold back a language because a few companies cannot deal with it. C++ should really start to deprecate thing, and reduce backward compatibility. Not to mention there are probably a few ways to use old code and to wrap it. It sort of boggles my mind how new C++ versions were able to maintain backward compatibility. Compilers must have a hard time doing it. That's why I think C++ should be forked... Not to mention microsoft is probably the main culprit here.
- gpderetta 5y agoOn the contrary Microsoft has happily broken the C++ ABI early and often with MSVC for a very long time. They started to guarantee more stability recently because users demanded it. In fact GCC (and indirectly Red Hat and the other Linux distros), have been the most vocal for not breaking the ABI as many still have the scars of past ABI breaks .
- int_19h 5y agoPart of it is that dynamic linking and globally shared libraries have been a norm on Linux for so long. On Windows, it is still more common to either statically link the stdlib or install it side by side.
- pjmlp 5y agoDynamic linking has a been a thing on Windows since Windows 16 bit days, and is central to COM, static libraries are seldom used. The big difference to UNIX shared objects (with exception of Aix), is that Windows dynamic libraries are namespaced and symbols are private by default. That makes huge difference.
- int_19h 5y agoDynamic linking is there, yes. But the only globally shared libraries tend to be the OS ones - everything else, even the CRT (before it also became part of the OS), is normally installed side by side with the .exe, and every app has its own copy. Thus C++ ABI compatibility was not really an issue, except when you had to use closed-source C++ libraries. Yes, the other big difference is that every DLL has its own namespace, so you can actually load two CRTs into the same process just fine - so long as you don't pass pointers/handles from one runtime to the other. But this kind of multi-runtime arrangement doesn't happen often in practice, except in plugin/extension scenarios (where different extensions might bring in different runtimes into the same process).
- benibela 5y agoWhy would you want to save C? C is already everywhere. And so unsafe, the sooner it dies, the better What we should be saving is Pascal.
- shikoba 5y agoKilling C :D There is always people dreaming of that.
- phendrenad2 5y ago> We could just deprecate std::regex and propose std::regex2, but the moment that proposal comes out everyone gets annoyed about the name and “why can’t we just fix regex?!” and then the ABI people pipe up and around and around the circle goes This is the core of the issue. I continue to maintain that these issues are not insurmountable, bit we have a "people problem" that makes them effectively insurmountable. There are too many people with deep expertise in one area of C++ (compiler implementation, language design, etc.), who don't have a big-picture view, yet will let their opinions known as though they know everything. And it's not just C++, all open-source languages have this problem. Design-by-committee never did anyone any favors. It's precisely WHY we keep getting half-assed implementations like regex and nested functions approved in the standard library pr GCC extension, and it's WHY no one is ever permitted to fix them. The only solution is for people to abandon their egos, admit that someone may know more than them (choc et horreur!), and try to do what's right for the users.
- int_19h 5y agoThere's nothing half-assed about gcc nested functions - they work exactly as they should. The problem is that the feature was implemented back when runtime constraints were very different - in particular, there were no limits on dynamic codegen, not even on the stack. Using thunks for callbacks was a common technique back then - and why not, given the perf benefits? On top of that, ironically, it was also the only way pointers to nested functions could be transparently compatible with the C ABI. So I don't think your criticism is valid in this case. It's more a general problem with all language extensions, and how they interplay with language evolution.
- michaelcampbell 5y agoTIL C needed saving.
- ncmncm 5y agoTIL C deserved saving.
- noobermin 5y agoIs there no way to make the asm labels be named specifically according to the return type + arguments? It doesn't fix the issues with changing order of fields/virtual functions (for C++) in types but it at least provides a "default" that may or may not be sane and I think fixes some of the problems that they're worried about and if that fails then sure bob go ahead and craft your own labels. That said, I think this idea is great, especially the fact that literally the libraries already craft their own labels, this just makes it a part of the language.
- int_19h 5y agoThis is exactly what C++ name mangling does. The problem is that introducing it is itself an ABI break.
- account42 5y agoAlmost, C++ name mangling does not include the return type even though it is ABI-relevant.
- noobermin 5y agoThat's specifically one of the examples they gave in the video the article references at the top. It wouldn't be breaking ABI if the symbol differs because that is the "correct" API breakage, one that callers have to be aware of, will generate an error when the application tries to load the shared library and does not find the matching symbol. The entire issue is when things change on the binary level but the application is not aware of it, and continues a subroutine call with wrong arguments or misinterprets the return value. Also, as the video states, the issue can happen even if the type of one of the arguments is the same but the fields in the type are rearranged, leading to a different offset during derreferencing.
- eternityforest 5y agoThat is a very long article that requires lots of other things to understand. But what I am confident about is that compatibility with everything is... just about the only good thing about C. Why mess with it at all? It's portable assembly. Why are we trying to use it as more than that? Even the kernel doesn't need it or only needs tiny bits of it. See Redox OS and all the other similar projects. Why do we still talk about ABIs? We can statically link a whole binary, and still use far less disk and ram than we do now with containers. We have high level languages that "link" at the level of names and modules, not bits and bytes. C works well enough for what it's used for. Rust and the others are up and coming. Instead of saving C, why not improve Rust and JavaScript and the others?
- gmfawcett 5y ago> Why mess with it at all? It's portable assembly. At your level and mine, it looks like portable assembly. At the level of the OP and the compiler implementations, it is absolutely not. > Instead of saving C, why not improve Rust and JavaScript and the others? "Platform effects" is a pretty complete answer. C isn't going anywhere soon. You and everyone here depends upon a mountain of C code every day.
- eternityforest 5y agoA mountain of old C that's still maintained because it's universal and ubiquitous. There's a few possible transition paths though. Like a backwards compatible strict superset or conversion tool. The sooner other languages mature, the sooner we can stop writing new projects that use large amounts of C.
- pjmlp 5y agoLike UNIX clones....so easier said than done.
- whatshisface 5y agoThe Zig compiler can compile C and seamlessly interoperates with it.
- Taniwha 5y agoI don't know what happens in the microsoft world, but here in unix-land we learned to address this issue way back when types started to work seriously in C (like the mid 1980s) we put a declaration of "do_stuff()" in a common include file (let's call it an "ABI definition") and include it into both the place where do_stuff is defined and where it is used - if one is different we expect the compiler to barf
- zozbot234 5y agoRight, and modern languages make it idiomatic to place even more stuff inside whatever their equivalent to a shared header is (in C++, it's either headers or modules) precisely because the raw C ABI has very limited semantics.
- dzaima 5y agoThe problem is that, as-is, there's no way to add a 128-bit integer type (which'd be very uselful!!) to the C standard (among other things like time_t or something), because that'd require changing intmax_t to 128-bit, and that'd break all existing dynamically linked code. The problem of being forced to have a "common [never ever ever ever possible to be in any way shape or form changed] include file" is exactly what this solves.
- whatshisface 5y agoWhy can't intmax_t stay intmax_t when bigintmax_t is introduced? What code actually needs to know the size of the largest integer type supported by the current compiler? Also, there's already the possibility that someone has defined a struct containing two ints to act as a 128bit integer, intmax_t is already sometimes smaller than the largest integer type. Is intmax_t supposed to be the largest integer in the standard or supported natively by the platform? If it's the second, leaving it unchanged when introducing larger ints wouldn't be a problem.
- dzaima 5y agohaving a "bigintmax_t" would...... work, but it's absolutely horrible and defeats the purpose of intmax_t being.. um.. the maximum integer type. A struct of two integers couldn't be used in regular & bitwise arithmetic, added to pointers, index arrays, cast up and down to other integer types, etc. As-is, you can pass in & out intmax_t as a default case and, worst-case, you just waste space. But "uint128_t a = ~UINTMAX_C(0)" not making a variable of all 1s bits would be just straight up broken.
- slaymaker1907 5y agoSo fun fact: if you use CGO, your binary effectively ends up with two versions of libc. It statically compiles musl into the binary itself for Go code and it dynamically links in libc for any C code. I'm not sure why there isn't an option with CGO to skip statically linking musl. You're depending on the platform libc regardless and at least then you end up with a smaller binary.
- terinjokes 5y agoCitation needed? Go makes system calls directly for its runtime, then yes you'll get a libc for CGO, but musl isn't involved in the former.