10 ms·
A Special Kind of Hell: intmax_t in C and C++
- db48x 6y agoThankfully you don't actually have to write code in C any more.
- klyrs 6y agoMaybe you don't. What of the Linux kernel, Python language, and countless other tools our world is built upon?
- the8472 6y agoI don't think the linux kernel is a good example. They define their own internal ABI. There are no libraries. They don't try to be portable to exotic compilers, in fact only GCC is officially supported. They even get to opt out of things like signed overflow being undefined behavior.
- Gibbon1 6y agoWell signed overflow being undefined is something that needs to be fixed.
- saagarjha 6y agoSigned overflow being undefined enables a broad range of loop optimizations in a language that has no other way of allowing these to exist.
- inetknght 6y agoSigned overflow should be defined and a platform that provides a normal int which has different overflow behavior should have their int named different. Maybe noverflowed_int or
- saagarjha 6y agoAgain, this would kill many loop optimizations.
- inetknght 6y agoNo it wouldn't. It would mean loop optimizations shouldn't use `int` as it's defined. It means they should use whatever other integer type _does_ do what the optimization wants. And if that means creating a new integer type then so be it.
- Gibbon1 6y agoExactly, if the coder thinks those optimizations will buy him anything then let them enable them for that one case. And stop pretending that micro benchmarks matter for the rest of the 99.9% of code out there. Last time I looked at the speedup you got from loop optimization it at most a few percent for a very simple loop. For a loop that has some meat it'd be noise.
- fooker 6y ago> Last time I looked at the speedup you got from loop optimization it at most a few percent for a very simple loop. Wow, no. Most loop optimizations make loops significantly faster, even an order of magnitude is not uncommon. Write a few loops and compare the code generated by any compiler with O0 vs O3.
- UncleMeat 6y agoDefining signed overflow requires sticking more instructions for every integer addition, since the platform is not guaranteed to actually overflow when you want it to so the program will have to detect this and force the overflow to happen. Major slowdowns. Lots of loop stuff also depends on non-overflowing signed integers. If integers can shrink on addition then you cannot safely do a lot of unrolling or fancier stuff based on affine loop transformations.
- jcranmer 6y ago
- johnisgood 6y agoUmm yes, yes people do. Thankfully. I like C. Learn string handling and pointers, go through https://wiki.sei.cmu.edu/confluence/display/c/SEI+CERT+C+Coding+Standard https://wiki.sei.cmu.edu/confluence/display/c/SEI+CERT+C+Cod... and you will write relatively safe programs. Oh, and read the man pages. I do not know who was against man pages, but God, man pages are a God-send. It must stay. I never liked info though. --- In any case, I write in Forth, Ada, Go, and JavaScript (Vue.js). Not much C these days. If someone asks me a simple program to write to extract data from a format specified by him, C is what I would choose; mainly: structs. I could do it in Forth, but easier in C, he has GCC, I can create a binary easily, etc.
- saagarjha 6y agoPerhaps I’m misunderstanding the problem here, but isn’t the solution here that a libc must carry multiple ABI-compatible functions and pick the right one based on metadata in the binary? Is there anything specific here that applies only to intmax_t or is this just a general issue with ABIs?
- mulle_nat 6y agoThe solution is to recompile, which you have to do with such ABI changes, since the dawn of time.
- toast0 6y agoI would expect it will work a lot better if you have one libc library per ABI, and link in the matching one to the executable (and the same for all your other libraries). The only thing that may need to deal with multiple supported ABIs simultaneously is the kernel. But maybe kernels are using more explicit types? Personally, I don't like to write code with int/long/etc. I want to know how many bits are in the thing, because it usually doesn't make sense for the program to sometimes have one number of bits, and sometimes another (pointer/memory offsets excluded).
- loeg 6y ago> The only thing that may need to deal with multiple supported ABIs simultaneously is the kernel. But maybe kernels are using more explicit types? Right. Kernel ABIs use explicit-width types, and if an ABI change is needed, a new syscall is added. The old one continues to exist with the old ABI indefinitely.
- loeg 6y agoYeah, I think you can handle this with symbol versioning. Which is a sort of hand-coded linker magic for symbol mangling that the author complains about. But I don't buy that it's the major problem the author claims.
- WalterBright 6y agoAny language 50 years old is going to have problems like this. C has done rather well.
- m463 6y agoI don't know how D handles it, but I got an extraordinary "all is right with the world" years ago when I saw ada with common definitions for bit order, max integers and digits.
- WalterBright 6y agoIn D, the only size that varies is pointer size. An int is 32 bits, a long is 64. Makes life much easier.
- colejohnson66 6y agoSame with C#. int is an alias for System.Int32 and long is an alias for System.Int64. (Or is it the other way around?) Just don’t do what Java does and make Integer a class and int a value type. Then you end up with references and the “integer cache” garbage. https://viveksb007.github.io/2020/06/integer-cache-in-java https://viveksb007.github.io/2020/06/integer-cache-in-java
- teddyh 6y agoThe pain seems to originate from the wish to keep one ABI for an architecture even though the architecture is mutable and constantly changing. The intmax_t type was made to represent the largest signed integer type on an architecture. If you modify the architecture to support a new, larger, integer type, you have now created a new architecture, with a new ABI. If you do this, there is no problem with intmax_t. But of course, ABI’s are important, since proprietary binary blobs are what is important.
- samatman 6y agoABIs are important even in the absence of proprietary binary blobs. They allow different compiler chains to interoperate, they're the basis of FFIs, there are so many situations where I have the source to both sides of some pipe-fitting operation and having a common ABI is what makes my job possible.
- teddyh 6y agoABIs are important within an architecture. But there is no good reason to stretch an ABI over too many variants of the same architecture, except to preserve compatibility with a single binary.
- stefan_ 6y agoIt seems the original sin was having an intmax type in the first place, instead of only fixed-size types. And then of course the stupidity in using it: just because a high-end CPU can do math on 512 bit integers within some 10-20 cycles shouldn't mean that after a recompile all the chrono code is suddenly a huge register stomping inefficient mess, for no good fucking reason at all. And on the other end of the spectrum, someone might well decide 16 bit is the intmax for an AVR8. Now your chrono code doesn't work at all.
- saagarjha 6y agoBut if you used a fixed size type, you end up doing things like using 64-bit integers on AVR, where you’re going to again have a bad time. Flexible and fixed width types both have their uses.
- PaulDavisThe1st 6y agoThis, this and more this. intmax_t is as bad as int, for precisely the same reason. The developer has no idea what the capabilities of the type are. "Give me the biggest integer type you have" is not something any developer should say, ever, at least not in code that is intended to be remotely portable across time, space and architecture.
- SloopJon 6y agoI used intmax_t just yesterday in an ODBC type conversion test. I wrote a template for use in a Google Test typed test to find the highest and lowest values supported by the client and server type. Something like: max( static_cast<intmax_t>( numeric_limits<T>::lowest()), static_cast<intmax_t>( numeric_limits<U>::lowest())) I tried std::common_type first, but it didn't quite do the trick.
- secondcoming 6y agoOut of curiosity... did you ever fall into the trap of using numeric_limits<T>::min() instead of numeric_limits<T>::lowest() ? Because I did.
- simias 6y ago>The glibc maintainers decide they’re going to change from long long as their intmax_t and move to __int256_t for most platforms, because most platforms support it and they have a lot of customers asking for it. Is this a practical concern? That seems like such a strong calling convention change that it wouldn't really bother me if you had to recompile all your software in this situation. I mean, let's even imagine that there would be a way to make this change somewhat transparent and automated. You still end up with some types magically now taking 256 bits instead of (probably) 64bits before that. If you use this type a lot the performance implications are pretty huge, both in terms of actually doing maths with a 256 bit integer, but also for RAM and cache usage. If you suddenly changed int to be 64 instead of 32bits you'd have a similar problem for instance. I don't think anybody would expect such a change not to require a full recompile. So what's the difference here?
- stncls 6y ago> Is this a practical concern? > That seems like such a strong calling convention change I think you are spot on here. The OP's concern may be due to things like this: > For example, let’s take a C Standard function that uses intmax_t, imaxabs It looks like an unusual departure from typical libc functions like strtol(), strtoll(), etc. which have a primary type right in the name. To the untrained eye, intmax_t may look like some promise of portability, when it is actually no more portable than specifying int or long.
- loeg 6y ago> If you suddenly changed int to be 64 instead of 32bits you'd have a similar problem for instance. I don't think anybody would expect such a change not to require a full recompile. So what's the difference here? intmax_t is supposed to store any sized integer type. If you introduce a int128_t, it doesn't have any impact on 'int.' But it does mean that your intmax_t must be int128_t, and if it was 64-bit before, now your ABI changes (for intmax_t-using ABIs). Ordinarily introducing a new type or function wouldn't cause ABI incompatibility, but introducing a wider integer type that intmax_t cannot represent is not a valid choice. I'd suggest considering ABIs with intmax_t in them unstable, and probably just avoiding use of intmax_t in ABIs entirely. Integer size-generic routines are best served with wrapper macros and use of the C11 _Generic() construction, or some of the typeof() magic used by, e.g., Linux.
- CaliforniaKarl 6y agoI’m confused by one line from the first code block in the “A Demonstration” section: intmax_t val = imaxabs(val); This is the first time `val` is defined in the block, so wouldn’t this raise a C compiler compilation error? If not, then I don’t understand the effect of this line.
- SAI_Peregrinus 6y agoIt should be intmax_t val = imaxabs(original); for the snippet to make any sense.
- jcelerier 6y agotechnically, int x = x; compiles, but is UB - you can turn it into a compile error with `-Werror=uninitialized` in decent compilers.
- mh7 6y agoIf you change the size or padding in a struct of course a dynamic library is gonna break every program using it, I'm not sure why intmax_t was singled out here.
- saagarjha 6y agoAnd that can happen after just a recompile.
- flohofwoe 6y agoInteresting read but IMHO fairly irrelevant for language users. I've been coding in C and C++ for more than 25 years and never encountered intmax_t or any problems related to it. It's better to accept that C (and C++) are only portable in theory, in reality it's nearly impossible to write non-trivial code which compiles without warnings in a different compiler or runs out of the box on a different hardware architecture (still the language does a pretty good job at it everything considered). IMHO the standard committees should focus on the (handful) platforms that exist today, and not care too much about legacy or future platforms. The C standard shouldn't be some "pure ideal", just harmonize the differences between compilers and platforms, and only so far as it makes sense for real-world code. Personally I'd rather see extensions from gcc and clang moved into the standard which are already battle proven instead of wasting time fixing 30 year old problems.
- jcelerier 6y ago> Personally I'd rather see real-world extensions from gcc and clang moved into the standard which are already battle proven instead of wasting time fixing 30 year old problems. there's nothing that prevents you from using clang on all platforms, which will be exactly the same experience that if you used $LANG_WITHOUT_AN_ISO_STANDARD
- flohofwoe 6y agoOh but there is :) As a library author I really don't want to dictate users what compiler to use, they should at least be free to choose between the "big 3": gcc, clang and MSVC. Having those 3 compilers agree on more language features would be very helpful (this doesn't even need to happen through the standard, it would be enough if gcc, clang and MSVC would agree on a handful of important non-standard extensions).
- enriquto 6y agoicc is arguably bigger than msvc (which is not even a modern C compiler)
- comex 6y agoFor C++, you don’t need that fancy struct. You can just use inline namespaces, which were specifically designed as a portable form of symbol versioning. But making the standard library work with a changed ABI is the easy part. The hard part is every other library. Users expect to be able to compile libraries (both static and dynamic) and then keep using the binary artifacts even after upgrading their operating system. Admittedly, most libraries won’t use intmax_t in their public API, but if one does, changing the definition of intmax_t will cause the same types of ABI breaks as with the standard library. And unlike the standard library developers, those users probably won’t know how to fix them, nor appreciate being made to.
- dvfjsdhgfv 6y agoFirst, you are not forced to use intmax_t (unless of course you inherit it from someone else). Second, in my personal opinion, it's really much safer to use C89/C90, unless you have very good reasons to use the extensions present in the newer versions of the standard. Even large projects like SQLite appreciate this approach. And if you are developing for an environment supporting C99 and newer standards, you could as well use C++ (or a newer language).
- dvfjsdhgfv 6y agoTo downvoters: please show me one scenario where intmax_t is indispensable.
- bhk 6y agoThis article presents as a language flaw what is really an operating system flaw. Exposing system functionality through dynamically-linked C libraries, and using C features that do not guarantee ABI compatibility, results in an OS interface that does not provide backwards compatibility. When I ran into similar problems trying to distribute a binary for Linux users, I thought "Screw it, I'll just statically link everything." But the older statically linked C libraries would crash on newer kernels, and the newer static C libraries would refuse to run on older kernels. (Or do I have that backwards? Anyway, I ended up concluding that Linux doesn't really have a stable ABI.) Now, if you think "operating system" == "kernel", then, yes this is a language problem. And yes, people who write in x86 assembly and directly invoke the Linux kernel don't have all of these worries. They have much more manageable backwards compatibility issues. But for most of us, "programming to Linux" means using the APIs defined in man pages. Perhaps the actual "OS API" should be more completely divorced from C library issues. The kernel should define its own contract in some kind of IDL from which one can generate APIs for any language (including all of the variants of C that differ by implementation choices). Yes, it should map to C without any impedance mismatch, and the C folks can deal with how exactly it maps to different implementations of C, but other language runtimes shouldn't have to go through C and re-live all its problems.
- Shorel 6y agoI am thinking the best bet for achieving this kind of backwards compatibility is to make Windows binaries and test them with Wine.
- pjmlp 6y agoThat is an UNIX problem, not every OS uses C as the OS ABI, e.g. Android, Windows, IBM i, z/OS, ... Also given the target domain of C, the standard also must take into account possible free standing deployments, where an OS doesn't even come into play.
- colejohnson66 6y agoI thought Windows does use C as an ABI?
- johnisgood 6y agoKind of unrelated: the font is a bit too high. I decreased it though.