5 ms·
No way to parse integers in C (2022)
- zokier 4mo agoI thought it was pretty well known that everything related to strings in C stdlib (including all str... functions) is bad. You just need to bring in your own string library.
- bhk 4mo agoNot just the string-related functions. If you want robust error checking, re-entrant code, and bounds checking performed in library functions (instead of performing bespoke validations all across your code base), you have some work to do. Yes, some improvements have been tacked on over the years, but many problems ("current locale", for one) remain endemic. In my experience, the worst part of the C standard library is not its existence, but the fact that so many developers insist on slavishly using it directly, instead of safer wrappers.
- voidUpdate 4mo agoCant you just: for(int i = 0; i < len(characters); i++) { if(characters[i]-48 <= 9 && characters[i]-48 >= 0) { ret = ret * 10 + characters[i] - 48; } else { return ERROR; } } return ret; Adjust until it actually works, but you get the picture.
- bitwize 4mo agoYou cannot "just" anything in C without hitting a minefield of UB. It is, probably, more economical to convert your entire project to Rust than it is to do the pufferfish spine removal procedure of auditing the code base for UB and replacing the problem areas. With generative AI, the size of project for which this remains true may be as large as "the entire Linux kernel".
- Sharlin 4mo agoAnd how does this avoid returning nonsense if the number is too large? (Wrapping if the accumulator is unsigned, straight to UB land if signed.) Not reporting overflows as errors is one of the major problems demonstrated by TFA.
- voidUpdate 4mo agoyou could check if ret > ret * 10 + characters[i]-48, if so it has wrapped around and you return an error
- Thiez 4mo ago[dead]
- thomashabets2 4mo agoFor unsigned that could work, but signed overflow is UB.
- knome 4mo agothis wouldn't catch overflow or underflow errors, nor does it allow non-base-10 numbers, nor does it handle negative numbers. and writing your own parser is a failure case by op's logic. they are complaining about the builtin parsing functions. the author admits you can parse signed integers in their second example, but for unsigned, they don't like seem to like that unsigned parsing will accept negative numbers and then automatically wrap them to their unsigned equivalents, nor do they like that C number parsing often bails with best effort on non-numeric trailing data rather than flagging it an error, nor do they like that ULONG_MAX is used as a sentinel value by sscanf. I'm not sure what they mean by "output raw" vs "output" $ cat t.c #include <stdlib.h> #include <math.h> #include <stdio.h> int main(int argc, char \* argv){ char * enda = NULL; unsigned long long a = strtoull("-18446744073709551614", &enda, 10); printf("in = -18446744073709551614, out = %llu\n", a); char * endb = NULL; unsigned long long b = strtoull("-18446744073709551615", &endb, 10); printf("in = -18446744073709551615, out = %llu\n", b); return 0; } $ gcc t.c $ ./a.out in = -18446744073709551614, out = 2 in = -18446744073709551615, out = 1 $ I get their "output raw" value. I don't know what their "output" value is coming from. I don't see anywhere they describe what they are representing in the raw vs not columns.
- card_zero 4mo agoI think "output" is just supposed to be a human-readable version of "output raw". So the line in the table where "output raw" is 2 but "output" is 1 looks like a mistake. It's repeated in the table for sscanf().
- thomashabets2 4mo agoYup. Sorry about that.
- thomashabets2 4mo ago> they don't like seem to like that unsigned parsing will accept negative numbers and then automatically wrap them to their unsigned equivalents, nor do they like that C number parsing often bails with best effort on non-numeric trailing data rather than flagging it an error, nor do they like that ULONG_MAX is used as a sentinel value by sscanf. That's right. I don't like asking it to parse the number contained inside a string, and getting a different number as a result. That's just simply not the right answer. > I'm not sure what they mean by "output raw" vs "output" I can see how that's very unclear. Changed now to "Readable".
- fhdkweig 4mo agoWhat if the number you want to return just happens to be the value of ERROR? You need an error flag that can't be represented as an int, but then C wouldn't let you return it from a function that only returns "int". It is why some languages throw exceptions and why databases have the special "null" value.
- voidUpdate 4mo agoI don't use C enough to know what the convention is for throwing an error when the function can return a number anyway. You'd have to ask someone else
- zbentley 4mo agoIn C, errors are usually indicated by a negative return value constant, crashing the program with abort, or setting the errno global (thread-local, but whatever) and expecting callers to check it. Sometimes multiple of those.
- QuercusMax 4mo agoOne reasonably common pattern is to have the return value indicate success / error, and you pass in a pointer to the value which will be mutated if successful.
- jamesfinlayson 4mo agoYep, lots of the Windows API does it this way.
- jerf 4mo agoAnd why some very, very special languages have an effectively-global variable called "errno" that you have to check after the call manually, and worry about whether maybe it was populated from some previous error. Nothing says "production-quality language that an entire civilization's code base should be based on" like "sometimes (but only sometimes!) functions return additional information through global values".
- dlcarrier 4mo agoHere's a readability tip for working with ASCII numbers: Treat adding and subtracting the ASCIIness as you would multiplying and dividing by a unit in physics. You can add '0' to convert a numeral to ASCII and subtract '0' to convert it back, and you can do direct comparisons between ASCII numerals. if(characters[i] <= '9' && characters[i] >= '0') { ret = ret * 10 + characters[i] - '0'; }
- deleted 4mo ago[deleted]
- voidUpdate 4mo agoI was trying to remember how to do that, I forgot you can subtract '0', and was thinking that - 0 obviously wouldn't work
- bsenftner 4mo agoOne of the first homework assignments when I learned C back in '83 was after a long lecture on how the string functions are fundamentally broken, and the class introduction to writing C was fixing all of them.
- psvv 4mo agoMy memory growing up is that making your own C library was basically an inevitable rite of passage for any aspiring programmer.
- prerok 4mo agoYeah, it's a shame we never got something like boost for C. Every company I ever worked for had its own common C library solving these problems.
- ndesaulniers 4mo agoIt's a shame we never got a package manager for C (or C++). EDIT: perhaps I should have been clearer; by not having one early on, we now have multiple competing package managers, with no clear winner. Responses prove that point.
- bsenftner 4mo agoI worked at a shop where we used Boost in a C++ code base that the only use of C++ was the harness to use Boost. After that, it was all C, object-styled C, as that code base started before C++ compilers were not a template overlay on C.
- stephc_int13 4mo agoAs a C programmer, I find this kind of bad faith article very irritating. Yes, the standard library is bad. This is by far the worst part of the C legacy. But it is not that hard to write your own. String functions like this are not difficult at all, and you can use better naming and semantics, write faster code etc. C is not the C standard library, ffs.
- konmok 4mo agoI don't think it's in bad faith. The distinction between a language and its standard library gets blurry even in theory, and in practice they're nearly inseparable. If a language's standard library has four ways of doing almost the same thing, and they're all fundamentally broken, that's a problem.
- dosisking 4mo ago[flagged]
- stephc_int13 4mo agoIf you read the other articles by the same author on his blog, you'll see that he has some strong and weird opinions about C and UB. Complete BS in my opinion.
- alexfoo 4mo agoExactly. A wrapper that handles all of the edge cases properly and gives proper reporting just gets added to your own library of functions and the devs get used to using it. Much like the code for abstract data types like lists/hashmaps/etc which neither C nor the standard libraries provide. Bonus points for having bespoke linting rules to point out the use of known “bad” functions. In one old project we went through and replaced all instances of sprintf() with snprintf() or equivalent. Once we were happy that we’d got every occurrence we could then add lint rules to flag up any new use of sprintf() so that devs didn’t introduce new possible problems into the code. (Obviously you can still introduce plenty of problems with snprintf() but we learned to give that more scrutiny.)
- alexfoo 4mo agoI remember an old project that ran into something like this. I think we just used atoi() or similar and the error check was a string comparison between the original input and a sprintf() of the converted value. Ugly (and not performant if in a hot path) but it works.
- ramon156 4mo agoWhy not look at how other languages attack this? e.g. how does "42".parse() work in rust? Edit: https://doc.rust-lang.org/src/core/num/mod.rs.html#1537 https://doc.rust-lang.org/src/core/num/mod.rs.html#1537 interesting! It boils down to this pub const fn from_ascii_radix(src: &[u8], radix: u32) -> Result<u32, ParseIntError> { use self::IntErrorKind::*; use self::ParseIntError as PIE; // guard: radix must be 2..=36 if 2 > radix || radix > 36 { from_ascii_radix_panic(radix); } if src.is_empty() { return Err(PIE { kind: Empty }); } // Strip leading '+' or '-', detect sign // (a bare '+' or '-' with nothing after it is an error) // accumulate digits, checking for overflow Ok(result) }
- marcosdumay 4mo agoIt's not an overwhelming hard problem. There are some issues with radix signaling, exponent notation, decimal points being allowed or not, and group separators that make parsing numbers incredibly irritating. So you usually don't want to do it yourself. But it's not hard at all. It's not even as full of small issues that you can't handle the load, like dates. It's just annoying as hell. The problem is exclusive to C and C++. It's created by the several rounds of standardization of broken behavior.
- eithed 4mo agoCan't you regex that given string contains just numbers and then use any of the provided methods? Then check if the returning value is a number to cater for edge cases Ok, having a method to do that for you would be nice, but the post reads like it's an issue that std library doesn't provide you with a method behaving as you exactly want
- jervant 4mo agohttps://man.openbsd.org/strtonum https://man.openbsd.org/strtonum
- bmandale 4mo agoInterestingly fails as well, in two ways. First: > The string may begin with an arbitrary amount of whitespace (as determined by isspace(3)) Second is that it only applies to signed long long, not unsigned.
- chadgpt3 4mo ago... say users of only language with no way to parse integers. :)
- CodesInChaos 4mo agoAnother case many integer parsing functions get wrong is that they interpret a leading 0 as an octal indicator. That should be opt-in via a flag, if it needs to be supported at all. Unix file permissions are the only deliberate use of octal I've ever seen.
- kevin_thibedeau 4mo agoIt used to be much more common. In the 70s there was a lot of collective hesitance to use hex with its strange letter digits. Octal was the compact representation of choice.
- adrianmonk 4mo agoAlso, some very old computers had 36-bit words. Word sizes on modern computers are virtually always powers of 2, but it hasn't always been that way. And octal is more convenient for output via 7-segment LEDs and for input via numeric keypads.
- orthoxerox 4mo agoI wasn't in this class myself, but one prof at my alma mater started his "Programming 201" class with the simplest assignment: write a C program that accepts two integers from the user and prints their sum. It actually was the only assignment for the rest of the semester, since he has a test suite that would humiliate the students gently at first, but would ultimately pipe a billion nines into stdin as the first argument.
- clark_dent 4mo agoCould you humor a coding noob--how do you deal with utterly insane inputs like that?
- doubled112 4mo agoCrash and report an error.
- chowells 4mo agoYou report an error and exit cleanly with a proper operating system error code. Crashing is a quick hack, acceptable for throwaway projects but not in software used long-term.
- SoftTalker 4mo agoCrashing (in the sense of "give up and exit with an error") on invalid inputs is valid (and often the best thing) in many cases. Fix your inputs.
- chowells 4mo agoI think you're using "crash" to mean "exit early". I am using "crash" in the sense of "this program did something causing the OS to terminate it externally". I suppose that's a real point of difficulty in communication across different programming languages. We agree that the program should exit early. I think we agree it should do it cleanly and intentionally. I'm adding the constraint that "crash" doesn't necessarily mean "cleanly and intentionally", especially when talking about a C program.
- contubernio 4mo agoOne of the great virtues of C is that this sort of thing is not part of the language ...
- thomashabets2 4mo agoOnly literally. 7.24.1 in the C programming language spec has these poor parsers.
- rbanffy 4mo agoIs their misbehavior part of the spec as well? If not, we can always add the correct behavior to the spec and let anyone who implemented a broken version deal with fixing every program compiled using it.
- thomashabets2 4mo agoFair enough. For strtoul and friends, maybe? 7.24.1 is pretty dense, but the key parts are "the expected form of the subject sequence is a sequence of letters and digits representing an integer with the radix specified by base, optionally preceded by a plus or minus sign […] If the correct value is outside the range of representable values […] ULONG_MAX […] is returned". So the "expected form" allows a minus sign, but then it's clearly "outside the range of representable values" for strtoul to try parsing a negative value. So maybe it should return ULONG_MAX on those. So arguably a minus sign present could already be treated as an error, and still be standard compliant. Unless I'm misreading.
- rbanffy 4mo agoPassing a negative value to a function that is specifically for converting strings into unsigned numbers is pretty much an error. In the case of functions that return an unsigned number, at least, negative return values can represent errors. It’s more fun when the result can be signed though. Maybe strcmp with the representation of the LONG_MAX, and if it doesn’t match, call strtol and watch for a LONG_MAX indicating an error. C is a bit messy. Would be nicer to return a struct with a possible error and the desired value, Golang style.
- deleted 4mo ago[deleted]
- norir 4mo agoThis is not a hard thing to do without using a library. The code below is easily adapted to the unsigned case and/or arbitrary base rather than 10. #include <stdio.h> int main(int argc, char **argv) { if (argc != 2) { fprintf(stderr, "usage: require one numeric argument"); } char *nump = argv[1]; unsigned neg = 0; unsigned long long ures = 0; if (*nump == '-') { neg = 1; nump = nump + 1; } if (!*nump) { fprintf(stderr, "require non empty string\n"); return 1; } char b; while (b = *nump++) { if (b >= '0' && b <= '9') { unsigned long long nres = (ures * 10) + (b - '0'); if (nres < ures) { fprintf(stderr, "overflow in '%s'\n", argv[1]); return 1; } ures = nres; } else { if (b >= ' ') { fprintf(stderr, "invalid char '%c' in '%s'\n", b, argv[1]); } else { fprintf(stderr, "invalid byte '%d' in '%s'\n", b, argv[1]); } return 1; } } long long res = (long long) ures; if (neg) { if (ures <= 0x8000000000000000ULL) { res = -res; } else { fprintf(stderr, "underflow in '%s'\n", argv[1]); return 1; } } else if (ures > 0x7FFFFFFFFFFFFFFFULL) { fprintf(stderr, "overflow in '%s'\n", argv[1]); return 1; } fprintf(stdout, "result: %lld\n", res); return 0; }
- wCxV8HzziQBb 4mo agoThe bound on ures <= 0x80[...] should be either ures < 0x80[...] or ures <= 0x7F[...]. Otherwise, parsing negative `0x8000000000000000` will run code to negate the signed integer INT64_MIN (-0x80[...]) to 0x80[...], which doesn't fit in an integer (INT*_MAX is 0x80[...]). $ clang parseint.c -fsanitize=undefined -O0 -g -o parseint $ ./parseint -9223372036854775808 parseint.c:38:23: runtime error: negation of -9223372036854775808 cannot be represented in type 'long long'; cast to an unsigned type to negate this value to itself result: -9223372036854775808 edit: this is just to show that getting undefined behavior right is hard!
- lacewing 4mo agoThere's no one correct way to parse integers. Do you want to support 0x prefixes? Is a leading zero an indicator or octal, a zero-padded decimal, or a syntax error? Are you willing to accept a leading "+"? Are leading whitespaces OK? Trailing ones? Is 0x0c a whitespace? What about all the weird Unicode ones? Do you allow exponential notation (1e1)? Etc, etc. In every language, the standard library makes some assumptions about this. In JavaScript, an empty string parses to zero. The standard C library, which dates back to the stone age, does the simplest thing you can do without range checking, because, well, that's kinda the C paradigm. If you want parsing that handles edge cases in a specific way, you do it yourself. It's just digits.
- mike_hock 4mo ago> There's no one correct way to parse integers. No, but there are a myriad of incorrect ways and the C library's way is one of them. It's perfectly fine to make reasonable choices for all those options and then implement them correctly.
- fastaguy88 4mo agoAnd yet, thousands and thousands of 'C' programs parse integers every hour successfully. Perhaps the right title should be "No way to parse pathological edge cases in 'C'" And then see how other languages do.
- derefr 4mo ago> It is not OK to stop at the first sign of trouble, and return whatever maybe is right. “123timmy” is not a number, nor is the empty string. None of the C functions referenced (atol, strtol, sscanf) are number-parsing functions per se. Rather, they're numeric-lexeme scanning+extraction functions. These functions are all designed to avoid making any assumptions about the syntax of the larger document the numeric lexeme might be embedded in. You might, after all, be using a syntax where numbers can come with units on the end. Or you might be reading numbers as comma-separated values. And, as a key point the author might be missing: C, in being co-designed with UNIX, offers primitives tuned for the context of: - writing UNIX CLI tools that work with unbounded streams of input (i.e. piped output from other UNIX CLI tools), - where, crucially, the stream is just text, and so carries no TLV-esque framing protocol to tell you the definitive length of a thing; - and nor (especially in early memory-constrained systems) are you able to perform allocations of heap memory in order to employ an unbounded growable buffer for retaining the current lexeme until you do reach the end of it (which, if you could, would let you use a scanner state-machine that doubles as a parser/validator, returning either a parsed value or an error) - but instead, to deal with the 1. unbounded input, 2. of textual encoding, 3. in constant memory, you must eagerly scan the input stream (i.e. synchronously reduce over each received byte, or at most each fixed-length N-byte chunk using a static or stack-allocated fixed-length buffer, discarding the original string bytes once reduced-over) to produce lexically-decoded (but not parsed/validated) lexemes; and then do this again, on a higher level, feeding your stream of lexemes into a fixed-sized sum-typed ring-buffer (i.e. an array-of-union-typed-lexeme-struct-type-entries), where you can then invoke a function that attempts to scan over + consume them (but unlike the original stream-parsing function, doesn't consume the buffer unless successful, and so isn't functioning as a scanner per se, but rather as an LR parser.) If you're not writing UNIX CLI tools, direct use of the C-stdlib numeric-lexeme scan functions is operating on the wrong abstraction layer. What you want, if you have pre-framed strings that are "either valid numbers or parse errors", is to implement an actual parsing function... that can then invoke these numeric-lexer functions to do the majority of its work. And if you're writing C, and yet you're not in UNIX-pipeline unbounded-text-stream land, but rather are parsing well-defined bounded-length "documents" (like, say, C source files)... then you probably want to use a real lexer-generator (like flex) to feed a parser-generator (like yacc/bison). Where: - you'd validate the token in context, in the parsing phaase; - and your lexing rules would make certain classes of input invalid at lexing time. (E.g. you can write your lexeme matching rules such that multi-digit numbers with leading zeroes, or floating-point values with no digits before/after the decimal place, simply aren't "numbers" from your lexer's perspective.) ...which means that, once again, you can "get away with" invokeing the regular C numeric-lexeme scanner functions; i.e. `yylval = atoi(yytext);` in bison terms. (And you'd want to, since doing so saves memory vs. keeping the numbers around as strings.)
- mike_hock 4mo agoThe problem is that float parsing is highly non-trivial if you want it to be correct for all edge cases. For integers, you're faster (in both development time and runtime) to write your own parser than to try and assemble the pieces in this pile of shit into a half-working one. C++17 from_chars excluded. Incidentally, 2022 seems about right for the year that ONE open source implementation finally actually implemented the float part of that. Or was it more like 2024?
- alkonaut 4mo agoHow could an api for number parsing ever be designed to return 0 for invalid input, for a function where 0 is also a common (perhaps the most common) return value for a valid input? This wouldn't even pass a cursory sanity check of the api from a beginner developer, how did it end up in a standard library at all? Was it a mistake and then it was just too late to remove it? Any function that can either succeed or fail, which is basically every parsing function, must typically indicate success or failure. You can terminate the program or you can return an object that itself indicates failure (such as -1 when finding a positive index) but if ALL values of the return type CAN be valid then the success state must be a separate return value. What's the purpose of the function atol() if it doesn't have that? Is it "It's still useful for trusted input we know is a string representation of a long" (E.g. for bounded number roundtrip)? That seems awfully limited. But perhaps such a scenario was perhaps more common in 1960?