11 ms·
As a bit of historical context: "This is horribly inefficient BSD crap. Using these function only leads to other errors. Correct string handling means that y
by avar 3y ago
As a bit of historical context:
"This is horribly inefficient BSD crap. Using these function only
leads to other errors. Correct string handling means that you always
know how long your strings are and therefore you can you memcpy
(instead of strcpy).
Beside, those who are using strcat or variants deserved to be punished."
- Ulrich Drepper, around 23 years ago: https://sourceware.org/legacy-ml/libc-alpha/2000-08/msg00053.html https://sourceware.org/legacy-ml/libc-alpha/2000-08/msg00053...
- chris_wot 3y agoHe was a jerk, but often he had a reason for his abusiveness. Was the reason in this case valid?
- sanxiyn 3y agoYes.
- pnpnp 3y agoWhy?
- saagarjha 3y agoPeople typically do not realize that it has a return value that is expensive to compute.
- dezgeg 3y agoIt's not any slower in the typical case where destination buffer is large enough to fit the source thing. And if that's not the case then we are most likely in a error case (either caller notices the truncation and decides to abort, or ignores the truncation and things may soon go boom), and not many people care about optimizing error paths. Furthermore, when coders dont't have strlcpy() the alternatives are often even worse than strlcpy(): 1) They use strcpy() and have buffer overflows. 2) They use strncpy() which is slower than strlcpy() in the common (non truncating) case, and in the truncating case leave the string unterminated (thus segfault potential) 3) They use snprintf(dst, len, "%s", src); which is strictly slower than strlcpy()
- saagarjha 3y agoSince the error path is the largest one (the string doesn’t fit…) it makes sense to bound its execution. I would not recommend the others FWIW for exactly the reasons you mentioned.
- tedunangst 3y agoWhy would you optimize for the error case and not the common case? You've already done an unbounded amount of work copying the string in from the network or wherever. If anybody cared that much, they wouldn't let the string get that long in the first place.
- saagarjha 3y agoIt can be appropriate to bound the runtime of certain components of a system while allowing looser constraints elsewhere. For example I would perhaps not want to do an O(n) string operation on a collection of strings even though the user would be pretty upset if they can’t paste infinite input into my app.
- jstimpfle 3y agoIt's only as expensive as what you pass in. Joke's on you.
- saagarjha 3y agoqsort is also only as expensive as what I pass in. If it did a bubble sort internally I would be pretty upset though.
- sanxiyn 3y agoInefficiency probably doesn't need any comment (these functions traverse string twice instead of once). His argument that string length should be always known is correct in theory although not in practice.
- tedunangst 3y agoCan you name a program that runs too slowly because it uses strlcpy?
- saagarjha 3y agoYou're looking at it wrong. strlcpy is defined to be slow in certain cases. The API requires it. Other interfaces may be slow today but can be improved in the future because they don't have a return value that is inconvenient. (Notably, memccpy today is typically a memchr followed by memcpy, since this is faster than a naive implementation. Obviously if it gets used more then it will get replaced with a single-pass, machine optimized implementation.)
- skitter 3y agoAs the top level comment was about knowing the length of a string: GTA Online's loading times were atrocious because of a null-terminated string.
- dezgeg 3y agoNot really, more that the implementation of sscanf() is stupid and calls strlen() even though implementing sscanf() that doesn't require that is perfectly possible.
- jstimpfle 3y agoInstead of putting up with people constantly complaining how C is bad because of zero-terminated strings, we should better educate folks that there is absolutely zero reason why one has to rely on a NUL byte in-band signal. And APIs like sscanf() shouldn't be used beyond their historic purposes and there are easier ways to program. C doesn't really "have" zero-terminated strings other than supporting them with string literals as well as having an atrocious "strings" library for historical reasons. C has storage and gives you the means to copy data around, that's it. (Although I fully agree that the GTA issue can be seen as a bug in the implementation of sscanf()).
- IvyMike 3y agoThe question is: Is string truncation a good solution when the strings you have are unexpectedly long? Like, it's probably ok in a lot of cases, and once you start using these functions, it's very tempting to use them almost everywhere... but truncating "Attack at dawn on Friday" to "Attack at dawn" could be a disaster as well. On the other hand, his recommendation to always know string lengths and use memcpy didn't really become common practice over the last 20+ years either, so I'm not sure it was worth all the arguing. At this point, I'm kind of joining the camp of "C has proven to be too bug-prone for most organizations to use safely and therefore we should all go to Rust".
- saagarjha 3y agoI suspect the eventual end result is major compilers start implementing a "fat pointer" string ABI for internal translation units (decaying to char * at the edge where necessary) and people start turning that on.
- eru 3y ago> On the other hand, his recommendation to always know string lengths and use memcpy didn't really become common practice over the last 20+ years either, so I'm not sure it was worth all the arguing. It hasn't become common practice in C. But other languages (like JavaScript or Python) have become hugely popular, and don't use null-terminated strings.
- tomjakubowski 3y agoEven languages in C's niche encode strings as pointer + length, like Rust.
- bonzini 3y agoIt has in the sense that people allocate strings much more than using fixed-size, stack-allocated arrays. Modern C uses things like glib's GString, which (in addition to keeping the NUL terminator) track the length and can resize the underlying memory. And people also use a lot more asprintf instead of strcpy and strcat.
- 3y ago
- paulv 3y ago> but often he had a reason for his abusiveness There is never, ever, under any circumstances, a reason to be abusive.
- bufio 3y agoNo.
- arp242 3y agoNot really; he was frequently a jerk right out of the starting gates for no particular reason. That quote is the initial reply to the proposed patch, and the only "reason" I see for the insults is to satisfy Drepper's own emotional needs. It's petty and pathetic. This is very different from e.g. Torvalds who sometimes rants a bit after someone who he feels ought to know better screwed up. I'm not saying that's brilliant either, but I can be a lot more understanding when people are abrasive out of passion for their project after something went wrong.
- thomashabets2 3y agoWell, he does actually have a point. strlcpy is a faster (well, safer) horse than strncpy, but it's still a horse. We should not use horses as the main mode of transport anymore. "Doctor, it hurts when I strcpy — so don't do that". He's being a jerk about it, but I would not say that he doesn't have a point.
- arp242 3y agoMerely "having a point" is not "a reason for his abusiveness". I think I "have a point" for almost any HN comment I post (or at least, I'd like to think so) and have just as much "reason" to be a jerk as Drepper had. This applies to most posts on HN.
- thomashabets2 3y agoAh, true. I think I cross-read comments here. Sorry.
- yyyk 3y agoMostly no. True, the C NUL-terminated string definition is bad, but it's baked into the API. You need some semi-sane way to work with it that isn't 'Everyone writes their own wrappers to memccpy' (some people will get that wrong - e.g. the Linux manpage stpecpy wrapper just begs for misuse, and it's what most initiate C programmers will see if they know enough to check manpages). strlcpy may not be the best API, but it's sane enough and by now ubiquitous enough to deserve inclusion. Had glibc and others engaged we may have had a better API. Regardless, glibc should never have had such a long veto here.
- sanxiyn 3y agoHe is not wrong, is he? If you are using null terminated strings that's the thing you need to fix. I still support this addition. If you are doing methamphetamine with needle sharing you should stop methamphetamine, but distributing clean needles is still an improvement.
- parasti 3y agoWhat a crazy metaphor! You're equating using zero terminated strings in C to doing drugs.
- saagarjha 3y agoI feel like the success rate of getting someone off of null terminated strings is probably lower than most rehabilitation programs.
- IshKebab 3y agoWe can't entirely because of the C ABI but apart from that it's as simple as not using C which is not too difficult. C is not a popular language these days.
- masklinn 3y ago“Apart from that” does a lot of work here: FFI layers generally talk nul-terminated string unless otherwise specified, so do syscalls.
- IshKebab 3y agoYes that's what I said. You can generally wrap those layers so you aren't actually manipulating null terminated strings; just converting to/from them which is not too bad.
- chasil 3y agoWell, you will need to give up SQLite if you really feel this way, and reimplement it in a safe language. It will also be some time before Rust has substantial penetration into Linux; you might need to find a kernel that implements the POSIX interfaces safely. These will not be easy problems to solve.
- sillywalk 3y ago-edit- I'm not a C programmer, nor do I have any opinion on whether api is garbage or less worse or whatever. They seemed useful enough to get added to the other BSDs, Solaris, Mac OS X, Irix(!), QNX, and Cygwin as well as used in the Linux kernel.
- a_bonobo 3y agoHN discussion around this quote, around 12 years ago: https://news.ycombinator.com/item?id=2378013 https://news.ycombinator.com/item?id=2378013
- dobin 3y agoImho its pretty simple: Strings in C are 0-terminated char arrays. If the char array is not 0-terminated, its not a string. strncpy() can make a string into a non-string (depending on size), which is clearly bad.
- masklinn 3y agoThat’s because strncpy does not return a nul-terminated (“C”) strings, but a fixed-size nul-padded strings. That, as it turns out, is the case of most (but not all) strn* functions. Of course strncpy adds the injury that it’s specified to alllow nul-terminated inputs (it stops at the first nul byte, before filling the target buffer with nuls).
- lelanthran 3y agoIt also, in some situations, returns a *string" that doesn't have the null terminator, which means it is giving the caller something that literally isn't a string.
- masklinn 3y agoIt always “returns” the same thing: a fixed size nul-padded buffer. Call it a char array if you want, that’s always been it’s role and contract.
- alcover 3y ago> Strings in C are 0-terminated char arrays To be pedantic, they're pointers to char. Nothing more. Calling them array confuses non-C coders. The length is just an unenforced contract and has to be passed.
- TillE 3y agoIt's a pointer to a chunk of memory which contains an array of characters. You pass around the pointer because copying an array is expensive and wasteful. I think (or hope) the concepts are pretty clear if you understand what a pointer is.
- imtringued 3y ago> Correct string handling means that you always know how long your strings are Well, I couldn't think of a stronger argument against NULL terminated strings than this. After all, NULL terminated strings make no guarantee about having a finite length. Nothing prevents you from building a memory mapped string that is being generated on demand that never ends.
- jstimpfle 3y agoExcept that's a non-sequitur because you can totally keep separate string length fields. The only NUL that C requires is the NUL following C string literals, and you can even easily define char-arrays without NUL. char buf[5] = "Hello"; or even #define DEFINE_NONZ_STRING(name, lit) char name[sizeof lit - 1] = lit ""; Can also easily build pointer + length representations, without even a runtime strlen() cost. struct String { const char *buf, int len; }; #define STRING(lit) ((String) { (lit ""), sizeof lit - 1 })
- Natsu 3y agoWhat do you do when the strings might have more than MAX_INT characters?
- jstimpfle 3y agoWhat will you do on your 200th birthday? In case you're more interested in theory than practice, I have a different answer: I use a different API. However, I'm aware not even that could stop you, because you could still ask "what do you do when the strings might have more than SIZE_MAX characters?", which is entirely possible (as a combination of 2 or more strings). And to answer that, we're coming back to my original answer: It doesn't happen. I'm not calling the API with such huge strings. (And no, I usually don't keep formal proofs that it couldn't happen -- there are also an infinite number of other properties that I don't verify).
- GoblinSlayer 3y agoYep, and what if I want and to make an arbitrarily large array without much copying?
- olgeni 3y ago[flagged]
- loeg 3y agoI don't think OP intended this quote to glorify Drepper. He is correctly regarded as a giant asshole. Very smart, but also an awful person to work with.
- aumerle 3y ago[flagged]
- jacquesm 3y ago> Or just another do-nothing internet blowhard? I don't know about the OP but you are crossing the line here.
- runlevel1 3y agoBack in the 00s when Ruby was hot, the Ruby community had a remarkably constructive and helpful attitude. Even when offering criticism. Many folks attributed it to its creator with the acronym, MINASWAN ("Matz is nice and so we are nice"). No community is perfect, but once you've seen how good it can be it's hard to have much patience for brilliant assholes.
- jacquesm 3y agoI credit this more than anything for the success of Ruby. Just like I credit the 'holier than thou' attitude of the proponents of some other languages for their relative lack of success compared to where they could have been by now. Dutch proverb, not sure if it translates or if there is a better English version: you catch more flies with sugar than with vinegar.
- sfink 3y agoThe English version is "you catch more flies with honey than with vinegar", which at least in English makes more sense, since in English "sugar" generally implies dry granulated sugar. You're not going to catch any flies with that. (Ironically, you'd probably catch more with the vinegar, since some would go to it for the moisture and a few would drown.) /tangent
- pantalaimon 3y agoI'm surprised they didn't go with strscpy() directly https://archive.kernel.org/oldlinux/htmldocs/kernel-api/API-strscpy.html https://archive.kernel.org/oldlinux/htmldocs/kernel-api/API-...
- GoblinSlayer 3y agoBecause strlcpy exists in bsd since 1999: https://man.netbsd.org/strlcpy.3 https://man.netbsd.org/strlcpy.3
- deleted 3y ago[deleted]
- AstralStorm 3y agoOr more likely strncpy plus forced last NUL. Return a flag on truncation unlike messing with return code or errno. Call it safe_strncpy and be done with it. Otherwise asprintf and snprintf exist. strlcpy is a more garbage version of snprintf.