7 ms·
I'm not a fan of Unix man page section numbers in parentheses. strlcpy is a stopgap, whack-a-mole solution for buffer overflows. It is rationalized by the reas
by kazinator 2y ago
I'm not a fan of Unix man page section numbers in parentheses.
strlcpy is a stopgap, whack-a-mole solution for buffer overflows. It is rationalized by the reasoning that it does not make the program less wrong, while (probably) making it more secure.
When truncation matters and you have a fixed size buffer, that buffer should be large enough in order for it to be justifiable to say that someone is misusing the application. Perhaps a tester trying to break it.
Nobody’s surname needs 128+ bytes. No reasonable URL for a firmware update download needs 4096 bytes.
If truncation matters, no, it does not always make sense to accept a gig of data and be ready for more. You can impose a limit. A violation of the limit is an error, treated like a case of bad input.
- saagarjha 2y agoPeople have long surnames, especially if you take multi-byte characters into account. And someone with the same mindset as you is the reason why my profile picture in Google’s employee directory never did actually show up in Safari–one of the path components ended up being a couple thousand characters long (I wonder if it was literally a base64 encoding of the image itself?)
- CoastalCoder 2y agoOut of curiosity, would you mind sharing some details about your surname? E.g., its length in its native alphabet, or its length as a UTF-8 string?
- saagarjha 2y ago$ swift Welcome to Apple Swift version 6.0 (swiftlang-6.0.0.5.15 clang-1600.0.22.6). Type :help for assistance. 1> "झा".count $R0: Int = 1 2> "झा".utf8.count $R1: Int = 6
- kazinator 2y agoNo, my reasoning does not say that a browser shouldn't handle a long URL (which is a substring of a page it has already accepted and rendered).
- saagarjha 2y agoI fail to see the difference here?
- bell-cot 2y agoTHIS. Not for every use case, obviously. But for huge number of them. And the "FooBaz length 5307 bytes in check_input(), truncating to 4095 bytes..." errors (which are trivial to log, or ignore, as you wish) can reveal many interesting things.
- cjs_ac 2y ago> Nobody’s surname needs 128+ bytes. This attitude ensures that the US software industry will never conquer the world.
- sgt 2y agoThe US software industry has already conquered the world, and continues to do so. It doesn't mean that it'll always be this way, but if you wanted to associate software with a single country, then the US would be your answer.
- deleted 2y ago[deleted]
- shrimp_emoji 2y agoWhy does it need to conquer the world? I just want to deal with sane, performant software.
- estebank 2y agoAnd people in other locales want to be able to use their own name without software mangling it.
- gav 2y agoAt some point there is a physical limitation, there's no passport in the world that accepts a 666-character name. The US only gives you 21 characters on the DS-11 for a surname.
- Narishma 2y agoYou're assuming one character is stored as one byte, which is only the case for English.
- shrimp_emoji 2y agoThat sounds like a them problem. ;D I once heard that "decision" comes from a Latin root word meaning "to cut off (the other options; to pay opportunity cost)". I will decide to optimize for my use case.
- dark-star 2y ago> Nobody’s surname needs 128+ bytes. No reasonable URL for a firmware update download needs 4096 bytes. ...and surely the "seconds" field of a timestamp is always between 0 and 59 inclusive, addresses will include a state and a building number, phone numbers contain only digits and maybe a leading + sign, etc. Wrong assumptions like this are one of the root causes of (in the best case) bad UI or (worst case) annyoing bugs. 128 bytes for a surname is only about 60 unicode characters, less if you include RTL markings and characters outside the BMP. A URL can contain SHA hashes (think: reproducible builds) and can thus be very long (okay, 4k characters is pushing it quite a bit but I wouldn't rule it out like you did...)
- smaudet 2y ago> Wrong assumptions like this Assumptions, or features? I'm all for inclusive behavior, however I'm also for well tailored solutions. Having support for 8k characters when you are going to usually use maybe 20 isn't smart or correct either. That's why we have utf-8, not utf-32, you can grow the bytes when you need to, but only then. > A URL can contain SHA hashes It can, or it can not - again, perhaps a feature, not a bug. The hash can also live in a file named by convention, and downloaded/checked separately. Maybe there are other scenarios where you might need a really long url, but domain + release path + name + major.minor.patch should get you 99% of the way. What's "reasonable" is relative, always designing for the edge case is good in some cases, but its also OK (and perhaps better) to optimize on occasion.
- bigstrat2003 2y agoThere have to be limits somewhere. Memory and storage space is not infinite. On the other hand, "how many characters can someone have in their name" is infinite. That means that no matter what limit you choose, someone will eventually exceed that limit. And you have to have a limit. This is not a matter of "wrong assumptions". At the end of the day, all you can do is set a limit such that you're comfortable with the risk that someone will be outside the limit you have set. And risk tolerance, as always, is a matter of opinion and not fact.
- margalabargala 2y ago
- nathell 2y ago> Nobody’s surname needs 128+ bytes. https://en.wikipedia.org/wiki/Hubert_Blaine_Wolfeschlegelsteinhausenbergerdorff_Sr https://en.wikipedia.org/wiki/Hubert_Blaine_Wolfeschlegelste.... would beg to differ.
- dgfitz 2y agoI'm not sure his name is long enough for this to be an issue.
- Izkata 2y agoThe URL uses a shortened version, his full name on the page is: Adolph Blaine Charles David Earl Frederick Gerald Hubert Irvin John Kenneth Lloyd Martin Nero Oliver Paul Quincy Randolph Sherman Thomas Uncas Victor William Xerxes Yancy Zeus Wolfeschlegelsteinhausenbergerdorffwelchevoralternwarengewissenhaftschaferswessenschafewarenwohlgepflegeundsorgfaltigkeitbeschutzenvorangreifendurchihrraubgierigfeindewelchevoralternzwolfhunderttausendjahresvorandieerscheinenvonderersteerdemenschderraumschiffgenachtmittungsteinundsiebeniridiumelektrischmotorsgebrauchlichtalsseinursprungvonkraftgestartseinlangefahrthinzwischensternartigraumaufdersuchennachbarschaftdersternwelchegehabtbewohnbarplanetenkreisedrehensichundwohinderneuerassevonverstandigmenschlichkeitkonntefortpflanzenundsicherfreuenanlebenslanglichfreudeundruhemitnichteinfurchtvorangreifenvorandererintelligentgeschopfsvonhinzwischensternartigraum Sr.
- xyst 2y agoActive Directory would cry.
- mattigames 2y agoHe should have added a few numbers in there to really throw off some login systems.
- eru 2y agoAnd spaces.
- bloak 2y ago> Nobody’s surname needs 128+ bytes. According to Wikipedia some joker did have a 666-character surname, officially, in the USA. Perhaps the best thing to do would be to truncate the field at some reasonable limit to prevent people from crashing your system with ridiculous values but make sure your system works properly with the truncated names so, for example, it doesn't panic because the truncated name isn't equal to the original name. With spaces, punctuation and diacritics, comparing even short names for equality is a bit dangerous and probably best avoided. If you expect two text fields to match and they don't, even after normalisation, you could consider flagging the case for human review later but continuing without an error for the time being.
- saghm 2y agoIf I'm understanding correctly what you mean by "joker", it sounds like the changed their name to be this long intentionally? That seems like the type of thing where most software probably can get away with just not supporting then; with the possible exception of mandatory government services or something, there's no reason software should need to account for people taking such extreme steps of their own volition.
- mynameisvlad 2y agoBecause programmers are too lazy to properly handle long names? That's a stretch for denying someone service and you know it. Like, yes, nobody is forced to accept their name unless they're running a government service, but using it as an excuse is just that, an excuse.
- saghm 2y agoTo be clear, I'm not arguing for or against a specific value as the "maximum length" of a name. I'm drawing a distinction here in terms of a potential user's intentional choices and what that means for providing support. > Because programmers are too lazy to properly handle long names? That's a stretch for denying someone service and you know it. I don't think someone should be denied service if they happen to have a long name, but I genuinely don't think it's a stretch not to try to handle people going out of their way to subvert expected norms. In this case, the argument is more philosophical than technical because there isn't an obvious heuristic for determining whether a name is intentionally made long or not, but there are places where I do think it's worth it for programmers to consider. As an aside, I'd argue there's more nuance than "properly handling long names" or "being lazy. There's already an inherent limit on how large a name can fit into memory, and that limit might even fluctuate depending on the number of users being processed in a server at a given time and how much memory a given server has. Is a 1 GB name too long to be worth handling, or is not handling it "laziness"? If you're arguing that any name that the government accepts should be accepted by any software, how do you know what the limit is that the government will accept? If you have international customers, is the limit larger? If there's no documented limit, do you just need to hope your software is at least as robust as the government's? My point isn't that these situations are equivalent to a name that's 666 characters long, but that arguing that not handling 666 characters is lazy already is a blend of implicit technical assumptions (servers have enough memory that handling names with 666 characters isn't an issue) and social assumptions (it's possible for someone to actually have a name that long), and I don't think that "pretend all names can fit into memory fine and just crash or time out or something if there are too many names that are too long according to the parameters of the runtime and the hardware" is the obvious best choice from a fairness perspective.
- bear8642 2y ago> I'm not a fan of Unix man page section numbers in parentheses. Why not? Confusion if function call with section number as argument?
- shrimp_emoji 2y agoI once learned what they mean. But I forgot and now do not.
- Karellen 2y agoThey describe the section of the manual, where 1 = programs, 2 = syscalls, 3 = userspace functions, 4 = special files, 5 = file formats, 6 = games, 7 = misc/overviews. So printf(1) is the man page for the /usr/bin/printf command, while printf(3) is the man page for the libc printf() function. Alternatively, readdir(2) is the man page for the readdir syscall, while readdir(3) is the man page for the libc wrapper, which no longer actually calls readdir(2). See also syslog(2) and syslog(3). Or, time(1) is the man page for /usr/bin/time to time how long commands take. time(2) is the syscall to return the number of seconds since the epoch. And time(7) gives you an overview of time and timers on the system.
- _kst_ 2y agoI just discovered that typing man 'printf(1)' or man 'printf(3)' actually works (at least with the man command on Ubuntu, provided by the "man-db" package). You have to quote or escape the parentheses because they're shell metacharacters, which IMHO makes that syntax more trouble than it's worth. Other commands that work are "man 3 printf", "man -s 3 printf", and "man printf.3". I think "man 3 printf" is the oldest version. At least one other version of the man command (NetBSD) doesn't accept "man printf.3" or "man 'printf(3)'".
- spauldo 2y agoUnless you're on a UNIX system the number isn't important. If you are on a UNIX system, then it's useful for telling the difference between commands (like write(1), hostname(1), or printf(1)), system calls (like write(2)), library functions (like printf(3)), or config files (like hostname(5)).
- morpheuskafka 2y ago128 bytes is only 4 32-bit characters. Now, I think 4-byte UTF-8 characters are pretty rare, but at least 3-byte ones are certainly common in names, even legal names. If you allow users to type emojis in their name you will definitely run out as the color/gender selectors take up an additional code point.
- cpburns2009 2y agoYou confused bits with bytes. 128 bytes can encode 32 4-byte long characters.
- mjevans 2y agoNames are hard, don't force things into your assumptions. If you must, record fields such as: Full Legal Name - Freeform input no string input limitation. If you feel like this is an attack vector, send the data out for human review. Full Mailing Address - Don't try to break this down, allow multi-line, free form input. This is something you might want to validate with your shipping carrier and/or a human. A short 'nickname' used as such.
- tsimionescu 2y agoYou always need size limitation. You really don't want to allow 10GB strings to be stored in the full name field. Also, in almost any situation where you need a legal name, you actually want to follow a lot of rules. This idea that people's original names are somehow sacrosanct is a misunderstanding. If you're doing business in a European country for example, you have to write your name in Latin/Cyrillic letters, perhaps with a few symbols like ' or - allowed as well, and typically with a few accents/diacritics specific to each country. You certainly can't register as 田中 in any context that requires a legal name in France, you'd have to write that as Tanaka. And this is natural because legal records are meant for authorities in some specific country to read, and compare with other legal docs - so they need to at least be readable to those authorities.
- mjevans 2y agoA 10GB string does sound pathologically large; however it's an argument about the absurd. What size is for sure enough? Well I'm not so sure. What if someone has a lot of titles. What if society decides that someone's Legal Name requires a post quantum cryptography key that happens to be 20MB (binary) long? Also, FYI, at least PostgreSQL doesn't give you a free lunch for any variable length string; length requirements are a column constraint that DEGRADES performance because it has to check. An external name validator of some sort could check things. Commonly allowed cases could pass by computer check, while actual humans could review edge cases. Someone trying to abuse the name field like that probably needs human review elsewhere anyway.
- 2y ago
- thayne 2y ago> Nobody’s surname needs 128+ bytes 128 bytes would only be 42 characters if each character uses 3 bytes, as would be the case in some languages. Which isn't an unreasonable length, especially if the name has a lot of combining characters.
- kazinator 2y agoOk, let's make it 50 Unicode codepoints.
- hnfong 2y agoSurprised nobody has linked this yet: https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-... Also note that while the counter examples might sound extreme, in some languages each character might need 3 bytes in UTF-8, and 128/3 ~= 43 characters doesn't seem to be that outrageous.
- bawolff 2y ago> Nobody’s surname needs 128+ bytes That is only 32 astral characters... Seems kind of close for comfort. Not to mention combining characters
- smsm42 2y ago> Nobody’s surname needs 128+ bytes Oh we're doing that "false things that programmers believe about the world" again! Fun! Let's consider cultures where people can have more than one surname. Ever heard about Pablo Diego José Francisco de Paula Juan Nepomuceno María de los Remedios Cipriano de la Santísima Trinidad Ruiz y Picasso? Yes, that's his full name. Of course, he didn't frequently use it, but if you make a system dealing with people's names, you'd eventually end up having to support something like that.
- pezezin 2y agoIn this case, his family names (Spanish people have two) were just "Ruiz y Picasso", the rest was is given name. So you can argue that his family name fits within 128 bytes. But that brings us to the assumption that people have just one family name with is just one word, which is very much not the case in many cultures around the world.
- kazinator 2y agoA programmer who believes surnames are one word has not heard of John Von Neumann.
- sham1 2y agoAnd then there's the assumption that you'd write it in that way, and in that order. There might be situations where you'd want to use the Hungarian and write "Neumann János" instead. I.e. the assumption that the family names come last is not necessarily correct even in Europe. Let alone if one deals with the various Chinese languages, Japanese, or Korean. And probably others.
- pezezin 2y agoThat is also true. I am currently working in an international project in Japan, so when designing our systems and databases I always insists on the terms "given" and "family" names instead of "first" and "last" names, to prevent confusions. Same with the dates. I always ask my colleagues to please write the years in full, otherwise it can be very difficult to know if it is DMY or YMD (thankfully we don't have to deal with MDY).
- rat87 2y ago> nobody's surname needs 128 bytes I believe that's #6 falsehoods programmers believe about names (with examples) https://shinesolutions.com/2018/01/08/falsehoods-programmers-believe-about-names-with-examples/ https://shinesolutions.com/2018/01/08/falsehoods-programmers... Ingoring multi byte characters there are still plenty of long names https://www.ancestry.com/c/ancestry-blog/discovering-the-history-of-long-last-names https://www.ancestry.com/c/ancestry-blog/discovering-the-his... If you're going to try for a "reasonable" max name length it would probably need to be at least 4kb.