6 ms·
The closest to insight in this post is a quote from somebody else. “String length is about as meaningful a measurement as string height” Under that quote three
by tialaramex 7y ago
The closest to insight in this post is a quote from somebody else. “String length is about as meaningful a measurement as string height”
Under that quote three rationale are offered.
1. "to allocate memory". Fine, like an integer your very low level code needs to actually store strings in RAM and in this sense both integers and strings have a size. I don't think I'd name this size "length" though and certainly if length(5) is an error in your language it seems as though length("word") is also an error by this definition.
2. "quota limit". Just pick anything, there is definitely no reason to name the value you're using for this arbitrary quota "length". If you have a _reason_ for the quota, then you need to measure the reason, e.g. bytes on disk.
3. "display space". This one is a length! But it's in pixels (or milimeters, or some other type of distance unit) and it requires a LOT of additional parameters to calculate. And you'll notice that at last there is actually a string height too for this case, which likewise requires many additional parameters to calculate.
Treat code that cares about string "length" as code smell. If it's in your low-level fundamentals (whether in the memory allocator or the text renderer) it might, like a lot of other things which smell down there, have a good reason to be there, make sure it's seriously unit tested and clearly documented as to why it's necessary. If it's in your "brilliant" new social media web site backend, it's almost certainly wrong and figuring out what you ought to be doing instead of asking the "length" of a string will probably fix a bug you may not know you have.
- wongarsu 7y agoInterestingly, in many use cases all these "lengths" agree. For example an empty string is 0 in all these measures, and fuzzy metrics like "unreasonably long password" work equally well with "500 normalized characters" "500 bytes" "1000px at font size 8" etc. The problems only arise when taking about "a string of length 20" which indeed has a strong smell
- heavenlyblue 7y ago"unreasonably long password" must go into the area of "Content-Length is too long for this HTTP request to login/change password".
- coldtea 7y ago>The problems only arise when taking about "a string of length 20" which indeed has a strong smell It has no smell at all. We interface with all kinds of systems with limited display space, external forms, POS displays, databases with varchar columns, and so on all the time. Whether certain strings should have a specific length is a business decision, not a code smell...
- wongarsu 7y agolimited display space is a case for "string length in px", which is notoriously hard to calculate and has poor library support. Just because 20 "x" fit doesn't mean 20 "w" will fit. Fixed space fonts are an exception, but they have problems with Chinese. Databases with varchar columns exist, but varchar(20) sounds generally suspect unless it's a hash or something else that's fundamentally limited in length.
- coldtea 7y ago>limited display space is a case for "string length in px", which is notoriously hard to calculate and has poor library support. It is notoriously easy when the display is a LED display, a banking terminal, a form-based monospaced POS, something that goes to a printed out receipt (like a airline check-in or a luggage tag), a product / cargo label maker, and tons of other systems billions depend upon everyday, where one visible glyph = 1 length unit, and type design doesn't come into much play...
- yorwba 7y agoIt's only easy if the system forbids everything that would make calculating visible length hard, which I think constitutes extremely poor library support. I want to see the monospaced system that can correctly print Mongolian: ᠮᠣᠩᠭᠣᠯ ᠪᠢᠴᠢᠭ If properly implemented, it should join the characters and display them vertically. But your browser is probably showing them horizontally right now, because support for vertical writing is seriously broken: https://en.wikipedia.org/wiki/Mongolian_script#Font_issues https://en.wikipedia.org/wiki/Mongolian_script#Font_issues
- 7y ago
- contravariant 7y ago> "1000px at font size 8" That one is likely to get you into trouble when you allow 0 width characters.
- gpderetta 7y agofor what is worth, I have used the string byte length (i.e. the utf-8 code unit count) as a (weak) component for word weight in a clustering algorithms.
- coldtea 7y agoI'd say all of the above objections don't fix any of the actual issues (which revolve stem from there being multiple, similar, metrics one wants regarding the size in bytes, characters, normalized characters, etc of a unicode string), while introducing extra semantic bike-shedding! Whether the metric is called "length", or "size", or whatever, is irrelevant, and length for strings and arrays and lists is so well entrenched and understood that the objections don't make sense. What's actually the problem is not that "string length" is not "really" a length in the way that a leg or a wall has length, but that for unicode strings it's difficult to calculate, and confusing, unless one understands several unicode implementation mechanics... The name length itself, for example, or the measuring of said length, was never an issue with ASCII strings (the issue there was to remember to add/subtract the NUL at the end).
- mikeash 7y agoIt doesn’t matter how well you understand Unicode, it is impossible to compute “the length of a string” because there is no single metric that means that. It’s like measuring “the size of a box.” I want volume, you want maximum linear dimension, UPS wants length plus width plus height. The problem with “the size of a box” isn’t that it’s hard to measure, it’s that it doesn’t exist. Imagine your favorite language has a Box type with a “size” property. What does it return. How likely is it that the thing it measures is the thing you want? Of course it was never a problem for ASCII, because ASCII is structured to make most of the measurements people care about be the same value. I want bytes, you want code points, he wants “characters,” doesn’t matter, they’re all the same number. ASCII is also incapable of representing real world text with good fidelity. It’s inherently impossible to remedy that while maintaining a singular definition of “length.” If you value length measurement over fidelity, you can keep using ASCII.
- coldtea 7y ago>It doesn’t matter how well you understand Unicode, it is impossible to compute “the length of a string” because there is no single metric that means that. That's already covered in my comment: ("[the problem] stems from there being multiple, similar, metrics one wants"). Whether we call all of them length or a specialized name is not the real problem. The real problem is you need to know what you want of each, and some of them (e.g. regarding normalization, decomposition, and so on) can be hard to grasp. In 99% of cases people either want to know "how many bytes", or "how many discreet character glyphs of final output" (even if they have combining diacritics etc).
- deleted 7y ago[deleted]
- Manishearth 7y agoFun fact: the iOS bug where you could brick a device by sending some strings in Arabic to the device was essentially due to assuming that there is a monotonic relationship between some measure of "string length" and "display space". (essentially, iOS was trying to truncate strings in notifications to make them fit available space, except it did that in a very naive way, and would crash on any case where removing codepoints from a string made it display _longer_. Pretty easy to trigger with letters like ي)
- pavel_lishin 7y agoCan you go into a little more detail how that works? I know almost nothing about Arabic.
- lalaithion 7y agoThe same letter can look different depending on whether it's at the beginning, in the middle of, or at the end of a word. So if you remove a letter with a fairly short final form, which is preceeded by a letter with a fairly short medial form but a fairly long final form, it's possible that the new rendering of this letter would be longer than the previous two.
- Manishearth 7y agoThis problem isn't exactly Arabic-specific, it's just super easy to trigger in Arabic. You've got kerning in the Latin script as well, and you could theoretically have a font that makes `the` look smaller than `th`. But in Arabic, letters change form based on where they are in the word. All three things with two dots on them are the same letter in تتت, but the middle one is much smaller. I can imagine something like يم being truncated to ي and the end result being longer visually, depending on the font.
- deleted 7y ago[deleted]
- XCabbage 7y ago> Treat code that cares about string "length" as code smell This is nonsense. Pretty much anywhere that you're accepting user input on the web you want to have some sort of length limit. Even if you're Hacker News rather than Twitter, you probably don't want to let anybody post a 10GB comment. You just need to pick an appropriate length metric. That's part of the post's point, and it's also apparently your point a few sentences after you rubbish it; why are you insulting the post and then suddenly agreeing with it?
- cryptonector 7y agoIf you're not using a modern iterator, you'll want string length and string indexing. Otherwise you don't need string lengths except for the reasons given in TFA and above, but (1) (to allocate memory) can be hidden in most cases, (2) is mostly not necessary (account for encoded bytes where possible and be done), and there's very little code that needs to figure out (3). For almost every task, an iterator that iterates graphemes (extended grapheme clusters) is the right thing to do. I do think it's annoying that Unicode ended up with such a complex extended grapheme cluster scheme though.
- beatgammit 7y ago1. You don't iterate over bytes in an integer, so you don't need to know the size of an int since all uses are handled by the compiler. If you do care, you can always find out with a sizeof check (or preferably specify the size directly). The compiler can't handle Unicode for you since it's typically dynamic data. 2. This is reasonable. 3. What about fixed-width fonts like a terminal screen. In this case, I care about how many distinct (non-combined) code points exist, and a count of code points is often a acceptable approximation. That being said, when talking about string length, I mostly care when iterating and allocating memory, and those are two different usages of the term "length". I like how Go handles it: len(string) gives number of Unicode code points, len([]byte(string)) gives number of bytes. Everything else is a special case IMO that shouldn't use length, except perhaps as a conservative estimate (e.g. fixed width rendering).