4 ms·
Discussions of unicode often centre around the issue of counting symbols/graphemes/bytes/etc. and I often wonder what the use case is for counting anything othe
by stormbrew 13y ago
Discussions of unicode often centre around the issue of counting symbols/graphemes/bytes/etc. and I often wonder what the use case is for counting anything other than either the number of bytes (for storage) or the size in device units of the output text from a rendering engine (for display) is. All the options between seem like pure exercise.
The reality seems to be that the 'size' of text is entirely dependent on context and even forward thinking articles on the subject seem to get hung up on counting things that don't matter.
- byroot 13y agoWell, that depends. If you want to implement a simple `truncate`[0][1] function, then you need to count graphemes [0] http://api.rubyonrails.org/classes/ActionView/Helpers/TextHelper.html#method-i-truncate http://api.rubyonrails.org/classes/ActionView/Helpers/TextHe... [1] https://github.com/epeli/underscore.string https://github.com/epeli/underscore.string
- al2o3cr 13y agoIf you're truncating by character (or WHATEVER) counts, you are guaranteed to be doing it wrong - maybe not in your native language, but in somebody's. Heck, even in one graphemically-straightforward language you can get silliness: http://www.images.generallyawesome2.com/photos/funny/photos/testimonial-fail.png http://www.images.generallyawesome2.com/photos/funny/photos/...
- millstone 13y agoBut truncation is very often needed when you have more text than space. What's the alternative?
- gliese1337 13y agoIf it's storage space, you truncate by bytes, rounding down to the nearest complete grapheme- no need to count graphemes. If it's display space, truncate by pixels, in which case you need "size in device units of the output text from a rendering engine". Again, no need to count graphemes.
- dustyleary 13y agoCounting graphemes and detecting "the nearest complete grapheme" are basically the same problem. The only reason counting graphemes is hard is because detecting grapheme boundaries is hard.
- oneeyedpigeon 13y agoOf course, neither of those approaches solves the 'ass-' problem, which is possibly a red herring.
- bcoates 13y agoI'm trying to imagine a use case for grapheme-wise truncate that wouldn't be better served by counting bytes, code points, pixels, or some sort of localized letter-counting convention.
- millstone 13y agoHow about implementing backspace in a text editor?
- gliese1337 13y agoThat only requires being able to identify the boundaries of the last grapheme. It does not require counting them all.
- millstone 13y agoSo you acknowledge that determining grapheme ranges is important - you just don't see the need to count the ranges? If so, sure, I agree. The counting example is contrived. Truncation, navigation, text selection, etc. are more interesting and practical applications.
- Daniel_Newby 13y agoMost editors identify a grapheme by (row,column) coordinate.
- deleted 13y ago[deleted]
- mathias 13y agoThe article mentions several examples of real-world issues (in popular libraries etc.) that stem from this behavior. Writing a regular expression to match a single symbol sure seems easy until you realize some characters have a length of `2`, not `1`, in JavaScript (and some other languages).