2 ms·
> Naive heuristic: Unicode has a fully spelled out and testable algorithm that covers this. There's no need to make anything up on the spot. You don't need c
by JohnHaugeland 5y ago
> Naive heuristic:
Unicode has a fully spelled out and testable algorithm that covers this. There's no need to make anything up on the spot.
You don't need cultural context. It's not trivial. The algorithm is six printed pages.
No individual part of it is particularly hard, but every language has its own weird stuff to add.
I feel strongly that programmers should implement Unicode case folding with tests at least once, the way they should implement sorting and network connections and so on.
.
> I think it’s a reasonable answer on the whole falsehood-programmers-believe-about-names validation deal
All these "false things programmers believe about X" documents? Go through Unicode just one time, you know 80% of all of them.
I get that you like the framing you just made up, but there's an official one that the world's language nerds have been arguing over for 20 years, including having most of the population actually using it.
I suggest you consider that it might be pretty well developed by now.
.
> The use-case I’m getting at is answering the question “does this render anything visible (including white space)”?
This is not the appropriate approach, as white space can be rendered by non-characters, and characters can render nothing and still be characters under Unicode rules.
.
> It’s the most intuitive and practically useful interpretation of “length” for arbitrary Unicode as text.
It also fails for Mongol, math symbols, music symbols, the zero-width space, the conjoiner system, et cetera.
.
> The fact that it’s hard, complex, inelegant and arguably inconsistent
Fortunately, the real approach is easy, complex, inelegant, and wholly consistant. It's also well documented, and works reliably between programming languages that aren't Perl.
At any rate, you're off in another thread instructing me that the linguists are wrong with their centuries of focus because of things you thought up on the spot, and that Unicode is wrong because it counts spaces when counting characters, so, I think I'm going to disconnect now.
Thanks for understanding. Have a good day.