5 ms·
Unicode, Perl 6, and You
- wtetzner 11y agoI haven't finished reading through the whole post yet, but if Perl 6 works on graphemes, are ligatures considered to be only one character?
- lelf 11y agoYes. Bear in mind however that ligatures are in Unicode only for the backward compatibility. > 'ffl'.chars 1
- logicallee 11y agothat's sure to ruffle some feathers. (see what I did there)
- evmar 11y agoTry highlighting substrings of that text -- my browser doesn't even know it's multiple characters. (Separate is the idea of a browser displaying ligatures, which already works. But that's because ligatures are a display issue and the source text must have non-ligature text in it.)
- deleted 11y ago[deleted]
- deleted 11y ago[deleted]
- SwellJoe 11y agoPerl has had decent Unicode support longer than most similar languages (years before Ruby and Python, for instance), but Perl 6 is just ridiculously good at it, and I hope other languages follow suit. I'm unaware of any other language that handles Unicode this well...am I missing any languages that do? I guess JavaScript is coming along on this front and ES6 includes support for Unicode regexps, which is progress, so maybe that's the closest mainstream language.
- lelf 11y ago> ES6 includes support for Unicode regexps Will it provide \X for example? (\X matches extended grapheme cluster.) > am I missing any languages that do? Swift is one notable example. It has built-in and simple enough grapheme handling.
- SwellJoe 11y agoI haven't looked at Swift, at all. I don't buy Apple products, so have no familiarity with their ecosystem. But, now that it's been opened up, I'll have a look at it, though it seems likely to remain predominantly a language for Apple products for the foreseeable future (I think?), so not something I'd find myself using in production any time soon. But, I guess we'll see how that shakes out over time now that it is open. Given the rate at which JavaScript is converging on a really nice set of modern features and is having warts removed and performance is accelerating, I wonder if any other language is as relevant long-term.
- saurik 11y agoSwift's Unicode support is sufficiently awesome that my web browser was having issues rendering the documentation for their string class due to the epic working examples ;P.
- OopsCriticality 11y ago> ...am I missing any languages that do? Tcl supported Unicode in 1999 with version 8.1. Much like Zathras, Tcl is the beast of burden that is easy to overlook.
- kbenson 11y ago> Don’t worry though, standard Perl 6 does not demand that you be able to type Unicode. If you can’t, there are so-called “Texas” variants: I've always loved the "everything's bigger in Texas" joke implicit inthe "texas" variant on some operators. > If you’re interested in working within a particular normalization, there’s the self-explanatory types of NFC, NFD, NFKC, and NFKD. That would probably be better with a "Well, it's self explanatory at the point you know you want to work in a particular normalization", since I only vaguely know what those are, and I've beenhearing about some of them for years. ;) Great post though!
- Grue3 11y ago>say "नि".codes; # returns 3 How is "नि" 3 codepoints? There is only two: न and ि . Could this be a bug?
- kamaal 11y agoNot sure how to interpret this, are you suggesting the hindi नि should be two code points because when translated to English it is 'Ni'(two letters)?
- Grue3 11y agoI'm suggesting this string has two Unicode codepoints. I don't know anything about Hindi language or Devanagari script.
- EdiX 11y agoHexdumped the html of the webpage, there's only two codepoints here.
- hotkeys 11y agoThey were probably using an outdated version of rakudo, the latest version returns 2 as you would expect.
- cursork 11y agoSeems to be a typo? $ perl6 -e 'say "नि".codes' 2
- cygx 11y agoPerl6 agrees that it consists of only 2 codepoints: say "नि".NFC>>.uniname.perl; #=> ("DEVANAGARI LETTER NA", "DEVANAGARI VOWEL SIGN I") Nothing more serious than a typo in the article would be my guess.