4 ms·

Here's one I've started to focus on in the last few months: say "ปฏิบัติจริง".chars; # says 7 [1] Perl 6 may be a game changer in years to come among thos

by raiph 11y ago

Here's one I've started to focus on in the last few months: say "ปฏิบัติจริง".chars; # says 7 [1] Perl 6 may be a game changer in years to come among those who need to munge text that's Thai, Chinese, etc. http://www.reddit.com/r/perl/comments/2szv7m/beginner_programmer_here_looking_for_help_for_an/cqzk3hi http://www.reddit.com/r/perl/comments/2szv7m/beginner_progra...

ceronman 11y ago
I don't see how this is a language killer feature. It looks more like a library feature. And yes, Perl 6 has a superb unicode library. In other languages, you can add the functionality as libraries. This is an example in Python: import unicodedata def count_graphemes(string): count = 0 for codepoint in string: if not unicodedata.category(codepoint).startswith('M'): count += 1 return count print(count_graphemes("ปฏิบัติจริง")) # prints 7 This example is over simplified as grapheme clusters are trickier to detect, but proper support is coming to Python's standard library: http://bugs.python.org/issue18406 http://bugs.python.org/issue18406
- raiph 11y ago
  (Edited to say "regular folk" instead of just "folk" in the last paragraph to make it clear I'm talking about non-expert coders who would much prefer that stuff like converting mixed case text to all lower or upper case, or making sure some text visually fits in a fixed length field, be a reasonably simple operation, like it already is when handling most Western languages.) ---- Would you agree that you are saying that you think it's OK that: * The appropriate long term solution is to use library code when doing character or substring level operations on arbitrary Unicode text; * Language implementors have generally not yet written said library code; * The built in character and substring operations in mainstream languages unpredictably misinterpret and corrupt international text such as Thai, Chinese, etc.; * The current solution for correct, non-corrupting character and substring handling of arbitrary Unicode text is to hand build custom code to do things like get the length of a string or replace a substring. If you disagree with the above, then I'm confused. If you agree, then you're in agreement with most folk and I think you and they are blind to the needs of regular folk who need to handle, say, Chinese text, and these latter folk will find the simple and correct built in character and substring handling in Perl 6 to be one of its killer features.

by raiph 11y ago

ceronman 11y ago

I don't see how this is a language killer feature. It looks more like a library feature. And yes, Perl 6 has a superb unicode library. In other languages, you can add the functionality as libraries. This is an example in Python: import unicodedata def count_graphemes(string): count = 0 for codepoint in string: if not unicodedata.category(codepoint).startswith('M'): count += 1 return count print(count_graphemes("ปฏิบัติจริง")) # prints 7 This example is over simplified as grapheme clusters are trickier to detect, but proper support is coming to Python's standard library: http://bugs.python.org/issue18406 http://bugs.python.org/issue18406

raiph 11y ago
(Edited to say "regular folk" instead of just "folk" in the last paragraph to make it clear I'm talking about non-expert coders who would much prefer that stuff like converting mixed case text to all lower or upper case, or making sure some text visually fits in a fixed length field, be a reasonably simple operation, like it already is when handling most Western languages.) ---- Would you agree that you are saying that you think it's OK that: * The appropriate long term solution is to use library code when doing character or substring level operations on arbitrary Unicode text; * Language implementors have generally not yet written said library code; * The built in character and substring operations in mainstream languages unpredictably misinterpret and corrupt international text such as Thai, Chinese, etc.; * The current solution for correct, non-corrupting character and substring handling of arbitrary Unicode text is to hand build custom code to do things like get the length of a string or replace a substring. If you disagree with the above, then I'm confused. If you agree, then you're in agreement with most folk and I think you and they are blind to the needs of regular folk who need to handle, say, Chinese text, and these latter folk will find the simple and correct built in character and substring handling in Perl 6 to be one of its killer features.