3 ms·

> Python 3 was released in 2008 which changed to use Unicode by default. Sure, but Python 3 only took on correct handling at the whole string level. It ignored

by raiph 10y ago

> Python 3 was released in 2008 which changed to use Unicode by default. Sure, but Python 3 only took on correct handling at the whole string level. It ignored correct handling at the character and sub-string level. Something similar applies for many programming languages. I think this was Zoffix's point.[1] > explicitly use an arbitrary-precision data type to avoid the classic floating-point math problems: > 123456789 - 1e-5 123456788.99999 Literals of the form `1e-5` are not arbitrary precision in Perl 6. They are floating point (called Num in Perl 6).Hence your above result. Similarly: > my $a = Num(12356789) 12356789 > $a + FatRat(0.1) - $a 0.099999999627471 would work if instead you wrote: > my $a = 12356789 12356789 > $a + 0.1 - $a 0.1 > $a + 0.00001 - $a 0.00001 If all inputs are 100% accurate, arbitrary precision Ints and/or FatRats then all results will be too. But the same applies even if some Rats are also involved provided the final result requires a denominator less than 18,446,744,073,709,551,615. [1] Of the 150+ languages with Rosettacode solutions for returning the length of a string (at http://rosettacode.org/wiki/String_length http://rosettacode.org/wiki/String_length) just 3 (Elixir, Perl 6, Swift) have a built in way to get the right result for what Unicode defines as "what a user thinks of as a character".

acdha 10y ago
> Sure, but Python 3 only took on correct handling at the whole string level. It ignored correct handling at the character and sub-string level. Something similar applies for many programming languages. I think this was Zoffix's point It very well could have been what Zoffix had in mind but do note that this goes back to the problem with the rant style of post. As for the specific question of character handling which you raised, given the wide number of people who have made other choices on the question of whether a length function should count codepoints or graphemes that I'm reluctant to say using one versus the other is a question of “correct” or “incorrect” versus simply “different”. I suspect most programmers are rarely going to care and the ones who do are going to need to learn enough more about Unicode and i18n that this difference will not be a deciding factor for anyone. (This is not to detract from the great history the Perl community has with taking seriously the benefits of having a rich Unicode API – I've routinely used this as an example to follow – but simply that Unicode is a deceptively simple-looking topic) > Literals of the form `1e-5` are not arbitrary precision in Perl 6. They are floating point (called Num in Perl 6). Yes. Again, my point wasn't that Perl 6 is bad but rather that it's lazy and ineffective advocacy to say something like “you can write accounting software worry-free” when you know full well that anyone doing that for real will still need to understand the differences and that most users will never care because they aren't writing financial software.
- deleted 10y ago
  [deleted]
- b2gills 10y ago
  There definitely are good reasons for a length function to count both codepoints and graphemes. Which is why Perl 6 has a method for both. This is also why neither of them is called `length`. In fact if you ever attempt to call the `length` method on an object Rakudo will ask you "Did you mean 'elems', 'chars' or 'codes'"
  - acdha 10y ago
    Yes - it's a good distinction to make and when you need it, it's extremely useful. As I said, Perl deserves respect for Unicode in general and raising awareness of this issue is a key reason for that. However, keep in mind that I was responding to a comment which simply referred to “Unicode issues or needing to use silly "u" prefixes”. This whole tangent started with a guess about what the author might have had in mind. On this specific point, note that I wasn't saying that it wasn't good to have both but that I wouldn't call a language incorrect for working only with Unicode codepoints. That's because for most programmers this entire class of problem is someone else's problem – usually whoever wrote the text rendering engine in your browser or OS – and the people who do need to care have needed to learn most of e.g. http://unicode.org/reports/tr29/ http://unicode.org/reports/tr29/ anyway and understand which portions are relevant to whatever task and data they're working with. It's kind of cool that e.g. 'क्षि'.elems == 1, 'क्षि'.chars == 2, etc. but on the rare occasions where that would be more than trivia, I was more interested in questions like measured width in a certain font or language-specific collation or word-breaking rules. This is all coming back to why I don't think attacking other languages is effective advocacy unless you're very knowledgeable on the details and impact for working programmers. Telling someone that a commonly used tool which works well for millions of users is incorrect is unlikely to produce the desired outcome. Showing them a cool thing which your favorite tool does better is usually going to be more effective because it gives you something concrete to talk about and it's not confrontational. Programming languages are a major commitment and very few people are going to switch because of one bullet point – that either takes market requirements (e.g. Objective C/Swift, JavaScript) or gradually building up a good reputation over time.

by raiph 10y ago

acdha 10y ago

> Sure, but Python 3 only took on correct handling at the whole string level. It ignored correct handling at the character and sub-string level. Something similar applies for many programming languages. I think this was Zoffix's point It very well could have been what Zoffix had in mind but do note that this goes back to the problem with the rant style of post. As for the specific question of character handling which you raised, given the wide number of people who have made other choices on the question of whether a length function should count codepoints or graphemes that I'm reluctant to say using one versus the other is a question of “correct” or “incorrect” versus simply “different”. I suspect most programmers are rarely going to care and the ones who do are going to need to learn enough more about Unicode and i18n that this difference will not be a deciding factor for anyone. (This is not to detract from the great history the Perl community has with taking seriously the benefits of having a rich Unicode API – I've routinely used this as an example to follow – but simply that Unicode is a deceptively simple-looking topic) > Literals of the form `1e-5` are not arbitrary precision in Perl 6. They are floating point (called Num in Perl 6). Yes. Again, my point wasn't that Perl 6 is bad but rather that it's lazy and ineffective advocacy to say something like “you can write accounting software worry-free” when you know full well that anyone doing that for real will still need to understand the differences and that most users will never care because they aren't writing financial software.

deleted 10y ago
[deleted]
b2gills 10y ago
There definitely are good reasons for a length function to count both codepoints and graphemes. Which is why Perl 6 has a method for both. This is also why neither of them is called `length`. In fact if you ever attempt to call the `length` method on an object Rakudo will ask you "Did you mean 'elems', 'chars' or 'codes'"
- acdha 10y ago
  Yes - it's a good distinction to make and when you need it, it's extremely useful. As I said, Perl deserves respect for Unicode in general and raising awareness of this issue is a key reason for that. However, keep in mind that I was responding to a comment which simply referred to “Unicode issues or needing to use silly "u" prefixes”. This whole tangent started with a guess about what the author might have had in mind. On this specific point, note that I wasn't saying that it wasn't good to have both but that I wouldn't call a language incorrect for working only with Unicode codepoints. That's because for most programmers this entire class of problem is someone else's problem – usually whoever wrote the text rendering engine in your browser or OS – and the people who do need to care have needed to learn most of e.g. http://unicode.org/reports/tr29/ http://unicode.org/reports/tr29/ anyway and understand which portions are relevant to whatever task and data they're working with. It's kind of cool that e.g. 'क्षि'.elems == 1, 'क्षि'.chars == 2, etc. but on the rare occasions where that would be more than trivia, I was more interested in questions like measured width in a certain font or language-specific collation or word-breaking rules. This is all coming back to why I don't think attacking other languages is effective advocacy unless you're very knowledgeable on the details and impact for working programmers. Telling someone that a commonly used tool which works well for millions of users is incorrect is unlikely to produce the desired outcome. Showing them a cool thing which your favorite tool does better is usually going to be more effective because it gives you something concrete to talk about and it's not confrontational. Programming languages are a major commitment and very few people are going to switch because of one bullet point – that either takes market requirements (e.g. Objective C/Swift, JavaScript) or gradually building up a good reputation over time.