6 ms·

Can you elaborate on what exactly you mean by “natively support”?

by hnarn 2y ago

gbuk2013 2y ago
https://phptherightway.com/#php_and_utf8 https://phptherightway.com/#php_and_utf8
- MissTake 2y ago
  I’d argue that it does natively support MultiByte strings, albeit with an extension library that’s part of the language. Even your supplied link states they’re not supported at a “low level”, but states nothing about “native”.
  - gbuk2013 2y ago
    If you want to be pedantic then multi-byte string !== UTF-8 support. ;) Consider the intended purpose of the language and then consider whether the abstraction offered is appropriate. IMO in the case of PHP and UTF-8 it is not. In my specific case it made my job harder than I would like on 2 projects I used PHP for, which is why I am complaining.
SXX 2y ago
While proper description is linked already in neighbouring comment. TLDR: in 2024 with PHP8 you still need mbstring extension and also you should be careful around UTF-8 if you do any text processing. In almost all other modern programming languages it's just works.
- mgaunard 2y ago
  Anyone claiming such things doesn't understand Unicode at all. The whole concept of having special Unicode-aware strlen or substr is nonsense.
- dfgdfg34545456 2y ago
  In NodeJS for example don't you have use Buffers and special decoders to deal with UTF-8 strings? I.e it's a pain there too.
  - gbuk2013 2y ago
    No, JS strings are UTF-8: > '蛋糕'.substr(0,1) '蛋' > '蛋糕'.length 2 > Buffer.byteLength('蛋糕') 6 You do have to be careful when working with binary data (e.g. streams) but this is expected.
    - trurl42 2y ago
      And with that you're completely wrong, since strings in JavaScript are UTF-16. It just so happens that your example consists of two UTF-16 codepoints. (Node.js' Buffer uses UTF-8 by default).
      - kiitos 2y ago
        One ambiguity here might be that Javascript defines strings as UTF-16, but JSON defines strings as UTF-8.
    - njuw 2y ago
      They're UTF-16, and substr(), length, etc, work at the code unit level. Hence, the above isn't actually valid for all strings - any characters that are represented by codepoints between U+10000 and U+10FFFF require 2 code units [1]. For example U+10429 Deseret Small Letter Long E [2] > '𐐩'.substr(0, 1) '\ud801' > '𐐩'.length 2 [1] https://en.wikipedia.org/wiki/UTF-16#Description https://en.wikipedia.org/wiki/UTF-16#Description [2] https://codepoints.net/U+10429 https://codepoints.net/U+10429
      - deleted 2y ago
        [deleted]
      - deleted 2y ago
        [deleted]

Can you elaborate on what exactly you mean by “natively support”?

by hnarn 2y ago

Can you elaborate on what exactly you mean by “natively support”?

gbuk2013 2y ago

https://phptherightway.com/#php_and_utf8 https://phptherightway.com/#php_and_utf8

MissTake 2y ago
I’d argue that it does natively support MultiByte strings, albeit with an extension library that’s part of the language. Even your supplied link states they’re not supported at a “low level”, but states nothing about “native”.
- gbuk2013 2y ago
  If you want to be pedantic then multi-byte string !== UTF-8 support. ;) Consider the intended purpose of the language and then consider whether the abstraction offered is appropriate. IMO in the case of PHP and UTF-8 it is not. In my specific case it made my job harder than I would like on 2 projects I used PHP for, which is why I am complaining.

SXX 2y ago

While proper description is linked already in neighbouring comment. TLDR: in 2024 with PHP8 you still need mbstring extension and also you should be careful around UTF-8 if you do any text processing. In almost all other modern programming languages it's just works.

mgaunard 2y ago
Anyone claiming such things doesn't understand Unicode at all. The whole concept of having special Unicode-aware strlen or substr is nonsense.
dfgdfg34545456 2y ago
In NodeJS for example don't you have use Buffers and special decoders to deal with UTF-8 strings? I.e it's a pain there too.
- gbuk2013 2y ago
  No, JS strings are UTF-8: > '蛋糕'.substr(0,1) '蛋' > '蛋糕'.length 2 > Buffer.byteLength('蛋糕') 6 You do have to be careful when working with binary data (e.g. streams) but this is expected.
  - trurl42 2y ago
    And with that you're completely wrong, since strings in JavaScript are UTF-16. It just so happens that your example consists of two UTF-16 codepoints. (Node.js' Buffer uses UTF-8 by default).
    - kiitos 2y ago
      One ambiguity here might be that Javascript defines strings as UTF-16, but JSON defines strings as UTF-8.
  - njuw 2y ago
    They're UTF-16, and substr(), length, etc, work at the code unit level. Hence, the above isn't actually valid for all strings - any characters that are represented by codepoints between U+10000 and U+10FFFF require 2 code units [1]. For example U+10429 Deseret Small Letter Long E [2] > '𐐩'.substr(0, 1) '\ud801' > '𐐩'.length 2 [1] https://en.wikipedia.org/wiki/UTF-16#Description https://en.wikipedia.org/wiki/UTF-16#Description [2] https://codepoints.net/U+10429 https://codepoints.net/U+10429
    - deleted 2y ago
      [deleted]
    - deleted 2y ago
      [deleted]