3 ms·
It's not pretty in PHP either. Many of the Perl issues have their PHP counterparts. • You need to be aware that most built-in string functions won't work and u
by pornel 15y ago
It's not pretty in PHP either. Many of the Perl issues have their PHP counterparts.
• You need to be aware that most built-in string functions won't work and use mb_ equivalents where available (and not all are available).
• You need to set internal encoding to UTF-8 for some extensions in php.ini.
• You need to normalize all input yourself (thankfully PHP5.3 has Normalizer class. In 5.2 it was a nightmare)
• PCRE library doesn't support all uncode ranges (e.g. {InFoo} is unsupported). All gotchas of regexes apply to PHP/PCRE as well.
The list goes on…
"Binary-safe" strings and UTF-8 cleverness will let you roundtrip characters safely in most cases, but full, correct Unicode support is really hard.
- ars 15y ago> You need to be aware that most built-in string functions won't work Actually the non-mb string functions will work perfectly fine for almost everything. For example splitting on space or comma, or joining strings will work just fine with UTF-8. The only things that don't work are those that split by character position. Even search works fine. > You need to set internal encoding to UTF-8 for some extensions in php.ini. ini_set('default_charset', 'UTF-8'); mb_internal_encoding("UTF-8"); > You need to normalize all input yourself Only if you need to compare against it (for example to see if the username was taken). Most of the time you just store the input as is. For PCRE make sure to add the /u option.
- dexen 15y agoI'd just stick to mb_ereg_...() functions. They follow the standard setlocale(LC_...). They support the standard POSIX regex syntax. They support Unicode always when the platform does (depends only on standard C library) -- and I've seen PHP's PCRE distributed without Unicode support. Last but not least, PCRE suffers from http://swtch.com/~rsc/regexp/regexp1.html http://swtch.com/~rsc/regexp/regexp1.html setlocale(LC_ALL, 'pl_PL.utf-8'); # if not set explicitly defaults to system's $LC_ALL, thus fits most of the time mb_internal_encoding('UTF-8'); mb_ereg_match('[[:alpha:]]', 'Ł') -> true mb_ereg_match('[[:alpha:]]', '♙') -> false # that's a white chess pawn character, U+2659. mb_ereg_match('[[:graph:]]', '♙') -> true
- ars 15y agoI don't like setlocale because usually I'm writing a general application, usable for any language.
- dexen 15y agoI like setlocale() as a central knob that other functions obey. Works a-OK for me on several different websites. Normally my website's code starts with platform defaults, reads configuration early on, and, if need be, issues setlocale(). Should an error be triggered before the config is applied, it would be handled with default locales; I believe that's the only noticeable downside. Hopefully a rare one ;-) setlocale() can be switched at run-time, even multiple times if you want. You could imagine running most of the code with user's preferred locale to output UI or report any errors, switching to another locale to format and send message to a foreign customer, and then back to normal settings to continue with outputing UI, all in one pass of PHP.