3 ms·
I once wrote a programming language that used UTF-8. I was rather proud of this hack that succinctly allowed UTF-8 variable / function names. I went searching f
by sambeau 13y ago
I once wrote a programming language that used UTF-8. I was rather proud of this hack that succinctly allowed UTF-8 variable / function names. I went searching for it and, sure enough, I still like it.
BAD_UTF8 = [\xC0\xC1\xF5-\xFF];
UTF8_CB = [\x80-\xBF];
UTF8_2B = [\xC2-\xDF];
UTF8_3B = [\xE0-\xEF];
UTF82 = UTF8_2B UTF8_CB;
UTF83 = UTF8_3B UTF8_CB UTF8_CB;
UTF8 = UTF82 | UTF83 ;
ATOM = ([_a-zA-Z]|UTF8)([_a-zA-Z0-9]|UTF8)*;
- ygra 13y agoThis looks limited to the BMP in the same way the broken utf8 charset for MySQL is. You're simply ignoring the existence of the other 16 planes. While not necessarily a problem in practice, especially with identifiers in programs, it still hints at a greater problem that you nonetheless chose to name this UTF-8 which it isn't.
- sambeau 13y agoYes, I chose to stop there for pragmatic reasons, however, the intention was always to add the other 4, 5 & 6 byte forms if/when needed. This was as much as I needed to test my wide char code (and to parse written languages e.g. Chinese). At least left like this my code would error properly if it encountered a character it didn't know.