4 ms·

You’re definitely wrong. You’re designing a language that only works for “Americans” by default. Imagine how you would feel about a language that supports Arab

You’re definitely wrong. You’re designing a language that only works for “Americans” by default. Imagine how you would feel about a language that supports Arabic by default and needs special foo to work with American English? You need to start thinking of characters as a type. Characters do not fit in bytes unless you’re American. And even if you’re American, people will still throw emdashes and Unicode quote symbols and emojis into your text. I’d suggest that if a language doesn’t have a proper character type, it’s not a useful language for the majority of the world. UTF-8 is a reasonable compromise though.

alcover 3y ago
I mean the reserved words in my lang are in English, and the user-created identifiers must be ASCII at the moment. I could extend the latter to some wider range but my question was more about providing built-in Unicode string handling. My Buffer class lets you allocate bytes and it's up to your code to interpret those bytes. I get your concern though and realize parsing let foo = "<unicode>" would be a problem.
- zadokshi 3y ago
  Apologies if I come off a bit strong on this one. I deal mostly with non “American” text so it’s a pain point. Unicode is so messy, trying to push handling it on to each individual programmer is not an ideal situation. Let’s just say, no one in Greece, Iceland, or Korea is sitting around wondering if their new programming language should support their alphabet natively. Imagine being from a country where a language’s built in Strcmp doesn’t work?
  - alcover 3y ago
    No problem ! > Imagine being from a country where a language’s built in Strcmp doesn’t work? Problem is it's seemingly a massive headache. Citing the article, "Unicode handling is always best left to a library". I could bundle the recommended lib (ICU) into my lang's stdlib but it seems quite big...
  - qalmakka 3y ago
    To be fair, that has never stopped people from using C and C++ all around the world... EDIT: and most languages made before 1990 too

alcover 3y ago

I mean the reserved words in my lang are in English, and the user-created identifiers must be ASCII at the moment. I could extend the latter to some wider range but my question was more about providing built-in Unicode string handling. My Buffer class lets you allocate bytes and it's up to your code to interpret those bytes. I get your concern though and realize parsing let foo = "<unicode>" would be a problem.

zadokshi 3y ago
Apologies if I come off a bit strong on this one. I deal mostly with non “American” text so it’s a pain point. Unicode is so messy, trying to push handling it on to each individual programmer is not an ideal situation. Let’s just say, no one in Greece, Iceland, or Korea is sitting around wondering if their new programming language should support their alphabet natively. Imagine being from a country where a language’s built in Strcmp doesn’t work?
- alcover 3y ago
  No problem ! > Imagine being from a country where a language’s built in Strcmp doesn’t work? Problem is it's seemingly a massive headache. Citing the article, "Unicode handling is always best left to a library". I could bundle the recommended lib (ICU) into my lang's stdlib but it seems quite big...
- qalmakka 3y ago
  To be fair, that has never stopped people from using C and C++ all around the world... EDIT: and most languages made before 1990 too