9 ms·

In gamedev there is simple rule: don't try to do any of that. If it is text game needs to show to user then every version of the text that is needed is a trans

In gamedev there is simple rule: don't try to do any of that. If it is text game needs to show to user then every version of the text that is needed is a translated text. Programmer will never know if context or locale will need word order changes or anything complicated. Just trust the translation team. If text is coming from user - then change design until its not needed to 'convert'. There are major issues just to show user back what he entered! Because the font for editing and displayed text could be different. Not even mentioning RTL and other issues. Once ppl learn about localization the questions like why a programming language does not do this 'simple text operation' are just a newcomer detector. :)

zahlman 2y ago
>If text is coming from user - then change design until its not needed to 'convert' In games, you can possibly get away with this. Most other people need to worry about things like string collation (locale-aware sorting) for user-supplied text.
- makeitdouble 2y ago
  TBF, if you are caring about string collation, you're already at the entrance of the rabbit hole and probably should go down to the deep end anyway. I'd assume SleepyMyroslav doesn't apply to devs willing to spend weeks at time to handle all the complexity in full.
fluoridation 2y ago
>Once ppl learn about localization the questions like why a programming language does not do this 'simple text operation' are just a newcomer detector. :) I think you are purposefully misinterpreting the question. They're not asking about converting the case of any Unicode string with locale sensitivity, they're asking about converting the case of ASCII characters. What if your game needs to talk to a server and do some string manipulation in between requests? Are you really going to architect everything so that the client doesn't need to handle any of that ever?
- squeaky-clean 2y ago
  > They're not asking about converting the case of any Unicode string with locale sensitivity, they're asking about converting the case of ASCII characters. I'm confused now. The article specifically mentions issues with UTF-16 and UTF-32 unicode characters outside the basic multilingual plane (BMP).
  - fluoridation 2y ago
    I'm referring to the people who call case conversion in general "a simple text operation". Say you have an std::string and you want to make it lower case. If you assume it contains just ASCII that's a simpler operation than if you assume it contains UTF-8, but C++ doesn't provide a single function that does either of them. A person can rightly complain that the former is a basic functionality that the language should include; personally, I would agree. And you could say "wow, doesn't this person realize that case conversion in Unicode is actually complicated? They must be really inexperienced." It could be that the other person really doesn't know about Unicode, or it could mean that you and them are thinking about entirely different problems and you're being judgemental a bit too eagerly.
    - squeaky-clean 2y ago
      For ascii in C++ isn't there std::tolower / std::toupper? If you're not dealing with unsigned char types there isn't a simple case conversion function, but that's for a good reason as the article lays out.
      - fluoridation 2y ago
        Those functions take and return single characters. What's missing is functions that operate on strings. You can use them in combination with std::transform(), but as the article points out, even if you're just dealing with ASCII you can easily do it wrong. I've been using C++ for over 20 years and I didn't know tolower() and toupper() were non-addressable. There's really no excuse for the library not having simple case conversion functions that operate on strings in-place.
deleted 2y ago
[deleted]
beeboobaa3 2y ago
> There are major issues just to show user back what he entered! Because the font for editing and displayed text could be different. Not even mentioning RTL and other issues. Your web browser is doing it right now as you are reading this comment.
- rty32 2y ago
  And web development is not game development? And chances are that games don't ship chromium with them?
  - moron4hire 2y ago
    Actually... https://github.com/baikety/uWebKit https://zenfulcrum.com/browser/docs/Readme.html https://github.com/roydejong/chromium-unity-server There are a lot more, I just got bored at 3. And it's not just Unity. Several exist for Unreal as well. Why? Specifically because 2D layout and text rendering suck so much in game engines. What's ~50MB matter when you're shipping several GB of game assets?
cheema33 2y ago
> In gamedev there is simple rule: don't try to do any of that. I am not in gamedev, but I frequently have to develop middleware that takes in user entered data and formats it in a way that will import into a 3rd party system without errors. And that sometimes means changing the case on strings. In my experience as a developer, this is very very common requirement. Luckily I am not forced to use a low level language for any of my work. In C# I can simply do this: "hello world".ToUpper();
- Smaug123 2y ago
  If you're putting data into a third-party system, you might want `ToUpperInvariant`, not `ToUpper`. (Just checking that you know the difference, because most people don't!)
- pjmlp 2y ago
  Note that the correct way to do that in C# would be to pass an instance of CultureInfo.
- crote 2y ago
  The problem is that such third-party requirements are usually wrong. Two decades ago some developer probably went "Yeah, obviously all names start with capital letters!", not realizing that there are in fact plenty of names which start with a lowercase letter. So they added an input validation test which checks for capitals, which meant everyone feeding that system had to format their data. A whole ecosystem grew around the format of the output of that system, and now you're suddenly rewriting the system and you run into weird and plain wrong capitalization requirements for no technical reason whatsoever. Alternatively, the same but start with punch cards which predate ASCII and don't distinguish between uppercase and lowercase letters. > In C# I can simply do this: "hello world".ToUpper() ... which does not work. Take a look at the German word "straße" (street), for example. Until very recently the "ß" character did not have an uppercase variant, so a ToUpper would convert it to "STRASSE". This is a lossy operation, as the reverse isn't true: the lowercase variant of "KONGRESSSTRASSE" (congress street) is not "kongreßstraße" - it's supposed to be "Kongressstraße". It can get even worse: the phrase "in Maßen" (in moderate amounts) naively has the uppercase variant "IN MASSEN" - but that means "in huge amounts"! In that case it is probably better to stick to "IN MASZEN". And then there's Turkish, where the uppercase variant of the letter "i" is of course "İ" rather than "I" - note the dot. So no, you cannot "simply" use ToUpper() / ToLower(). They might work well enough of basic ASCII for languages like English, but they have a habit of making a mess out of everything else. You're supposed to use CultureInfo.TextInfo.ToUpperCase() and explicitly specify what locale the text is in so that it can use the right converter. Which is of course essentially impossible in general-purpose text fields. In practice that means your options are a) giving up on the concept of uppercase/lowercase conversion and just passing it as-is, or b) accepting that you are inevitably going to be silently corrupting your data.
jameshart 2y ago
I don’t think you can say this is universally known in ‘game dev’. In fact just last week I stumbled using the UI in a game that let me enter a name for something, which it then displayed in uppercase. Game UI is the place I’d expect to most likely come across horrific abuses of localization precisely because game UI is such a cobbled together layer of hacks on hacks.

zahlman 2y ago

>If text is coming from user - then change design until its not needed to 'convert' In games, you can possibly get away with this. Most other people need to worry about things like string collation (locale-aware sorting) for user-supplied text.

makeitdouble 2y ago
TBF, if you are caring about string collation, you're already at the entrance of the rabbit hole and probably should go down to the deep end anyway. I'd assume SleepyMyroslav doesn't apply to devs willing to spend weeks at time to handle all the complexity in full.

fluoridation 2y ago

>Once ppl learn about localization the questions like why a programming language does not do this 'simple text operation' are just a newcomer detector. :) I think you are purposefully misinterpreting the question. They're not asking about converting the case of any Unicode string with locale sensitivity, they're asking about converting the case of ASCII characters. What if your game needs to talk to a server and do some string manipulation in between requests? Are you really going to architect everything so that the client doesn't need to handle any of that ever?

squeaky-clean 2y ago
> They're not asking about converting the case of any Unicode string with locale sensitivity, they're asking about converting the case of ASCII characters. I'm confused now. The article specifically mentions issues with UTF-16 and UTF-32 unicode characters outside the basic multilingual plane (BMP).
- fluoridation 2y ago
  I'm referring to the people who call case conversion in general "a simple text operation". Say you have an std::string and you want to make it lower case. If you assume it contains just ASCII that's a simpler operation than if you assume it contains UTF-8, but C++ doesn't provide a single function that does either of them. A person can rightly complain that the former is a basic functionality that the language should include; personally, I would agree. And you could say "wow, doesn't this person realize that case conversion in Unicode is actually complicated? They must be really inexperienced." It could be that the other person really doesn't know about Unicode, or it could mean that you and them are thinking about entirely different problems and you're being judgemental a bit too eagerly.
  - squeaky-clean 2y ago
    For ascii in C++ isn't there std::tolower / std::toupper? If you're not dealing with unsigned char types there isn't a simple case conversion function, but that's for a good reason as the article lays out.
    - fluoridation 2y ago
      Those functions take and return single characters. What's missing is functions that operate on strings. You can use them in combination with std::transform(), but as the article points out, even if you're just dealing with ASCII you can easily do it wrong. I've been using C++ for over 20 years and I didn't know tolower() and toupper() were non-addressable. There's really no excuse for the library not having simple case conversion functions that operate on strings in-place.

deleted 2y ago

[deleted]

beeboobaa3 2y ago

> There are major issues just to show user back what he entered! Because the font for editing and displayed text could be different. Not even mentioning RTL and other issues. Your web browser is doing it right now as you are reading this comment.

rty32 2y ago
And web development is not game development? And chances are that games don't ship chromium with them?
- moron4hire 2y ago
  Actually... https://github.com/baikety/uWebKit https://zenfulcrum.com/browser/docs/Readme.html https://github.com/roydejong/chromium-unity-server There are a lot more, I just got bored at 3. And it's not just Unity. Several exist for Unreal as well. Why? Specifically because 2D layout and text rendering suck so much in game engines. What's ~50MB matter when you're shipping several GB of game assets?

cheema33 2y ago

> In gamedev there is simple rule: don't try to do any of that. I am not in gamedev, but I frequently have to develop middleware that takes in user entered data and formats it in a way that will import into a 3rd party system without errors. And that sometimes means changing the case on strings. In my experience as a developer, this is very very common requirement. Luckily I am not forced to use a low level language for any of my work. In C# I can simply do this: "hello world".ToUpper();

Smaug123 2y ago
If you're putting data into a third-party system, you might want `ToUpperInvariant`, not `ToUpper`. (Just checking that you know the difference, because most people don't!)
pjmlp 2y ago
Note that the correct way to do that in C# would be to pass an instance of CultureInfo.
crote 2y ago
The problem is that such third-party requirements are usually wrong. Two decades ago some developer probably went "Yeah, obviously all names start with capital letters!", not realizing that there are in fact plenty of names which start with a lowercase letter. So they added an input validation test which checks for capitals, which meant everyone feeding that system had to format their data. A whole ecosystem grew around the format of the output of that system, and now you're suddenly rewriting the system and you run into weird and plain wrong capitalization requirements for no technical reason whatsoever. Alternatively, the same but start with punch cards which predate ASCII and don't distinguish between uppercase and lowercase letters. > In C# I can simply do this: "hello world".ToUpper() ... which does not work. Take a look at the German word "straße" (street), for example. Until very recently the "ß" character did not have an uppercase variant, so a ToUpper would convert it to "STRASSE". This is a lossy operation, as the reverse isn't true: the lowercase variant of "KONGRESSSTRASSE" (congress street) is not "kongreßstraße" - it's supposed to be "Kongressstraße". It can get even worse: the phrase "in Maßen" (in moderate amounts) naively has the uppercase variant "IN MASSEN" - but that means "in huge amounts"! In that case it is probably better to stick to "IN MASZEN". And then there's Turkish, where the uppercase variant of the letter "i" is of course "İ" rather than "I" - note the dot. So no, you cannot "simply" use ToUpper() / ToLower(). They might work well enough of basic ASCII for languages like English, but they have a habit of making a mess out of everything else. You're supposed to use CultureInfo.TextInfo.ToUpperCase() and explicitly specify what locale the text is in so that it can use the right converter. Which is of course essentially impossible in general-purpose text fields. In practice that means your options are a) giving up on the concept of uppercase/lowercase conversion and just passing it as-is, or b) accepting that you are inevitably going to be silently corrupting your data.

jameshart 2y ago

I don’t think you can say this is universally known in ‘game dev’. In fact just last week I stumbled using the UI in a game that let me enter a name for something, which it then displayed in uppercase. Game UI is the place I’d expect to most likely come across horrific abuses of localization precisely because game UI is such a cobbled together layer of hacks on hacks.