3 ms·

Also, C++11 adds std::u16string and std::u32string to make life a bit easier and write cross-platform code without worrying about whether wchar_t is 2 bytes (e.

Also, C++11 adds std::u16string and std::u32string to make life a bit easier and write cross-platform code without worrying about whether wchar_t is 2 bytes (e.g. Windows) or 4 bytes (e.g. Linux).

malkia 14y ago
Then again I would avoid anything non-8 bit for asset parsing. We have tons of text files for our level maps (Quake map format), used to have (transitioninig) all models/animation in text format again. Not only this takes significant time to sync these off P4, but they take more space on the disk, later more I/O, memory, and you are the mercy of strod, atof, etc. to work fast and properly (which usually is not the case). For assets that don't make much sense merging (polygon data) - we are moving to binary data. For the rest (level data), we might keep text files - some merging is possible there. The only time we'll ever need UTF8, UTF16, etc. is for the localized data - game text, etc. There is absolutely no need for such encoding to be in your assets.
- lloeki 14y ago
  > Then again I would avoid anything non-8 bit for asset parsing. Nitpick: you probably meant non-7 bit (there's a reason why ruby and python2 default to non-ext ascii source code, and barf at you otherwise)
  - malkia 14y ago
    Yup. 7-bit, but the point is even if character above 127 were used it would've been okay. 7-bit (8-bit) ASCII has a long life to live :) It's just machine optimal, and the same time not flexible for modern days needs... just like "C", and I love it. I'm using Qt5 these days, and there is QString and QByteArray (and also QLatin1String) - and I need to keep in memory lots of properties for our editor, going to QString (wide-chars) just uses more memory for no reason. Sticking to QByteArray/QLatin1String makes more sense for them.
  - yxhuvud 14y ago
    The default changed to utf8 in Ruby for 2.0 though, so I suppose that reason is less obvious than you think.

Also, C++11 adds std::u16string and std::u32string to make life a bit easier and write cross-platform code without worrying about whether wchar_t is 2 bytes (e.g. Windows) or 4 bytes (e.g. Linux).

malkia 14y ago

Then again I would avoid anything non-8 bit for asset parsing. We have tons of text files for our level maps (Quake map format), used to have (transitioninig) all models/animation in text format again. Not only this takes significant time to sync these off P4, but they take more space on the disk, later more I/O, memory, and you are the mercy of strod, atof, etc. to work fast and properly (which usually is not the case). For assets that don't make much sense merging (polygon data) - we are moving to binary data. For the rest (level data), we might keep text files - some merging is possible there. The only time we'll ever need UTF8, UTF16, etc. is for the localized data - game text, etc. There is absolutely no need for such encoding to be in your assets.

lloeki 14y ago
> Then again I would avoid anything non-8 bit for asset parsing. Nitpick: you probably meant non-7 bit (there's a reason why ruby and python2 default to non-ext ascii source code, and barf at you otherwise)
- malkia 14y ago
  Yup. 7-bit, but the point is even if character above 127 were used it would've been okay. 7-bit (8-bit) ASCII has a long life to live :) It's just machine optimal, and the same time not flexible for modern days needs... just like "C", and I love it. I'm using Qt5 these days, and there is QString and QByteArray (and also QLatin1String) - and I need to keep in memory lots of properties for our editor, going to QString (wide-chars) just uses more memory for no reason. Sticking to QByteArray/QLatin1String makes more sense for them.
- yxhuvud 14y ago
  The default changed to utf8 in Ruby for 2.0 though, so I suppose that reason is less obvious than you think.