3 ms·

std::string is basically just the following: struct std_string { union { char small[16]; struct { size_t len; char* buf; } big;

std::string is basically just the following: struct std_string { union { char small[16]; struct { size_t len; char* buf; } big; } value; unsigned char is_big; }; There is nothing it does regarding support for non-ASCII characters over what you get from buf and len. And for UTF-8, you don't even need len, plain old strlen from the 1980s works fine on valid UTF-8, so plain char* from C works just as well. And the union is just a performance optimization for small strings (is_big is probably not an additional field in good impls, but I separated it here), it's logically identical to just the buf and len. Which is all just to say, go for it with tcc!

bullen 5y ago
Yes, but the UTF-8 logic, how complex is that? I don't even know what union does, but I can imagine you have many smaller chunks?
- dundarious 5y ago
  The union has nothing to do with UTF-8, so let's ignore it. If you want more details about it, search for "c++ small string optimization", but the one-sentence version is that it's just a way to avoid a heap allocation for strings <= 16 bytes long (including any NUL terminator). So ignoring that irrelevant optimization, std::string is basically: struct std_string { size_t len; char* buf; }; For storing valid UTF-8, the len is unnecessary, since a NUL byte is not valid UTF-8. You can still tell how many bytes of UTF-8 you have by using strlen, because when you find a 0 byte, it is never part of the string, it's always the NUL terminator. So the len is not strictly necessary for valid UTF-8 -- leaving us with just char*. And not trying to dodge your question about UTF-8 logic, but my point was you can dodge that whole question, because std::string provides the same amount of UTF-8 logic as char* -- that is, none at all. If you've been getting by on std::string, then you can get by on char*. If you only need to support UTF-8 input and output, and you don't need to manipulate strings (replace characters, truncate them, normalize them for use as keys in a data structure, etc.) or only need to do simple substring searches for ASCII characters, then you can just use char* or std::string. UTF-8 has a great design, which was consciously chosen to make all of that possible.

bullen 5y ago

Yes, but the UTF-8 logic, how complex is that? I don't even know what union does, but I can imagine you have many smaller chunks?

dundarious 5y ago
The union has nothing to do with UTF-8, so let's ignore it. If you want more details about it, search for "c++ small string optimization", but the one-sentence version is that it's just a way to avoid a heap allocation for strings <= 16 bytes long (including any NUL terminator). So ignoring that irrelevant optimization, std::string is basically: struct std_string { size_t len; char* buf; }; For storing valid UTF-8, the len is unnecessary, since a NUL byte is not valid UTF-8. You can still tell how many bytes of UTF-8 you have by using strlen, because when you find a 0 byte, it is never part of the string, it's always the NUL terminator. So the len is not strictly necessary for valid UTF-8 -- leaving us with just char*. And not trying to dodge your question about UTF-8 logic, but my point was you can dodge that whole question, because std::string provides the same amount of UTF-8 logic as char* -- that is, none at all. If you've been getting by on std::string, then you can get by on char*. If you only need to support UTF-8 input and output, and you don't need to manipulate strings (replace characters, truncate them, normalize them for use as keys in a data structure, etc.) or only need to do simple substring searches for ASCII characters, then you can just use char* or std::string. UTF-8 has a great design, which was consciously chosen to make all of that possible.