4 ms·

Well, at least Windows' use of Unicode goes back to the late 1980s and back then the 16-bitness of Unicode still seemed like a sensible idea and UTF-8 definitel

by ygra 6y ago

Well, at least Windows' use of Unicode goes back to the late 1980s and back then the 16-bitness of Unicode still seemed like a sensible idea and UTF-8 definitely didn't exist yet.

bawolff 6y ago
Hmm. This stuff is hard to google, but based on https://docs.microsoft.com/en-us/previous-versions/cc194797(v=msdn.10)?redirectedfrom=MSDN https://docs.microsoft.com/en-us/previous-versions/cc194797(... it looks like windows NT was the first version to support UCS-2 (in 1993), so i think utf-8 did exist at that point in time, but its pretty hard to blame microsoft for going with ucs-2 at that point in time.
- chrismorgan 6y ago
  I find it in myself to blame them. They introduced a whole new completely incompatible encoding (or two, because of endianness—and that led eventually to the BOM), rather than going with a scheme that would maintain backwards compatibility with ASCII. Even if it was not yet apparent that UCS-2 would be insufficient (not sure about that one way or the other), UCS-2 should have compared unfavourably to UTF-8, with fixed encoding length being its only benefit—and that, of course, was quickly stripped away with the disaster that is UTF-16 that ruined Unicode for everyone with surrogates, and it took about another 15 years before the UCS-2/UTF-16 mess was resolved nearly everywhere.
  - ygra 6y ago
    NT was being designed starting in 1988 or 1989. You don't change your character encoding throughout the operating system on a whim a year before release, though.
    - chrismorgan 6y ago
      That’s fair, and careless of me to forget about. I wasn’t around at the time, but the whole A/W split still seems a bit strange to me—that most of the problems that followed only really happened because Microsoft blessed UCS-2 in this way (“use codepages, or Unicode via UCS-2”), rather than continuing to use the codepage mechanism to distinguish things, which would have allowed an easy transition to UTF-8 (where even switching shortly before release would probably have been quite feasible) rather than being stuck trying to perpetuate something bad by making it even worse. (Sure, there would have been different problems using the codepage mechanism in this way, but I think they would have been lesser problems and that they would have led to a much happier future where UTF-16 might even never have happened.) As it is, they’ve finally headed in this direction of using a UTF-8 codepage on the A functions, but only in the last three years. It’s a pity they didn’t do it this way from the start (well, with some other Unicode encoding that existed), 30 years ago.
  - gpderetta 6y ago
    Well, when windows NT was getting implemented UTF-8 hadn't been invented yet. Also as Windows NT was a new OS, there wasn't a big need for backward compatibility.
    - chrismorgan 6y ago
      This was most importantly about backwards compatibility with files. You definitely still care about that cross-platform. (Sure, there is also the API split where now you have two sets of functions, the A and the W variants, so that UCS-2 made things much more complicated for ever after, compared with implementing Unicode as a code page, by whatever means. Compatibility was not quite such a big deal there, though the approach they took seems to me to have been a poor choice. Again there were reasons why they did it the way they did, but those reasons have aged very poorly.)

by ygra 6y ago

Well, at least Windows' use of Unicode goes back to the late 1980s and back then the 16-bitness of Unicode still seemed like a sensible idea and UTF-8 definitely didn't exist yet.

bawolff 6y ago

Hmm. This stuff is hard to google, but based on https://docs.microsoft.com/en-us/previous-versions/cc194797(v=msdn.10)?redirectedfrom=MSDN https://docs.microsoft.com/en-us/previous-versions/cc194797(... it looks like windows NT was the first version to support UCS-2 (in 1993), so i think utf-8 did exist at that point in time, but its pretty hard to blame microsoft for going with ucs-2 at that point in time.

chrismorgan 6y ago
I find it in myself to blame them. They introduced a whole new completely incompatible encoding (or two, because of endianness—and that led eventually to the BOM), rather than going with a scheme that would maintain backwards compatibility with ASCII. Even if it was not yet apparent that UCS-2 would be insufficient (not sure about that one way or the other), UCS-2 should have compared unfavourably to UTF-8, with fixed encoding length being its only benefit—and that, of course, was quickly stripped away with the disaster that is UTF-16 that ruined Unicode for everyone with surrogates, and it took about another 15 years before the UCS-2/UTF-16 mess was resolved nearly everywhere.
- ygra 6y ago
  NT was being designed starting in 1988 or 1989. You don't change your character encoding throughout the operating system on a whim a year before release, though.
  - chrismorgan 6y ago
    That’s fair, and careless of me to forget about. I wasn’t around at the time, but the whole A/W split still seems a bit strange to me—that most of the problems that followed only really happened because Microsoft blessed UCS-2 in this way (“use codepages, or Unicode via UCS-2”), rather than continuing to use the codepage mechanism to distinguish things, which would have allowed an easy transition to UTF-8 (where even switching shortly before release would probably have been quite feasible) rather than being stuck trying to perpetuate something bad by making it even worse. (Sure, there would have been different problems using the codepage mechanism in this way, but I think they would have been lesser problems and that they would have led to a much happier future where UTF-16 might even never have happened.) As it is, they’ve finally headed in this direction of using a UTF-8 codepage on the A functions, but only in the last three years. It’s a pity they didn’t do it this way from the start (well, with some other Unicode encoding that existed), 30 years ago.
- gpderetta 6y ago
  Well, when windows NT was getting implemented UTF-8 hadn't been invented yet. Also as Windows NT was a new OS, there wasn't a big need for backward compatibility.
  - chrismorgan 6y ago
    This was most importantly about backwards compatibility with files. You definitely still care about that cross-platform. (Sure, there is also the API split where now you have two sets of functions, the A and the W variants, so that UCS-2 made things much more complicated for ever after, compared with implementing Unicode as a code page, by whatever means. Compatibility was not quite such a big deal there, though the approach they took seems to me to have been a poor choice. Again there were reasons why they did it the way they did, but those reasons have aged very poorly.)