3 ms·

Maybe I'm just a terrible programmer, but I think the author may be a little over emphasizing the seriousness of the bug. To me, this is one of those throw away

Maybe I'm just a terrible programmer, but I think the author may be a little over emphasizing the seriousness of the bug. To me, this is one of those throw away issues that you keep in the back of your mind. Unless I'm coding something that is extremely datacentric and critical in that sense. But it's not like its on one of my "top 20 must run tests" or anything. It's always one of those issues I ignore or assume is correct until I find out it isn't. When I find out, it's simply a matter of tossing in a code page translation at either the input or output end and I'm done with it. Or maybe I've just been fortunate enough to be in an environment where an occasional goof of this caliber doesn't have any serious consequences.

carbonica 15y ago
> Or maybe I've just been fortunate enough to be in an environment where an occasional goof of this caliber doesn't have any serious consequences. Primarily, this means you don't have to support internationalization - which is hardly a bad thing, especially if you work on a startup, where worldwide distribution should be the last thing on your mind. When your product is rendered in over 80 scripts, including right-to-left languages, you can't afford to figure encoding will sort itself later.
- city41 15y ago
  And Hanselman works for Microsoft, where i18n is a big deal. So yes, for someone who's been at MS for a while i18n related issues become second nature. But if you typically are only targeting the United States, it's more understandable to not have these things on the brain.
  - finnh 15y ago
    On the other hand, Outlook still has a ridiculous bug that uses the wrong encoding when presenting HTML email - that is, it uses the encoding of the email's text-body when presenting the html-body, even if the html-body specifies a different encoding. So, if the two bodies have differing encodings (charsets), then the HTML body will look wrong. Unless you force Outlook to always use UTF-8 for encoding emails (which is a setting, but not the default) then you'll end up sending emails that will look garbled to your recipient. This "differing charset" scenario actually happens pretty frequently, because of the following scenario: a) You write an email (or reply to an existing email - actually it happens most with replies). b) Outlook's text editor decides to insert a non-breaking space (codepoint U+00A0). Perhaps it generates HTML with   but before transmission this eventually turns into the single UTF-8 character 0xC2 0xA0. c) When generating the text-body, Outlook decides to just use a plain old space, so the text body is plain ASCII. d) Outlook, in its cleverness, then says "ooh, I can 'conserve' encoding-ness and use plain old iso-8859-1 for the text body, but I need to use UTF-8 for the HTML body because of that non-ascii character" e) Outlook generates this email (please excuse formatting woes due to HN). Content-Type: multipart/alternative; boundary="0016e64dbd929784310488b2b082" This is a multi-part message in MIME format. --0016e64dbd929784310488b2b082 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="ISO-8859-1" yo yo --0016e64dbd929784310488b2b082 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <html> <body> yo=C2=A0yo </body> <html> --0016e64dbd929784310488b2b082-- When you view the above email in Outlook, you see "yoÂ yo" instead of "yo yo"
    - city41 15y ago
      You should file this at connect.microsoft.com. MS devs and PMs really do read and triage bug reports coming from there. The more details you put in the bug report, the better.
anigbrowl 15y ago
You're a fine programmer, but you're terrible at UX. I can put up with this sort of thing in a command-line application, but in any other context it's at least as distracting as non-standard GUI widgets or broken color management. Although these are cosmetic rather than functional issues, they're terribly distracting ones.
KeyBoardG 15y ago
I wouldn't take it seriously. The article is written very tongue in cheek. If you follow Hanselman you'll see he's a funny guy.
skimbrel 15y ago
You're not a terrible programmer, just an uninformed and/or slightly Anglocentric one. Just last week, my co-worker had to waste two days debugging a two-year-old Sphinx setup (the person who implemented it no longer works with us) because a Japanese user of our blogging service wasn't seeing the post he was looking for in our search feature. The problem was that the conduit feeding the Sphinx indexer was handling Unicode incorrectly (to be specific, it was deleting certain bytes wholesale because this guy believed them never to be valid, and this broke multi-byte sequences horribly). Those two days would have been much better spent working on his current project. Additionally, not handling Unicode correctly can leave you open to certain types of security holes! The biggest of these is probably that Unicode means _a string is not an array of bytes_, so naïve allocators for languages with byte-array strings (read: C and its brethren) are susceptible to buffer overruns when handed multi-byte Unicode sequences when they're expecting ASCII. Here's another one: the Unicode character space contains many, many glyphs that look almost (or in some cases exactly) like ASCII characters. RFC 3492 defines a framework for internationalized domain names (IDN) that encodes non-ASCII code points using ASCII characters, and the most common implementations of this transparently go between the two. This means that you could register "bаnkofamerica.com" (actually xn--bnkofamerica-x9j.com) and put a phishing site there, and people would happily click on the identical-looking URL and give you their bank account. This was pointed out several years ago and most modern browsers have mechanisms in place to defend against it, but your custom application might not unless you're careful to check what you're doing. There are plenty of blog posts and articles out there designed to tell you how to be safe when dealing with Unicode (and you should assume that you will be). I highly suggest you go read one.
ak217 15y ago
Wrong. By neglecting to properly process encoding, you shut your application off from everyone except the English-speaking population of the world (and even so, with bugs like this one). Such mistakes were excusable in the 90s. They're the sign of an amateur programmer today.

carbonica 15y ago

> Or maybe I've just been fortunate enough to be in an environment where an occasional goof of this caliber doesn't have any serious consequences. Primarily, this means you don't have to support internationalization - which is hardly a bad thing, especially if you work on a startup, where worldwide distribution should be the last thing on your mind. When your product is rendered in over 80 scripts, including right-to-left languages, you can't afford to figure encoding will sort itself later.

city41 15y ago
And Hanselman works for Microsoft, where i18n is a big deal. So yes, for someone who's been at MS for a while i18n related issues become second nature. But if you typically are only targeting the United States, it's more understandable to not have these things on the brain.
- finnh 15y ago
  On the other hand, Outlook still has a ridiculous bug that uses the wrong encoding when presenting HTML email - that is, it uses the encoding of the email's text-body when presenting the html-body, even if the html-body specifies a different encoding. So, if the two bodies have differing encodings (charsets), then the HTML body will look wrong. Unless you force Outlook to always use UTF-8 for encoding emails (which is a setting, but not the default) then you'll end up sending emails that will look garbled to your recipient. This "differing charset" scenario actually happens pretty frequently, because of the following scenario: a) You write an email (or reply to an existing email - actually it happens most with replies). b) Outlook's text editor decides to insert a non-breaking space (codepoint U+00A0). Perhaps it generates HTML with   but before transmission this eventually turns into the single UTF-8 character 0xC2 0xA0. c) When generating the text-body, Outlook decides to just use a plain old space, so the text body is plain ASCII. d) Outlook, in its cleverness, then says "ooh, I can 'conserve' encoding-ness and use plain old iso-8859-1 for the text body, but I need to use UTF-8 for the HTML body because of that non-ascii character" e) Outlook generates this email (please excuse formatting woes due to HN). Content-Type: multipart/alternative; boundary="0016e64dbd929784310488b2b082" This is a multi-part message in MIME format. --0016e64dbd929784310488b2b082 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="ISO-8859-1" yo yo --0016e64dbd929784310488b2b082 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <html> <body> yo=C2=A0yo </body> <html> --0016e64dbd929784310488b2b082-- When you view the above email in Outlook, you see "yoÂ yo" instead of "yo yo"
  - city41 15y ago
    You should file this at connect.microsoft.com. MS devs and PMs really do read and triage bug reports coming from there. The more details you put in the bug report, the better.

anigbrowl 15y ago

You're a fine programmer, but you're terrible at UX. I can put up with this sort of thing in a command-line application, but in any other context it's at least as distracting as non-standard GUI widgets or broken color management. Although these are cosmetic rather than functional issues, they're terribly distracting ones.

KeyBoardG 15y ago

I wouldn't take it seriously. The article is written very tongue in cheek. If you follow Hanselman you'll see he's a funny guy.

skimbrel 15y ago

You're not a terrible programmer, just an uninformed and/or slightly Anglocentric one. Just last week, my co-worker had to waste two days debugging a two-year-old Sphinx setup (the person who implemented it no longer works with us) because a Japanese user of our blogging service wasn't seeing the post he was looking for in our search feature. The problem was that the conduit feeding the Sphinx indexer was handling Unicode incorrectly (to be specific, it was deleting certain bytes wholesale because this guy believed them never to be valid, and this broke multi-byte sequences horribly). Those two days would have been much better spent working on his current project. Additionally, not handling Unicode correctly can leave you open to certain types of security holes! The biggest of these is probably that Unicode means _a string is not an array of bytes_, so naïve allocators for languages with byte-array strings (read: C and its brethren) are susceptible to buffer overruns when handed multi-byte Unicode sequences when they're expecting ASCII. Here's another one: the Unicode character space contains many, many glyphs that look almost (or in some cases exactly) like ASCII characters. RFC 3492 defines a framework for internationalized domain names (IDN) that encodes non-ASCII code points using ASCII characters, and the most common implementations of this transparently go between the two. This means that you could register "bаnkofamerica.com" (actually xn--bnkofamerica-x9j.com) and put a phishing site there, and people would happily click on the identical-looking URL and give you their bank account. This was pointed out several years ago and most modern browsers have mechanisms in place to defend against it, but your custom application might not unless you're careful to check what you're doing. There are plenty of blog posts and articles out there designed to tell you how to be safe when dealing with Unicode (and you should assume that you will be). I highly suggest you go read one.

ak217 15y ago

Wrong. By neglecting to properly process encoding, you shut your application off from everyone except the English-speaking population of the world (and even so, with bugs like this one). Such mistakes were excusable in the 90s. They're the sign of an amateur programmer today.