3 ms·

> The minimum every software developer must know about Unicode Just a nitpick... Once more, as it is typical on HN, web programming is confused with the entir

by pif 3y ago

> The minimum every software developer must know about Unicode Just a nitpick... Once more, as it is typical on HN, web programming is confused with the entire universe of software development. There are plenty of software realms where ASCII not only is enough, but it actually MUST be enough.

lxgr 3y ago
What do you mean by “must be enough”? Not being able to support non-latin scripts sounds more like a limitation than a feature to me, although of course in many contexts it’s not in any individual organizations power to overcome it.
9dev 3y ago
Well, proper Unicode support affects pretty much any area handling data about, used by, or created by, humans. That’s a pretty broad scope, and certainly wider than just web software.
uxp8u61q 3y ago
This kind of assertiveness leads to garbage like C++ still not supporting UTF8 properly in 2023. My name contains diacritics. I am so, so, so tired of trying to work around information systems - not just web frontends - designed by people who don't care or worse, don't want to care. "Web" programmers can care all they want about Unicode, but if the backend people didn't deal properly with text encoding, then something will break no matter what. > There are plenty of software realms where ASCII not only is enough, but it actually MUST be enough. Name one.
- pif 3y ago
  > if the backend people didn't deal properly You are right. It's not a frontend/backend issue. It's a "for human" vs "not for human" issues. Personal names must be treated in an international-friendly manner. >> There are plenty of software realms where ASCII not only is enough, but it actually MUST be enough. > > Name one Joel himself described an example: > It would be convenient if you could put the Content-Type of the HTML file right in the HTML file itself, using some kind of special tag. Of course this drove purists crazy… how can you read the HTML file until you know what encoding it’s in?! Luckily, almost every encoding in common use does the same thing with characters between 32 and 127, so you can always get this far on the HTML page without starting to use funny letters: The content of a webpage is required to be expressed in every supported language, but the HTTP protocol must not. And it would make no sense at all to add internationalization to intra-machines protocol, where ASCII is enough and has been enough for decades. And if someone complains that ASCII only supports English, well... suck it up! I'm Italian and work in French, still I hate when a colleague sneaks in a comment not in English. Professional software development happens in English.
  - uxp8u61q 3y ago
    > The content of a webpage is required to be expressed in every supported language, but the HTTP protocol must not. And it would make no sense at all to add internationalization to intra-machines protocol, where ASCII is enough and has been enough for decades. I guess no URLs with funny characters then. "GET /profile/renée" => 500 error, woohoo. > And if someone complains that ASCII only supports English, well... suck it up! I'm Italian and work in French, still I hate when a colleague sneaks in a comment not in English. Professional software development happens in English. Get over yourself, a lot of professional development happens in languages other than English.
    - Dylan16807 3y ago
      > I guess no URLs with funny characters then. "GET /profile/renée" => 500 error, woohoo. That's not really a slam dunk. Lots of sites don't let you have your name in the URL at all, and the average person's experience is that their name would be taken by someone else before they signed up.
      - uxp8u61q 3y ago
        Try looking at the moon rather than the finger next time.
        Dylan16807 3y ago
        Why? I'm not obligated to take sides here, and just because I agree with your overall point doesn't mean I can't point out problems with your argument. In the metaphor, it's someone telling you your aim is off.
  - astrange 3y ago
    HTTP does support content-type tags and Unicode in URLs. Which funny enough comes in two different encodings, punycode and percent escapes.
- lelanthran 3y ago
  > This kind of assertiveness leads to garbage like C++ still not supporting UTF8 properly in 2023. My name contains diacritics. UTF8 encoded diacritics work just fine in C++.
  - uxp8u61q 3y ago
    What do you mean by "work"? That you can store arbitrary bytes in a string? That's a pretty low bar.
    - lelanthran 3y ago
      > What do you mean by "work"? That you can store arbitrary bytes in a string? That's a pretty low bar. That's all that's needed for a backend language. The backend does not need to understand, or even acknowledge the existence, of grapheme clusters. Because the frontend is already having to understand all of this, it should be normalising any multi-codepoint ambiguous cluster anyway.
      - JohnFen 3y ago
        The backend never needs to do things like figure out how long a string is or search for one string in a database of other strings?
        lelanthran 3y ago
        > The backend never needs to do things like figure out how long a string Not as measured by clusters, no. > search for one string in a database of other strings? Hence I said "normalisation". The frontend already has to do all the unicode twiddling, it may as well normalise the input too.
      - astrange 3y ago
        It does if it ever wants to trim, summarize, sort or compare equal a string.
- jeddy3 3y ago
  I can name one. At my job we do the kind of embedded programming were encoders inside machines send data to each other. Like reading optical sensors and sending bits indicating state to other controllers. We absolutely do not "need" to know about Unicode, outside of interest about other realms.

by pif 3y ago

lxgr 3y ago

What do you mean by “must be enough”? Not being able to support non-latin scripts sounds more like a limitation than a feature to me, although of course in many contexts it’s not in any individual organizations power to overcome it.

9dev 3y ago

Well, proper Unicode support affects pretty much any area handling data about, used by, or created by, humans. That’s a pretty broad scope, and certainly wider than just web software.

uxp8u61q 3y ago

This kind of assertiveness leads to garbage like C++ still not supporting UTF8 properly in 2023. My name contains diacritics. I am so, so, so tired of trying to work around information systems - not just web frontends - designed by people who don't care or worse, don't want to care. "Web" programmers can care all they want about Unicode, but if the backend people didn't deal properly with text encoding, then something will break no matter what. > There are plenty of software realms where ASCII not only is enough, but it actually MUST be enough. Name one.

pif 3y ago
> if the backend people didn't deal properly You are right. It's not a frontend/backend issue. It's a "for human" vs "not for human" issues. Personal names must be treated in an international-friendly manner. >> There are plenty of software realms where ASCII not only is enough, but it actually MUST be enough. > > Name one Joel himself described an example: > It would be convenient if you could put the Content-Type of the HTML file right in the HTML file itself, using some kind of special tag. Of course this drove purists crazy… how can you read the HTML file until you know what encoding it’s in?! Luckily, almost every encoding in common use does the same thing with characters between 32 and 127, so you can always get this far on the HTML page without starting to use funny letters: The content of a webpage is required to be expressed in every supported language, but the HTTP protocol must not. And it would make no sense at all to add internationalization to intra-machines protocol, where ASCII is enough and has been enough for decades. And if someone complains that ASCII only supports English, well... suck it up! I'm Italian and work in French, still I hate when a colleague sneaks in a comment not in English. Professional software development happens in English.
- uxp8u61q 3y ago
  > The content of a webpage is required to be expressed in every supported language, but the HTTP protocol must not. And it would make no sense at all to add internationalization to intra-machines protocol, where ASCII is enough and has been enough for decades. I guess no URLs with funny characters then. "GET /profile/renée" => 500 error, woohoo. > And if someone complains that ASCII only supports English, well... suck it up! I'm Italian and work in French, still I hate when a colleague sneaks in a comment not in English. Professional software development happens in English. Get over yourself, a lot of professional development happens in languages other than English.
  - Dylan16807 3y ago
    > I guess no URLs with funny characters then. "GET /profile/renée" => 500 error, woohoo. That's not really a slam dunk. Lots of sites don't let you have your name in the URL at all, and the average person's experience is that their name would be taken by someone else before they signed up.
    - uxp8u61q 3y ago
      Try looking at the moon rather than the finger next time.
      - Dylan16807 3y ago
        Why? I'm not obligated to take sides here, and just because I agree with your overall point doesn't mean I can't point out problems with your argument. In the metaphor, it's someone telling you your aim is off.
- astrange 3y ago
  HTTP does support content-type tags and Unicode in URLs. Which funny enough comes in two different encodings, punycode and percent escapes.
lelanthran 3y ago
> This kind of assertiveness leads to garbage like C++ still not supporting UTF8 properly in 2023. My name contains diacritics. UTF8 encoded diacritics work just fine in C++.
- uxp8u61q 3y ago
  What do you mean by "work"? That you can store arbitrary bytes in a string? That's a pretty low bar.
  - lelanthran 3y ago
    > What do you mean by "work"? That you can store arbitrary bytes in a string? That's a pretty low bar. That's all that's needed for a backend language. The backend does not need to understand, or even acknowledge the existence, of grapheme clusters. Because the frontend is already having to understand all of this, it should be normalising any multi-codepoint ambiguous cluster anyway.
    - JohnFen 3y ago
      The backend never needs to do things like figure out how long a string is or search for one string in a database of other strings?
      - lelanthran 3y ago
        > The backend never needs to do things like figure out how long a string Not as measured by clusters, no. > search for one string in a database of other strings? Hence I said "normalisation". The frontend already has to do all the unicode twiddling, it may as well normalise the input too.
    - astrange 3y ago
      It does if it ever wants to trim, summarize, sort or compare equal a string.
jeddy3 3y ago
I can name one. At my job we do the kind of embedded programming were encoders inside machines send data to each other. Like reading optical sensors and sending bits indicating state to other controllers. We absolutely do not "need" to know about Unicode, outside of interest about other realms.