3 ms·

All operating systems have the unicode database saved somewhere. There should just be a standard way of accessing it. Just like filesystem. Edit; that is; it d

All operating systems have the unicode database saved somewhere. There should just be a standard way of accessing it. Just like filesystem. Edit; that is; it does not have to be linked in the standard lib. Can be a data file somewhere, or a a shared lib.

duskwuff 4y ago
There's precedent for this, too: time zones! Time zone data can change over time, and as such it's typically stored in system files and loaded at runtime, rather than being embedded in executables. Locales have some similar behavior as well.
thwarted 4y ago
Like the timezone database, many things of which ship(ped) with their own copy.
- apaprocki 4y ago
  It's funny how much time I (and probably a number of others) spend undoing this decision and making things all refer to one copy of the data. There's usually also one per programming language/runtime, so the list only grows as new languages/runtime keep coming into the picture. Everyone's so good at patching vulnerabilities, but did you plan for your app breaking in production for all users in particular timezones because a random service in a chain of a 6, say, re-implemented in Go last quarter, or maybe... changed their Node app to port from using Moment to the standard Intl object 10 months ago, thus depending on the ICU C++ library, and a separate copy of the data that is no longer correct? Try to assess that risk and your head will hurt. These are the sharp edges in a polyglot environment that usually aren't considered when deciding to bring in new languages. Unicode CLDR has the same issue, but the pain usually isn't acute because it takes much, much longer for new glyphs or other data to appear in common use or start appearing in data feeds or from OS APIs/input. As long as you're at least on some update schedule, it's usually fine, and there's usually fewer things to update.
  - duskwuff 4y ago
    Yeah, updates to the Unicode database are generally pretty low-impact. For some time now, the properties of all existing characters have been frozen -- the updates are just adding new characters, most of which are either 1) extremely obscure languages that you'll probably never run into or 2) emoji. Either way, you're unlikely to see anything go horribly wrong because you're using an older version of the Unicode standard. Time zones, on the other hand...
    - int_19h 4y ago
      Have Unicode character properties ever not been frozen? If I remember correctly, there are some misnamed characters from 1.0 that still remain as such in perpetuity for the sake of compatibility.

duskwuff 4y ago

There's precedent for this, too: time zones! Time zone data can change over time, and as such it's typically stored in system files and loaded at runtime, rather than being embedded in executables. Locales have some similar behavior as well.

thwarted 4y ago

Like the timezone database, many things of which ship(ped) with their own copy.

apaprocki 4y ago
It's funny how much time I (and probably a number of others) spend undoing this decision and making things all refer to one copy of the data. There's usually also one per programming language/runtime, so the list only grows as new languages/runtime keep coming into the picture. Everyone's so good at patching vulnerabilities, but did you plan for your app breaking in production for all users in particular timezones because a random service in a chain of a 6, say, re-implemented in Go last quarter, or maybe... changed their Node app to port from using Moment to the standard Intl object 10 months ago, thus depending on the ICU C++ library, and a separate copy of the data that is no longer correct? Try to assess that risk and your head will hurt. These are the sharp edges in a polyglot environment that usually aren't considered when deciding to bring in new languages. Unicode CLDR has the same issue, but the pain usually isn't acute because it takes much, much longer for new glyphs or other data to appear in common use or start appearing in data feeds or from OS APIs/input. As long as you're at least on some update schedule, it's usually fine, and there's usually fewer things to update.
- duskwuff 4y ago
  Yeah, updates to the Unicode database are generally pretty low-impact. For some time now, the properties of all existing characters have been frozen -- the updates are just adding new characters, most of which are either 1) extremely obscure languages that you'll probably never run into or 2) emoji. Either way, you're unlikely to see anything go horribly wrong because you're using an older version of the Unicode standard. Time zones, on the other hand...
  - int_19h 4y ago
    Have Unicode character properties ever not been frozen? If I remember correctly, there are some misnamed characters from 1.0 that still remain as such in perpetuity for the sake of compatibility.