3 ms·

> This is called Normalization Form Canonical Composition (NFC). Is there something like a "Round Midnight and No Coffee Form" where the programmer just render

> This is called Normalization Form Canonical Composition (NFC). Is there something like a "Round Midnight and No Coffee Form" where the programmer just renders the text to check whether the output of each set of codepoints matches pixel for pixel?

a1369209993 7y ago
The closest unicode has seems to be NFKC, but last I checked it still didn't correctly handle greek and cyrillic aliases of latin characters, never mind anything more obscure.
- LunaSea 7y ago
  Do you have examples of this? I wonder if it's implementation specific or due to the NFKC method itself?
  - dfox 7y ago
    The issue does not have anything to do with normalization per se, but stems from the fact that there are Unicode characters that are semantically different, but nevertheless look exactly same in most fonts. For example 'a' (U+0061 LATIN SMALL LETTER A) vs. 'а' (U+0430 CYRILLIC SMALL LETTER A)
    - ygra 7y ago
      It's probably also futile to keep homoglyph tables without the context of the font being used, as т and т are the same letter, just with different styles (and in some fonts resemble T and m respectively in Latin). Unicode mentions homoglyphs briefly in UTR #36, but overall I feel that it's not really their job to solve that issue, as visually similar characters already exist in ASCII and other limited character sets and every context where those things are a problem probably needs to be evaluated differently with different mitigations.
efitz 7y ago
I know you were joking but http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=1&f=G&l=50&co1=AND&d=PTXT&s1=10127388&OS=10127388&RS=10127388 http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=H...
stevenwoo 7y ago
I cooped at IBM in the 1980s and this is how we tested and verified the newest IBM PC hardware (at this point IIRC it would have been around the 286/pc junior/os2/smaller form factor ibm pc designs) by creating tests on current systems with manual keyboard entry, and recording all the key strokes/timing and screen buffers and output files at intervals then running the same tests on new hardware with a playback of recorded input and comparing the new output to the saved runs.

a1369209993 7y ago

The closest unicode has seems to be NFKC, but last I checked it still didn't correctly handle greek and cyrillic aliases of latin characters, never mind anything more obscure.

LunaSea 7y ago
Do you have examples of this? I wonder if it's implementation specific or due to the NFKC method itself?
- dfox 7y ago
  The issue does not have anything to do with normalization per se, but stems from the fact that there are Unicode characters that are semantically different, but nevertheless look exactly same in most fonts. For example 'a' (U+0061 LATIN SMALL LETTER A) vs. 'а' (U+0430 CYRILLIC SMALL LETTER A)
  - ygra 7y ago
    It's probably also futile to keep homoglyph tables without the context of the font being used, as т and т are the same letter, just with different styles (and in some fonts resemble T and m respectively in Latin). Unicode mentions homoglyphs briefly in UTR #36, but overall I feel that it's not really their job to solve that issue, as visually similar characters already exist in ASCII and other limited character sets and every context where those things are a problem probably needs to be evaluated differently with different mitigations.

efitz 7y ago

I know you were joking but http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=1&f=G&l=50&co1=AND&d=PTXT&s1=10127388&OS=10127388&RS=10127388 http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=H...

stevenwoo 7y ago

I cooped at IBM in the 1980s and this is how we tested and verified the newest IBM PC hardware (at this point IIRC it would have been around the 286/pc junior/os2/smaller form factor ibm pc designs) by creating tests on current systems with manual keyboard entry, and recording all the key strokes/timing and screen buffers and output files at intervals then running the same tests on new hardware with a playback of recorded input and comparing the new output to the saved runs.