4 ms·

Perhaps they use the same technology as ligatures? There could be a glyph for the standalone character, but also special glyphs for certain combos? The page sa

Perhaps they use the same technology as ligatures? There could be a glyph for the standalone character, but also special glyphs for certain combos? The page says they do handle variations: Pronunciation in the Cantonese Font adapts to the context. Based on what comes before or after, the Jyutping romanization changes to the right one. The magic behind this is a careful curation from 100,000 contexts where the pronunciation differs from the standalone character.

jeffparsons 4y ago
I've seen ligatures (or whatever the underlying feature is in font formats) used for some wild stuff, but this takes the cake. They're effectively encoding a small amount of natural language processing in a font. Setting aside for a minute the question of whether you _should_, I wonder how far you can take this? I.e. what limits are there on how much context you can take into account, etc.?
- TylerE 4y ago
  You'd like https://www.notationcentral.com/product/musglyphs/ https://www.notationcentral.com/product/musglyphs/
- philsnow 4y ago
  I think it also uses ligatures to render words from the "phrasebook", where you can take a word like "friend" and change the font from Times to this font, and it renders it as 朋友. Beautiful.
- jkwchui 4y ago
  (Font author) This is all off the beaten path, so I suspect the answer is no one knows. Font tables have a limit of 65k characters, but this ceiling can be busted in whacky ways using multiple lookups, useExtension... Practically, font building tools / operations crash (mysteriously), stalls (mysteriously), or slows to a crawl (indistinguishable from stalling), and the Cantonese Font about pushes the limit.
jkwchui 4y ago
Hello. Font's author here. You and Jeff are correct in guessing this is (ab)using ligatures maximally :) To satisfy your curiosity, we can go deeper. ---- Conceptually it is simple: 1. assign a default (most likely) sound for each character, 2. loop through contexts, extracting words (char-combos) where the sound is different from the default ("alt-word") 3. create SVGs + font-paths (fallback for incompatible systems) for every char and every alt-word 4. assign a ligature to substitute each char-sequence that forms the alt-word (e.g., "when 乾隆 appears adjacently, replace with `uniF1234` (the codepoint for the alt-word 乾隆") It is not perfect, but I didn't expect this to work so well, and was stunned when the testers report high accuracy. I have always believed that bespoke computation with word segmentation (with some 1M frequency attached library) and large data-bank (100k+ words) was necessary. ---- Practically it was horrific, tedious, mind-numbing, gawd-awful set of "why this doesn't work": 1. SVG automation that works for 10^3 breaks with 10^5 2. what worked for Latin breaks for unicode 3. what worked for unicode breaks for PUA 4. what worked for monochrome breaks for color 5. what worked for single glyphs breaks for ligatures 6. what?! The assignments in the database is wrong?? 7. [...] As I was trying to coerce the system to do what it wasn't designed to do, many of these breaks are undocumented, pretty mysterious to solve, and some steps just got manually gritted through. (And each of the 15k+ glyphs got gritted through about five times.) It does look pretty elegant at the end ;)
- creamyhorror 4y ago
  That is amazing work. You've really plumbed the depths of what's possible with font technology, kudos.
  - jkwchui 4y ago
    Thank you. Who is really amazing is Simon Cozens, who wrote a set of articles on fonts/global script: https://simoncozens.github.io/fonts-and-layout/ https://simoncozens.github.io/fonts-and-layout/ The history of digital fonts added a great deal of complexity to font formats, and without him writing such a concise yet comprehensive guide, I would have been stuck for even longer.
- ackfoobar 4y ago

jeffparsons 4y ago

I've seen ligatures (or whatever the underlying feature is in font formats) used for some wild stuff, but this takes the cake. They're effectively encoding a small amount of natural language processing in a font. Setting aside for a minute the question of whether you _should_, I wonder how far you can take this? I.e. what limits are there on how much context you can take into account, etc.?

TylerE 4y ago
You'd like https://www.notationcentral.com/product/musglyphs/ https://www.notationcentral.com/product/musglyphs/
philsnow 4y ago
I think it also uses ligatures to render words from the "phrasebook", where you can take a word like "friend" and change the font from Times to this font, and it renders it as 朋友. Beautiful.
jkwchui 4y ago
(Font author) This is all off the beaten path, so I suspect the answer is no one knows. Font tables have a limit of 65k characters, but this ceiling can be busted in whacky ways using multiple lookups, useExtension... Practically, font building tools / operations crash (mysteriously), stalls (mysteriously), or slows to a crawl (indistinguishable from stalling), and the Cantonese Font about pushes the limit.

jkwchui 4y ago

Hello. Font's author here. You and Jeff are correct in guessing this is (ab)using ligatures maximally :) To satisfy your curiosity, we can go deeper. ---- Conceptually it is simple: 1. assign a default (most likely) sound for each character, 2. loop through contexts, extracting words (char-combos) where the sound is different from the default ("alt-word") 3. create SVGs + font-paths (fallback for incompatible systems) for every char and every alt-word 4. assign a ligature to substitute each char-sequence that forms the alt-word (e.g., "when 乾隆 appears adjacently, replace with `uniF1234` (the codepoint for the alt-word 乾隆") It is not perfect, but I didn't expect this to work so well, and was stunned when the testers report high accuracy. I have always believed that bespoke computation with word segmentation (with some 1M frequency attached library) and large data-bank (100k+ words) was necessary. ---- Practically it was horrific, tedious, mind-numbing, gawd-awful set of "why this doesn't work": 1. SVG automation that works for 10^3 breaks with 10^5 2. what worked for Latin breaks for unicode 3. what worked for unicode breaks for PUA 4. what worked for monochrome breaks for color 5. what worked for single glyphs breaks for ligatures 6. what?! The assignments in the database is wrong?? 7. [...] As I was trying to coerce the system to do what it wasn't designed to do, many of these breaks are undocumented, pretty mysterious to solve, and some steps just got manually gritted through. (And each of the 15k+ glyphs got gritted through about five times.) It does look pretty elegant at the end ;)

creamyhorror 4y ago
That is amazing work. You've really plumbed the depths of what's possible with font technology, kudos.
- jkwchui 4y ago
  Thank you. Who is really amazing is Simon Cozens, who wrote a set of articles on fonts/global script: https://simoncozens.github.io/fonts-and-layout/ https://simoncozens.github.io/fonts-and-layout/ The history of digital fonts added a great deal of complexity to font formats, and without him writing such a concise yet comprehensive guide, I would have been stuck for even longer.
ackfoobar 4y ago