4 ms·

> Its quite common in my > language I'd love to hear more details on why?

colechristensen 10y ago
Ligatures. It's easy to not notice them at all in english ﬂ (ﬂ) vs. fl [ﬂy vs. fly] but some languages use them very extensively and the combinations are more significant. https://en.wikipedia.org/wiki/Zero-width_non-joiner https://en.wikipedia.org/wiki/Zero-width_non-joiner * It's entirely possible that the browser you're using isn't doing a very good job with ligatures which explains the strange look of my examples
satbyy 10y ago
ZWJ and ZWNJ are also common in Indic scripts. It's basically used to control the appearance of glyphs, for example half-forms and consonant clusters (क्‍ष vs क्ष, both are kṣa). As usual, wikipedia has good examples. The Unicode Standard also contains details about these. ZW[N]J as a standalone character or at the beginning of a word is very unusual on a day-to-day basis, so it's understandable that Twitter fails to recognize this pattern. ¹ https://en.wikipedia.org/wiki/ZWJ https://en.wikipedia.org/wiki/ZWJ ² https://en.wikipedia.org/wiki/ZWNJ https://en.wikipedia.org/wiki/ZWNJ
- peteretep 10y ago
  Ah ha! > When a ZWJ is placed between two > emoji characters, it can also result > in a new form being shown, such as > the family emoji, made up of two adult > emoji and one or two child emoji That makes a lot of sense too, and I hadn't put sufficient work into how that's implemented -- retrospectively that makes perfect sense.
  - dvcrn 10y ago
    I noticed that on new Emojis on my MacBook. Some of the new emojis like ‍ are rendered as "guy behind a MacBook" on my PC but on phones without the emoji as "guy emoji" and "computer emoji". Same for ‍️ (male version of raise hand). On phones without the Emoji, it's just "male emoji" and "female raise hand emoji". /e: oh, HN is stripping Emojis
  - vanderZwan 10y ago
    This made me wonder if anyone had tried combining word2vec with emojis, and then I came across this: https://github.com/uclmr/emoji2ve https://github.com/uclmr/emoji2ve
    - peteretep 10y ago
      which is a dead link
      - satbyy 10y ago
        Correct link: https://github.com/uclmr/emoji2vec https://github.com/uclmr/emoji2vec
        vanderZwan 10y ago
        Apologies, and thanks!
kagamine 10y ago
Not OP, but in Norwegian the correct way to write "Tom's car" is "Tom sin bil", the car of Tom. But the creep of English and laziness allows for "Toms car", esp. in informal writing.

Ligatures. It's easy to not notice them at all in english ﬂ (ﬂ) vs. fl [ﬂy vs. fly] but some languages use them very extensively and the combinations are more significant. https://en.wikipedia.org/wiki/Zero-width_non-joiner https://en.wikipedia.org/wiki/Zero-width_non-joiner * It's entirely possible that the browser you're using isn't doing a very good job with ligatures which explains the strange look of my examples

satbyy 10y ago

ZWJ and ZWNJ are also common in Indic scripts. It's basically used to control the appearance of glyphs, for example half-forms and consonant clusters (क्‍ष vs क्ष, both are kṣa). As usual, wikipedia has good examples. The Unicode Standard also contains details about these. ZW[N]J as a standalone character or at the beginning of a word is very unusual on a day-to-day basis, so it's understandable that Twitter fails to recognize this pattern. ¹ https://en.wikipedia.org/wiki/ZWJ https://en.wikipedia.org/wiki/ZWJ ² https://en.wikipedia.org/wiki/ZWNJ https://en.wikipedia.org/wiki/ZWNJ

peteretep 10y ago
Ah ha! > When a ZWJ is placed between two > emoji characters, it can also result > in a new form being shown, such as > the family emoji, made up of two adult > emoji and one or two child emoji That makes a lot of sense too, and I hadn't put sufficient work into how that's implemented -- retrospectively that makes perfect sense.
- dvcrn 10y ago
  I noticed that on new Emojis on my MacBook. Some of the new emojis like ‍ are rendered as "guy behind a MacBook" on my PC but on phones without the emoji as "guy emoji" and "computer emoji". Same for ‍️ (male version of raise hand). On phones without the Emoji, it's just "male emoji" and "female raise hand emoji". /e: oh, HN is stripping Emojis
- vanderZwan 10y ago
  This made me wonder if anyone had tried combining word2vec with emojis, and then I came across this: https://github.com/uclmr/emoji2ve https://github.com/uclmr/emoji2ve
  - peteretep 10y ago
    which is a dead link
    - satbyy 10y ago
      Correct link: https://github.com/uclmr/emoji2vec https://github.com/uclmr/emoji2vec
      - vanderZwan 10y ago
        Apologies, and thanks!

kagamine 10y ago

Not OP, but in Norwegian the correct way to write "Tom's car" is "Tom sin bil", the car of Tom. But the creep of English and laziness allows for "Toms car", esp. in informal writing.