9 ms·

Terminal support for emoji

68 points by willm 5y ago

exebook 5y ago
Suckless Terminal st just crashes to this day on some emojis so I just have xterm around for things that can display emoji like twitter client.
- knz42 5y ago
  This is a known bug in st when the default emoji font contains color emojis. Explained here: https://github.com/LukeSmithxyz/voidrice/issues/284 https://github.com/LukeSmithxyz/voidrice/issues/284 One of the simplest workarounds is to ensure that the default emoji font is black&white; the linked issue above suggests other workarounds. Also in the official st FAQ: https://git.suckless.org/st/file/FAQ.html https://git.suckless.org/st/file/FAQ.html (search for "when trying to render emoji")
  - rnestler 5y ago
    According to the link you shared it's a bug in Xft and not st: > Please don't bother reporting this bug to st, but notify the upstream Xft developers about fixing this bug.
usrbinbash 5y ago
> In summary: a massive headache, avoid if possible. Will do. If its more complicated than a box-character, it doesn't belong in my terminal :-)
- Galanwe 5y ago
  Whoever started putting Unicode everywhere instead of sticking it to word processing destroyed an entire generation.
  - _ZeD_ 5y ago
    Keep in mind that "graphics" in console is ... old, as old as ascii-art and box-drawing characters https://en.wikipedia.org/wiki/Box-drawing_character#Examples https://en.wikipedia.org/wiki/Box-drawing_character#Examples
  - Symbiote 5y ago
    Are you suggesting Lærke shouldn't be able to put her name in a filename? Or that I can't type my address, which contains ø?
    - collegeburner 5y ago
      - colejohnson66 5y ago
        > use English as the "computer language" instead. That's how we get COBOL
      - willm 5y ago
        отъебись ابتعد भाड़ में जाओ Γαμήσου Sincerely, The rest of the world.
      - usrbinbash 5y ago
        > to use English as the "computer language" This is "computer language": for i := 0; i <= x.tctl_b(); i++ { v, e := x.getg[i].pop() if e != nil { log.Fatalf("%v\nloop", e) } } So is this imul 8(%rsp), %rax So is this shopt -s globstar && grep -sn '\xFF' ** Whether we translate the meaning of the above into Mandarin, English, Spanish, Japanese, Swedish, Farsi, Swhaili, Hindi, Latin, Klingon, Dothraki, Sindarin or any other spoken, written or invented language we chose to be comfortable with, makes exactly zero difference.
        collegeburner 5y ago
        I mean use English as a universal language for users interacting with computers. It is a good way to force people to learn and use English. Like how American materiel with English documentation did so before.
faho 5y ago
>For example, iTerm2 considers the "rosette" emoji to have width 1 It's not the case here, but for some emoji there's another issue: Unicode 9 changed the width for some codepoints (mostly emoji) from 1 to 2, and iTerm until very recently (don't know if it's released yet) defaulted to the Unicode 8 widths, with an opt-in escape sequence to change to Unicode 9. >This approach comes from the wcwidth utility, and the comment at the top of the C source file provides further insight into the difficulties faced here. That's link goes to Markus Kuhn's implementation from 2007. It supports Unicode 5, and is by now woefully out of date. You don't want to use it anymore. Most terminals have their own definition, and the annoying part is that the client application and the terminal need to have theirs in sync or they get weird glitches when moving the cursor. Shameless plug: Fish's solution is widecharwidth[0], which is a python script that parses the Unicode data files and generates a wcwidth for C++, Javascript and Rust. It's still a wcwidth, meaning that it has issues with joining code points, but it's at least a start. It's up-to-date with Unicode 14 and, unless they change the data format (again) should be easy to update to future Unicode releases. It's public domain and used by at least fish and WezTerm. [0]: https://github.com/ridiculousfish/widecharwidth https://github.com/ridiculousfish/widecharwidth
teddyh 5y ago
> All terminal emulators I tested consider codepoints with an "East Asian Width" of N to have cell width of 1. This is incorrect in the case of Emoji Presentation Sequences - Unicode recommends they should be always treated as "East Asian Wide" (W). Sounds like a bug which could be fixed.
titaniumtown 5y ago
Huh, very interesting. Never even considered emojis in the terminal before. Hope these issues get fixed though (even though I personally don't use emojis in my terminal).
bouvin 5y ago
kitty seems to do the right things.
- monzool 5y ago
  kitty does indeed seem to do best among the four terminals I tested (Kitty, wezTerm, Konsole and Terminator) https://imgur.com/X0cD3jA https://imgur.com/X0cD3jA The motorboat in Kitty is quite small though?
  - bonzini 5y ago
    That's because kitty correctly prints not an emoji motorboat, but rather a text motorboat; the \U0001F6E5 defaults to text presentation and that is why east_asian_width returns 'N'. You can get the fancy one with print("\U0001F6E5\uFE0Fhello"). However, the bug that the post alludes to is that not all terminals override the east_asian_width to 'W' when they see "\uFE0F". For example on gnome-terminal the "h" overlaps with the right half of the boat.
  - Per_Bothner 5y ago
    Please try DomTerm (https://domterm.org https://domterm.org). The 2.9.4 AppImage (https://github.com/PerBothner/DomTerm/releases/tag/2.9.4 https://github.com/PerBothner/DomTerm/releases/tag/2.9.4) should have the needed support for grapheme clusters and hopefully work on reasonably up-to-date Linux systems. Of course there are more recent fixes and improvements if you don't mind building from source.
- ape4 5y ago
  echo (the family) worked in putty too
ollybee 5y ago
On my terminal I can delete family members. With the family emoji from the article of each back space removes a child, three key presses are needed to remove all 3 people in the emoji. It makes sense but very funny to see.
- graftak 5y ago
  Wonder what reversing the string will do to the family, their members will likely be split up.
tambourine_man 5y ago
macOS Terminal.app, my terminal of choice, handles the example correctly.
- garblegarble 5y ago
  Terminal.app doesn't show the following correctly for me for what it's worth (Monterey 12.2) echo '\U0001F3F5\uFE0Fhello'
- acdha 5y ago
  This is why I've stuck with Terminal.app over the years — there's basically one feature I would like (deeper color support) but otherwise I've missed out on tons of correctness or performance issues. The other features are nice but I really value a rock-solid terminal more.
  - tambourine_man 5y ago
    I'm forever tempted by iTerm, I mean the tmux integration seems so good it's almost unbelievable. But I have a thing for using built-in tools and Terminal.app gets a lot of the basics right. And has for so many years.
    - edgyquant 5y ago
      I’ve installed iterm a few times but the built in app is basically perfect. I never make the switch
  - GekkePrutser 5y ago
    It even supports really obscure features such as VT220+ double width/height characters. Indeed a solid terminal.
thriftwy 5y ago
How do terminals handle CJK characters? I remember opening something with those in konsole, and I think they distinctly occupied two columns, so I assume some width variation is supported in terminals.
jedwhite 5y ago
There's another quick and dirty option that works effectively - convert them back to text sequences, for example `:smile:` or `:)` Here's a quick guide: https://towardsdatascience.com/converting-emojis-to-text-a28b69ed9f45 https://towardsdatascience.com/converting-emojis-to-text-a28...
emddudley 5y ago
Emoji support is currently the top requested feature for Windows Terminal. https://github.com/microsoft/terminal/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc https://github.com/microsoft/terminal/issues?q=is%3Aissue+is...
- dmitriid 5y ago
  Have they figured out how to output text to the terminal yet after the great big refterm bruhaha?
assbuttbuttass 5y ago
I used to use the rxvt-unicode terminal, but switched to lxterminal for emoji support.
Torwald 5y ago
Emojis are different from chars insofar as that the former are always (not by necessity, but in practice) multi-coloured. A set of mono-coloured emojis is probably a good idea.
jrm4 5y ago
Weird take on "avoid, if possible." I've just added a default Emoji font and begun to use them for little indicators of which machine I'm SSHed into. They work fine?
SjorsVG 5y ago
Emojis in the terminal are heresy anyway. Just kidding :)
Per_Bothner 5y ago
DomTerm (https://domterm.org https://domterm.org) does a pretty good job IMNSO. See screenshot here: https://domterm.org/Features.html https://domterm.org/Features.html . DomTerm mostly delegates to the browser and the font how to compose extended Grapheme Clusters. Unfortunately, as far as I know there is no "complete" monospace font set that handles emoji. Ideally, you want a font with two character widths, with double-width for emoji, hanji (CJK characters), and similar. Instead the browser will substitute these characters from some variable-width font, and then the spacing will be off. DomTerm handles this by putting double-width characters as well as Extended Grapheme Clusters in a separate span that is forced to have the correct width. I created a JavaScript library https://github.com/PerBothner/unicode-properties https://github.com/PerBothner/unicode-properties based on other people's code but optimized for DomTerm's needs: It provides both East Asian Width (for recognizing double-width characters) and character classes (for grapheme clusters) in a single efficient trie structure. The DomTerm equivalent of tmux's "select mode" is grapheme-cluster-aware, so left/right-arrow will correctly move over an entire grapheme cluster.
l0b0 5y ago
Is there an equivalent to the Acid2 CSS browser test[1] for terminal Unicode compliance? It would be great to move forward to a place where good Unicode support is table stakes for new as well as old terminals. [1] https://en.wikipedia.org/wiki/Acid2 https://en.wikipedia.org/wiki/Acid2

68 points by willm 5y ago

Suckless Terminal st just crashes to this day on some emojis so I just have xterm around for things that can display emoji like twitter client.

knz42 5y ago
This is a known bug in st when the default emoji font contains color emojis. Explained here: https://github.com/LukeSmithxyz/voidrice/issues/284 https://github.com/LukeSmithxyz/voidrice/issues/284 One of the simplest workarounds is to ensure that the default emoji font is black&white; the linked issue above suggests other workarounds. Also in the official st FAQ: https://git.suckless.org/st/file/FAQ.html https://git.suckless.org/st/file/FAQ.html (search for "when trying to render emoji")
- rnestler 5y ago
  According to the link you shared it's a bug in Xft and not st: > Please don't bother reporting this bug to st, but notify the upstream Xft developers about fixing this bug.

usrbinbash 5y ago

> In summary: a massive headache, avoid if possible. Will do. If its more complicated than a box-character, it doesn't belong in my terminal :-)

Galanwe 5y ago
Whoever started putting Unicode everywhere instead of sticking it to word processing destroyed an entire generation.
- _ZeD_ 5y ago
  Keep in mind that "graphics" in console is ... old, as old as ascii-art and box-drawing characters https://en.wikipedia.org/wiki/Box-drawing_character#Examples https://en.wikipedia.org/wiki/Box-drawing_character#Examples
- Symbiote 5y ago
  Are you suggesting Lærke shouldn't be able to put her name in a filename? Or that I can't type my address, which contains ø?
  - collegeburner 5y ago
    - colejohnson66 5y ago
      > use English as the "computer language" instead. That's how we get COBOL
    - willm 5y ago
      отъебись ابتعد भाड़ में जाओ Γαμήσου Sincerely, The rest of the world.
    - usrbinbash 5y ago
      > to use English as the "computer language" This is "computer language": for i := 0; i <= x.tctl_b(); i++ { v, e := x.getg[i].pop() if e != nil { log.Fatalf("%v\nloop", e) } } So is this imul 8(%rsp), %rax So is this shopt -s globstar && grep -sn '\xFF' ** Whether we translate the meaning of the above into Mandarin, English, Spanish, Japanese, Swedish, Farsi, Swhaili, Hindi, Latin, Klingon, Dothraki, Sindarin or any other spoken, written or invented language we chose to be comfortable with, makes exactly zero difference.
      - collegeburner 5y ago
        I mean use English as a universal language for users interacting with computers. It is a good way to force people to learn and use English. Like how American materiel with English documentation did so before.

faho 5y ago

>For example, iTerm2 considers the "rosette" emoji to have width 1 It's not the case here, but for some emoji there's another issue: Unicode 9 changed the width for some codepoints (mostly emoji) from 1 to 2, and iTerm until very recently (don't know if it's released yet) defaulted to the Unicode 8 widths, with an opt-in escape sequence to change to Unicode 9. >This approach comes from the wcwidth utility, and the comment at the top of the C source file provides further insight into the difficulties faced here. That's link goes to Markus Kuhn's implementation from 2007. It supports Unicode 5, and is by now woefully out of date. You don't want to use it anymore. Most terminals have their own definition, and the annoying part is that the client application and the terminal need to have theirs in sync or they get weird glitches when moving the cursor. Shameless plug: Fish's solution is widecharwidth[0], which is a python script that parses the Unicode data files and generates a wcwidth for C++, Javascript and Rust. It's still a wcwidth, meaning that it has issues with joining code points, but it's at least a start. It's up-to-date with Unicode 14 and, unless they change the data format (again) should be easy to update to future Unicode releases. It's public domain and used by at least fish and WezTerm. [0]: https://github.com/ridiculousfish/widecharwidth https://github.com/ridiculousfish/widecharwidth

teddyh 5y ago

> All terminal emulators I tested consider codepoints with an "East Asian Width" of N to have cell width of 1. This is incorrect in the case of Emoji Presentation Sequences - Unicode recommends they should be always treated as "East Asian Wide" (W). Sounds like a bug which could be fixed.

titaniumtown 5y ago

Huh, very interesting. Never even considered emojis in the terminal before. Hope these issues get fixed though (even though I personally don't use emojis in my terminal).

bouvin 5y ago

kitty seems to do the right things.

monzool 5y ago
kitty does indeed seem to do best among the four terminals I tested (Kitty, wezTerm, Konsole and Terminator) https://imgur.com/X0cD3jA https://imgur.com/X0cD3jA The motorboat in Kitty is quite small though?
- bonzini 5y ago
  That's because kitty correctly prints not an emoji motorboat, but rather a text motorboat; the \U0001F6E5 defaults to text presentation and that is why east_asian_width returns 'N'. You can get the fancy one with print("\U0001F6E5\uFE0Fhello"). However, the bug that the post alludes to is that not all terminals override the east_asian_width to 'W' when they see "\uFE0F". For example on gnome-terminal the "h" overlaps with the right half of the boat.
- Per_Bothner 5y ago
  Please try DomTerm (https://domterm.org https://domterm.org). The 2.9.4 AppImage (https://github.com/PerBothner/DomTerm/releases/tag/2.9.4 https://github.com/PerBothner/DomTerm/releases/tag/2.9.4) should have the needed support for grapheme clusters and hopefully work on reasonably up-to-date Linux systems. Of course there are more recent fixes and improvements if you don't mind building from source.
ape4 5y ago
echo (the family) worked in putty too

ollybee 5y ago

On my terminal I can delete family members. With the family emoji from the article of each back space removes a child, three key presses are needed to remove all 3 people in the emoji. It makes sense but very funny to see.

graftak 5y ago
Wonder what reversing the string will do to the family, their members will likely be split up.

tambourine_man 5y ago

macOS Terminal.app, my terminal of choice, handles the example correctly.

garblegarble 5y ago
Terminal.app doesn't show the following correctly for me for what it's worth (Monterey 12.2) echo '\U0001F3F5\uFE0Fhello'
acdha 5y ago
This is why I've stuck with Terminal.app over the years — there's basically one feature I would like (deeper color support) but otherwise I've missed out on tons of correctness or performance issues. The other features are nice but I really value a rock-solid terminal more.
- tambourine_man 5y ago
  I'm forever tempted by iTerm, I mean the tmux integration seems so good it's almost unbelievable. But I have a thing for using built-in tools and Terminal.app gets a lot of the basics right. And has for so many years.
  - edgyquant 5y ago
    I’ve installed iterm a few times but the built in app is basically perfect. I never make the switch
- GekkePrutser 5y ago
  It even supports really obscure features such as VT220+ double width/height characters. Indeed a solid terminal.

thriftwy 5y ago

How do terminals handle CJK characters? I remember opening something with those in konsole, and I think they distinctly occupied two columns, so I assume some width variation is supported in terminals.

jedwhite 5y ago

There's another quick and dirty option that works effectively - convert them back to text sequences, for example `:smile:` or `:)` Here's a quick guide: https://towardsdatascience.com/converting-emojis-to-text-a28b69ed9f45 https://towardsdatascience.com/converting-emojis-to-text-a28...

emddudley 5y ago

Emoji support is currently the top requested feature for Windows Terminal. https://github.com/microsoft/terminal/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc https://github.com/microsoft/terminal/issues?q=is%3Aissue+is...

dmitriid 5y ago
Have they figured out how to output text to the terminal yet after the great big refterm bruhaha?

assbuttbuttass 5y ago

I used to use the rxvt-unicode terminal, but switched to lxterminal for emoji support.

Torwald 5y ago

Emojis are different from chars insofar as that the former are always (not by necessity, but in practice) multi-coloured. A set of mono-coloured emojis is probably a good idea.

jrm4 5y ago

Weird take on "avoid, if possible." I've just added a default Emoji font and begun to use them for little indicators of which machine I'm SSHed into. They work fine?

SjorsVG 5y ago

Emojis in the terminal are heresy anyway. Just kidding :)

Per_Bothner 5y ago

DomTerm (https://domterm.org https://domterm.org) does a pretty good job IMNSO. See screenshot here: https://domterm.org/Features.html https://domterm.org/Features.html . DomTerm mostly delegates to the browser and the font how to compose extended Grapheme Clusters. Unfortunately, as far as I know there is no "complete" monospace font set that handles emoji. Ideally, you want a font with two character widths, with double-width for emoji, hanji (CJK characters), and similar. Instead the browser will substitute these characters from some variable-width font, and then the spacing will be off. DomTerm handles this by putting double-width characters as well as Extended Grapheme Clusters in a separate span that is forced to have the correct width. I created a JavaScript library https://github.com/PerBothner/unicode-properties https://github.com/PerBothner/unicode-properties based on other people's code but optimized for DomTerm's needs: It provides both East Asian Width (for recognizing double-width characters) and character classes (for grapheme clusters) in a single efficient trie structure. The DomTerm equivalent of tmux's "select mode" is grapheme-cluster-aware, so left/right-arrow will correctly move over an entire grapheme cluster.

l0b0 5y ago

Is there an equivalent to the Acid2 CSS browser test[1] for terminal Unicode compliance? It would be great to move forward to a place where good Unicode support is table stakes for new as well as old terminals. [1] https://en.wikipedia.org/wiki/Acid2 https://en.wikipedia.org/wiki/Acid2