9 ms·

Terminal Support for Emoji

80 points by willm 3y ago

pavlov 3y ago
It still pains me that Unicode decided to get in the business of curating an ever-growing clip art collection. It’s like the Oxford English Dictionary decided that they’re actually poets; their main job is suddenly to invent brand new words that let people write with an exciting level of density and poetic license; and those new dictionary words would also be multi-color because everybody owns a pack of colored pencils, right.
- avgcorrection 3y ago
  Unicode needed to include written-on-paper glyphs that existed in the world. That makes sense to most people. But a lot, lot, lot of communication these days happen digitally (digital first). You can't scribble things as freely in a textbox like this one. You can certainly make suggestive combinations like `:)`, but you are pretty limited. (How do you scribble "family of three: mom, father, daughter" without any emojis?) How do you propose that Unicode include glyph innovations that are digital-first?
  - JellyYelly 3y ago
    > How do you scribble "family of three: mom, father, daughter" without any emojis? How often do people need to do that; with that level of fidelity, right down to the composition of the family? My suspicion is not often. A short sentence does this description just as good and is much less ambiguous, as in written, concise communication that is "emoji-rich" is filled with so many details (such as the composition of the family and their skintone) it's often not clear which details are important for the message and which are not. In addition because emojis are rendered differently on different devices the meaning may be lost (e.g. when apple changed the gun emoji to a watergun, many messages took on different meanings depending on if a revolver was used or a watergun)
    - chii 3y ago
      > A short sentence does this description just as good and is much less ambiguous so you're now dictating what can be said, rather than designing a system for which _anything_ that could be said is possible? You're missing the goal of a system of writing and representation.
      - 0xBA5ED 3y ago
        The system exists. It's text.
        chii 3y ago
        how would you write something that's not ascii character based, such as hieroglyphs?
        0xBA5ED 3y ago
        A set of hieroglyphs for a formally designed and recognized language that is in use to communicate is one thing. Endless bits of arbitrary clip art is another.
    - TeMPOraL 3y ago
      > How often do people need to do that; Surprisingly, this happens. I recently started to use a lot of emojis, including ones like these, to name calendar events - because I use a watch face on my smartwatch that renders the next 12 hours worth of events on the clock face, and given the small space, many of the events can fit three or four letters of description. Emojis work as great workaround, because I can encode things like "takeoff, gate A11" in 4 visual characters, or "doctor's visit" in one. (I prefix the event titles with emojis rather than replace the longer form completely, because some of those events are shared, and sometimes I forget what an emoji stands for anyway...) Now, this is perhaps an unique use case, but I found myself doing this in other scenarios too, like task planners - the common thread is, "not enough space to fit full label".
  - pavlov 3y ago
    Are they glyphs or just vector graphics clip art? Unicode doesn’t include ligature code points for every English word either. Somehow we manage to keep glyphs and words separate. Why couldn’t we do the same for inline graphics? The copyright situation around emoji is actually worse now than if they’d been kept out of the standard. People expect emojis to look like on iPhone, but those specific graphics are owned by Apple, leaving everybody else scrambling for emojis that look close enough without infringing. If emojis had originally been released as an open source library of SVG graphics together with some kind of standard shorthand way to refer to them without embedding an entire inline URL, we could have truly open clip art instead of this weird semi-proprietary mess.
    - amelius 3y ago
      In my ideal world, people would have a standard library of SVG emoji on their phones, they can send them around, but they can also save the emojis others use to their own library. Of course, artists can add new SVG emoji and send them around. This way, we get an evolving library of emoji, just like how languages evolve. Unicode is stiffling this evolution.
      - biztos 3y ago
        Interestingly (but commercially) the very popular Line app, which IMO has the best "stickers," lets you do something like this. https://en.wikipedia.org/wiki/Line_(software) https://en.wikipedia.org/wiki/Line_(software) If I buy a set of stickers (little animations, much more expressive than emojis) I can send them to anyone in chats, or I can send the set as a gift to that user; in the latter case, the user can add them to their library. Your library is the standard stickers, any free stickers you've added, anything you've bought, and anything you've been gifted. The end result is that some people have a fairly distinct sticker-communication style because they have in their library relatively obscure ones, or just like to use the less-popular free ones. In theory anyone can make their own stickers but it's kind of a pain, and I haven't gotten around to it: https://creator.line.me/en/stickermaker/ https://creator.line.me/en/stickermaker/ So in my ideal world, we'd not have any emojis that can't be reduced to expressive ASCII, but we'd have a massive public library of freely available SVG Stickers with accessibility features, and all the chat apps would use them.
  - zokier 3y ago
    > How do you propose that Unicode include glyph innovations that are digital-first? Why does Unicode specifically need to include novel "innovations"? It's arguably failure of the community that we have not been able to standardize interoperable higher level rich text formats, so now everything and kitchen sink needs to be bolted on Unicode instead in the name of interop.
    - Sesse__ 3y ago
      > Why does Unicode specifically need to include novel "innovations"? Unicode chose to get into emoji because Japanese carriers were already making their own private character sets with emoji, and Unicode has a goal of being a superset of all other (relevant) character sets. (Arguably emoji support in Unicode has been a resounding success. Look at the world-wide enthusiasm around each new Unicode version that gets announced now!)
      - ygra 3y ago
        It also brought broken handling of code points beyond the BMP to the forefront and dealing with Unicode text is correct in much more places by now. Previously the only people who noticed were those who needed obscure Han ideagraphs, hieroglyphs, and other things that are not terribly useful or interesting to the vast majority of people.
  - ilyt 3y ago
    > (How do you scribble "family of three: mom, father, daughter" without any emojis?) Why you need to? We dealt without hieroglyphs just fine in communication.
    - Closi 3y ago
      The tahitans can deal with communication just fine with only 13 letters, so I guess we could probably trim the letters b, c, d and g out of unicode too. With enough optimisation we might just be able to communicate with just a boolean-based language. (i.e. why limit written communication rather than make it richer)
      - zamadatix 3y ago
        That you can take something too far in one direction doesn't give reason or explanation for anything except "don't take it too far in that direction". There are probably good explanations why not to take things infinitely far in either direction but they don't really help explain where the line should be drawn. I think the actual argument here is encoding of seemingly arbitrary colored pictorial combinations overcomplicates character encoding. If you want to display a colored drawing of an arbitrary family SVG is already a thing but if you want to textually encode an arbitrary family you should use characters in your language not expect the text encoding to pick up more arbitrary drawings. To me, using Emoji was probably a great way to force developer's hands on supporting certain encoding features more complicated languages use even in cases they only wanted to support latin text. That said, this job is already done. We don't need to continue putting everything into ever more complex pictorial encodings via Unicode for the rest of time.
        Closi 3y ago
        That’s true, but if your argument can be extended to absurdity it probably can't be used to justify where the 'correct' place on the scale is to land.
        avgcorrection 3y ago
        > That you can take something too far in one direction doesn't give reason or explanation for anything except "don't take it too far in that direction". Why or how does the Tahitian language take things too far?
  - phito 3y ago
    > How do you scribble "family of three: mom, father, daughter" without any emojis You don't. You use text.
    - ungamedplayer 3y ago
      I still was trying to figure out how an emoji represents a family.
      - aendruk 3y ago
        With the scheme demonstrated at the beginning of the article. 1F468 man 200D zero width joiner 1F469 woman 200D zero width joiner 1F467 girl
        worthless-trash 3y ago
        Thats just three people close together, could be a Torball team.
      - yomlica8 3y ago
        You use the pregnant man emoji, which represents mother, father and child together as one.
  - tambourine_man 3y ago
    You write them down. With letters and words. See, we invented this thing called the alphabet. You memorize some 20 symbols and combine them to express every possible emotion, instead of memorizing a gazillion symbols. It’s no wonder hieroglyphs have fallen out of favour in the last 5k years.
    - avgcorrection 3y ago
      Thank you for your contribution. :)
  - Cthulhu_ 3y ago
    I don't speak good emoji though; I understand the smiley face, but what does posting the family emojis mean? What is it used for? Nobody I know or have seen online replaces random words with emoji for brevity. That's annoying. At worst I've seen ironic posts or trying to be hip outlets adding the emoji for a word after the word itself, or use them as bullet point markers. It reminds me of a thing where it was like, "elephants are so smart because they can use this trumpet to communicate danger", and someone pointed out that humans can do that too by using words. Mind blown.
  - perryizgr8 3y ago
    > How do you scribble "family of three: mom, father, daughter" without any emojis? Using nifty little things called jpegs.
    - Timon3 3y ago
      How do you propose they get transmitted? In-band in the middle of text (good luck reading that with an unsupported editor), or through servers, necessitating NAT and all that good fun? Since the images can be arbitrary you also open yourself up to bugs in the parser. All around a much, much worse idea compared to text emojis.
      - perryizgr8 3y ago
        > good luck reading that with an unsupported editor You need a "supported editor" to display Unicode emojis too, as TFA shows. And supposedly it is quite hard. OTOH displaying jpegs is a solved problem.
        Timon3 3y ago
        > You need a "supported editor" to display Unicode emojis too, as TFA shows. Let's give this a try. This is roughly what a sentence with an emoji looks like in an unsupported editor: hello [] world (with [] being the well-known "placeholder" square). Now let's try the same with an emoji, I went with 16x16 to make this kind of bearable, but realistically you'd want at least 64x64: hello data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEASABIAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQH/2wBDAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQH/wAARCAAQABADAREAAhEBAxEB/8QAFwAAAwEAAAAAAAAAAAAAAAAAAAIICv/EABwQAAIDAQEBAQAAAAAAAAAAAAQFAgMGBwEIFP/EABoBAAEFAQAAAAAAAAAAAAAAAAACAwUGCAn/xAAhEQACAwEBAAICAwAAAAAAAAADBAECBQYHCBITFAAVFv/aAAwDAQACEQMRAD8A1xbv6+xizYBjaeOtvSMunT5XmVeTDaMAP1waXJyNft7FFlPoub9OHjK5gylJQmBICnfTMgi+6PJ31n5YubHpnY4S5H1eV89d3MumSj0TmCPdLhNmR0dDQNmtpM65HTrHjKxrGlSiY4JdczZL3ppUPljuLz2OdUaRNLYxkt119xdcp1qvKjcXzcyGqFquRYJq1YKGsNMNQSsFqCg6SYX6+xjPYGDZivWjpVnTYcr06vWBtFy/0qbSlPRsMTNtbb6VnPDSJSoYrfYqHIQxsqKIkUUX+nlHyybyPS+OwjkeZ5b0F3DzL5LvQubwsMu44FFB/PNotus45UjtB/tMehoUIpeb1XC0OhJDeWu7XPa5mqJD0cjGd3En011xHZ/RVK4fN04VGKpyMADaoSHiWlmfxRJLAuQf8hP6/wCB/SXGOkaDZ8hzrbo/P9m4cPaglrLqYUMzY1bMHRKs2rm7sSauQJTQgcNqyXkL2CyC+uZwRYNw5E5634bPnnYdjtF8WzPYuM7vo9DrQPL4mrodHyWlpkufUyT2wzhfpi2Nf9hMn1MvWZvFq0Na/wBrpyHYZHoGDgotdn/juj5vJTwzpuuZS+duLJCoBTRWtrhuuV2wxxRoNCCNF5mYggvrML8fcA+kuydHRbPryFxz3BZFumdXAs2PUzB9D4qbrnlCoCrpLwyxrewKVjCmN1q8ZetVTY0wNNKYVD0NeTeF39G7Pjdsfi+Z45xXBdCj1bDh8TUz+i67TziiPl5Qp3DnfJjjMKTtkn8K8zMfWtzVH9V9h2GN59z+8ir2Udl0nSZTeKuok7lsZ+Eu2Ei7WgxbICNcLlRkmq4bXIebxWZgYvvaf//Z world Now, which one is more readable?
  - GuB-42 3y ago
    > Unicode needed to include written-on-paper glyphs that existed in the world. If it was the case, Emoji is seriously lacking a penis. I mean, seriously, give men a way to draw and this is what you will get. There are already existing "subjective combinations" like 8===D and the eggplant emoji for which it is its most common use. There is a proper penis in the hieroglyphics, but just because there is a hieroglyphic doesn't mean there is no corresponding emoji (ex: eye). On the other hand, many emoji pass even though I have never seen anything remotely similar being scribbled ever, it is the addition to the emoji block that drove its use. Plus, because inclusivity, all its equivalents in different genders, skin tones, cultures, etc... Clearly, Unicode acts as an arbiter here. It decides on what it thinks it is "good" rather than what is really in use. And it is not just about penises, the hangman is also absent, as is the "gun pointed on head" sign (suggesting suicide) that is commonly used in real life. Worth noting that the first emoji were included because they were already in common use before Unicode, in Japanese phones specifically. Unicode just included them so that Japanese users could switch to Unicode without loss of functionality.
    - cnity 3y ago
      Unicode actually does include a penis: 𓂸 There is even a "phallus with emission" variant.
      - noAnswer 3y ago
        Interesting: U+130BA EGYPTIAN HIEROGLYPH D053 (phallus with liquid issuing from it) 𓂺 Not even BabelMap or BabelPad have a font showing more than a square.
        GuB-42 3y ago
        I just discovered that the default Windows 10 font censors phalluses except when they are next to another hieroglyph. 𓂲𓂺
  - khiqxj 3y ago
    [dead]
  - terracottalite 3y ago
    I think the emoji combining isn't that useful, at least in the family example. It is enough putting them side by side without merging them. That way you have even more possible combinations, like a family of an ogre and a human and a kid. Sorry if this doesn't make sense.
- yorwba 3y ago
  Unicode is only curating the clip-art collection, inventing brand-new entries is left to interested third parties. So a closer analogy would be the Oxford English Dictionary adding newly-coined words, which—of course—they do https://www.oed.com/discover/the-oed-september-2021-update/ https://www.oed.com/discover/the-oed-september-2021-update/ in addition to expanding coverage of older words (analogous to the Unicode Consortium working on historical scripts). NB if you don't like multi-color emoji, you can also use a monochrome font for them.
  - pavlov 3y ago
    It’s not the same thing because Unicode is the only gatekeeper for emoji adoption. You can’t use an emoji that isn’t yet in the standard and deployed by the OS vendors. Whereas OED only adds words that have substantial real world usage. Re: monochrome emoji rendering — an impossible proposition if you need to render any user-generated text. People simply can’t understand that their emoji might look different than when they picked it on their iPhone keyboard. The supposed rendering latitude on emojis is completely imaginary; in practice you need to get an emoji set that’s as close as possible to Apple’s without infringing on their design copyrights.
    - yorwba 3y ago
      > You can’t use an emoji that isn’t yet in the standard and deployed by the OS vendors. You can, as long as you control the font. Pick any codepoint in the Private Use Area and have your font define a picture for it. That's the whole idea behind icon fonts. > People simply can’t understand that their emoji might look different than when they picked it on their iPhone keyboard. Android users who pick an emoji on their Google keyboard and then have something different show up in the message they sent seem to be able to cope somehow. But if you can't get away from imitating the Apple look because of wrong users, you could still try converting it to monochrome. Maybe users will forgive the deviation if it makes sense in context, e.g. for a terminal emulator.
      - ElectricalUnion 3y ago
        The terminal emulator is one of the places where I actually expect coloured emoji support. Granted, that it's supposed to emulate a teletypewriter but I think that ship sailed a long time ago.
- Sesse__ 3y ago
  The Unicode Consortium's stated direction is that they want protocols to support stickers (arbitrary images) instead of emoji. But protocols don't seem to be doing that, so…
  - pavlov 3y ago
    If you’ve dug yourself into a hole and nobody is throwing you the rope, at least stop digging.
- amelius 3y ago
  My main problem with emoji is that they look different everywhere. For example "big grin" looks like that on native Android, but looks like an angry face on other platforms. Quite a difference in meaning. Another problem is that some emojis are horribly missing. For example during COVID it would have been nice to have a cotton swab thing. :face-palm:
  - Cthulhu_ 3y ago
    Yeah but... why? So social media outlets can make coming in for a test c00l and hip and down with the kids? Does a swab emoji improve the message or is it just colorful seasoning?
  - whywhywhywhy 3y ago
    > For example during COVID it would have been nice to have a cotton swab thing Why? It would be next to meaningless now but have to be supported forever. This actually exemplifies why they need to be more strict about what goes in.
    - amelius 3y ago
      No, it exemplifies that we don't need a central authority like Unicode to control what we can/cannot say in emojis.
      - Joker_vD 3y ago
        Funny how alphabetic text allows you to write whatever by just combining ~30 symbols in different order and quantities but for hieroglyphics you need thousands of symbols and it still is not enough, and never will be.
        amelius 3y ago
        A picture can say more than a thousand words ;)
- deleted 3y ago
  [deleted]
- Macha 3y ago
  Honestly I'm happy enough that emoji are there as a carrot to upgrade their platform support for newer unicode revisions which the US otherwise might not care about. It's only been 10 years since you couldn't reliably use the € symbol in web forms for example. Never mind things even further afield from the US
- pfp 3y ago
  Agreed. The whole emoji phenomenon is a kind of infantilizing cultural rot -- it makes serious, static documentation and tooling resemble a children's book and hinders live communication by encouraging vague single-pictogram messages and the expression of raw emotions (genuine or not) instead of mature and balanced thoughts.
  - giraffe_lady 3y ago
    None of these things are mutually exclusive the way you're implying. This is a "cultural degeneracy" argument, which are always suspect imo. You are certainly entitled to dislike the aesthetics of shifts in communication, but you're basically just assuming that the changes are inherently negative and there's no reason to think they are. "Single-pictogram messages and expression of raw emotion" are simply not mutually exclusive with mature, well-considered, intentional communication.
- khiqxj 3y ago
  [dead]
- deleted 3y ago
  [deleted]
panki27 3y ago
Interestingly, Microsoft Terminal renders all of these correctly!
- frizlab 3y ago
  With the proper width and all?
  - dr_zoidberg 3y ago
    I ran a few tests in Windows Terminal. The bomb emoji got width 2, while the motorboat got width 1 and correct aspect ratio, though I didn't quite get to see it properly until I zoomed in like 5x. The family was rendered all as one emoji, cells wide, but left 4 blank cells before it. So it was a bit better than the authors tests, but there's still room for improvement.
- xioxox 3y ago
  So does KDE's Konsole for me.
- foepys 3y ago
  Microsoft Terminal is actually pretty good. I was surprised when I configured git bash as my main shell there and when opening it via right-click in Explorer, it correctly translated the path and opened the shell in the correct folder.
  - chii 3y ago
    Is this the same terminal that Casey Muratori was complaining about how there's basic performance issues? (see https://news.ycombinator.com/item?id=28743687 https://news.ycombinator.com/item?id=28743687)
  - jlokier 3y ago
    I didn't like Windows Terminal when I used it over the last few months at the last $job because my choices of software were limited and it was better than the Windows Console. There seems to be no way to make the cursor clear and visible on all background colours that I used for CLI and text editing. I ended up having to use a difficult-to-see 50% grey cursor, as a compromise that was better than losing the cursor completely sometimes. It wouldn't send Control-Space or Control-@ from the keyboard (they are the same character, ^@ aka NUL), which is the key used to set the editing mark in Emacs so used a lot. No key combination is mapped to that character in Windows Terminal, and I couldn't find a way to customise it to do so either. Historical real terminals like the VT100, and of course xterm etc, always did so, which is why it's an essential key in some terminal applications. I compromised by binding Control-] to set-mark-command in Emacs but this was annoying when switching between devices. Finally, line drawing characters, boxes, progress bar characters and such had ugly gaps, as though line height was incorrect for them. All of these seem to work fine in every other terminal emulator I've used.
draw_down 3y ago
[dead]
LorenDB 3y ago
I never have understood the push to emojify everything. But commit messages, options in menus, status outputs, you name it: I think we'd be better off leaving it in plain ASCII (or Unicode if your language needs that, just as long as you don't dip into the emoji). Emoji are quite nice for messaging people and occasionally some of the more generic ones can be useful for status indicators (think the red and green circles), but I don't want to be presented with a little green worm every time somebody submits a bug fix on GitHub.
- avgcorrection 3y ago
  I think emojis in commit messages are very overrated. But emojis are great in informal communication when you want to compensate for the fact that your words are completely divorced from inflection, tone, and body language.
  - petepete 3y ago
    I once had a colleague who started every commit message with an emoji. It became tiring very quickly. Still, the odd facepalm, bug or bomb once in a while doesn't really do any harm. I use the robot emoji in my ~/.mailmap so I can easily identify bot commits (dependabot, snykbot etc) when scanning through commit logs locally.
    - LorenDB 3y ago
      Absolutely, emoji used spontaneously are great. When they are used as qualifiers or attributes constantly, it becomes just as bad as typing "feat:" or "chore:" or something similar in front of every commit message.
- Joker_vD 3y ago
  > or Unicode if your language needs that Благодаря ти много, добър непознат.
- the_gipsy 3y ago
  There are many languages that have multi-width symbols, it's not only emojis.
38 3y ago
works fine with Windows Terminal: http://0x0.st/H_S1.png http://0x0.st/H_S1.png but yes to be fair, the width on those ridiculous 5 codepoint emoji is off.
- maleldil 3y ago
  Kitty on macOS also works fine with all cases, including the correct width.
  - Sesse__ 3y ago
    Kitty has the wrong width: https://github.com/kovidgoyal/kitty/issues/3810 https://github.com/kovidgoyal/kitty/issues/3810 It doesn't immediately overwrite the emoji with the next character, but the cursor position is off, which leads to wrapping problems down the line.
- bazzargh 3y ago
  Is it windows or is it golang making the difference there? Golang's implementation of wcwidth is different, and a commonly used lib has an 'emoji' table which covers that bomb (1F4A3) https://github.com/mattn/go-runewidth/blob/master/runewidth_table.go#L433 https://github.com/mattn/go-runewidth/blob/master/runewidth_... The family emoji looks to have the wrong width, but it does resemble the behaviour supposed to be fixed by https://github.com/mattn/go-runewidth/pull/63 https://github.com/mattn/go-runewidth/pull/63 (see https://github.com/mattn/go-runewidth/issues/59 https://github.com/mattn/go-runewidth/issues/59 - like the flag example, that's multiple emojis with zwj right?)
  - 38 3y ago
    I just used the standard library, not even any imports: https://godocs.io/builtin#println https://godocs.io/builtin#println
frizlab 3y ago
Haaa yes the eternal emoji width problem when using monospaced fonts… I added an explicit workaround for this in my logger lib, with an environment variable to be able to change the behavior. https://github.com/xcode-actions/clt-logger/blob/12a5ebc1b00ec0d3588fe168cef29bad96e0c7f6/Sources/CLTLogger.swift#L135 https://github.com/xcode-actions/clt-logger/blob/12a5ebc1b00...
jansan 3y ago
IMO we should have left emojis out of everything that is rendered by convention with monospaced fonts. But of course I respect the desire for displaying rainbow farting unicorns even in consoles, so let's find a solution for this.
Joker_vD 3y ago
Honestly, seems like the problem with wcwidth's implementations. Most of them have been reporting almost all emojis as having width 1 (and some very non-invisible symbols as having with 0) for years, to this very day, for some unexplained reason despite regular web rants of how this is wrong. Check out this one: [0]. It's 8 years old! The problems are still there. [0] https://eev.ee/blog/2015/09/12/dark-corners-of-unicode/#combining-characters-and-character-width https://eev.ee/blog/2015/09/12/dark-corners-of-unicode/#comb...
- appplication 3y ago
  I cannot even begin to fathom how you would of about fixing this more broadly. I’m sure there are many who have come to count on the current behavior for some critical use case, and would vehemently oppose any attempts to change of correct. Hyrum’s law, etc.
Sesse__ 3y ago
I don't know of a single terminal that actually gets these ZWJ emoji sequences right. There are several terminals who will render them, but I haven't seen any actually get the width calculations correct. (E.g., https://github.com/kovidgoyal/kitty/issues/3810 https://github.com/kovidgoyal/kitty/issues/3810.) Part of the problem is that it really messes up the idea of a rectangular grid of base characters in memory, so you'll need some sort of indirection. Obviously I haven't tested all terminal emulators in existence, though :-) I patched a similar bug in screen earlier this year (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1039503 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1039503), which indicates the kind of stuff you have to go through.
- hnlmorg 3y ago
  I found wide character support to be surprisingly good across the terminal emulators I tried on macOS and Linux. This testing was done earlier this year when I was building support for Unicode variable names into my $SHELL (more to support foreign languages than glyphs but the end result is the same) It’s worth noting that wider character support needs to be implemented in both the terminal emulator AND and the console applications that run on the terminal emulator too.
  - Sesse__ 3y ago
    So which ones support the ZWJ sequences correctly and fully?
    - hnlmorg 3y ago
      I cannot claim to have tested every ZWJ character on every terminal emulator but if you find any ZWJ related bugs in murex shell https://GitHub.com/lmorg/murex https://GitHub.com/lmorg/murex then it will be treated as a serious bug because Murex does have international users so aims for greater compatibility with other language writing systems (albeit the documentation is presently only available in English).
- 3np 3y ago
  Have you checked mlterm? Its supported for other otherwise-neglected encodings and fonts is otherwise unparalleled AFAIK. https://wiki.archlinux.org/title/Mlterm https://wiki.archlinux.org/title/Mlterm
RedNifre 3y ago
Are there some well defined Unicode subsets, e.g. one without Emoji? (I'm only vaguely aware of XML specifying "Unicode without control characters" in the beginning).
- Joker_vD 3y ago
  Yes. ASCII is one. Latin-1 is another.
  - ElectricalUnion 3y ago
    Depending on what degenerate system you are in, U+00A9 © COPYRIGHT SIGN might render like a emoji, so ISO/IEC 8859-1 isn't safe.
- avgcorrection 3y ago
  Basic Multilingual Plane should be it.
  - deathanatos 3y ago
    While most emoji are outside the BMP, not all are. E.g., {U+2622, U+FE0F} (radiation symbol + variation selector) or even just U+2728 (sparkles). See https://unicode.org/reports/tr51/#Presentation_Style https://unicode.org/reports/tr51/#Presentation_Style
  - ElectricalUnion 3y ago
    Unicode Character 'HOURGLASS' ⌛ (U+231B) says hello.
    - avgcorrection 3y ago
      Hello.
joshka 3y ago
Some deep context on this is at: https://gitlab.freedesktop.org/terminal-wg/specifications/-/issues/9 https://gitlab.freedesktop.org/terminal-wg/specifications/-/...
mcluck 3y ago
I'm dying to know what the point of N is if it should be treated the same as W
- wodenokoto 3y ago
  I think it is because those are unicode points that can be displayed as either emoji or text. (e.g., https://www.compart.com/en/unicode/U+262F https://www.compart.com/en/unicode/U+262F). so when displayed as text they are N
tambourine_man 3y ago
I don’t want emoji in my terminal. They are already everywhere. I chose my color scheme carefully for some candy color to pop up. It is useful to be able to see them on database dumps, to be able to clean them with sed, etc. But it should be a toggle, IMO.
rivo 3y ago
I spent quite a bit of time coming up with an algorithm that gives me the correct width for any Unicode character, including emojis [1]. The reality is much more complicated than what wcwidth does and using the East Asian Width attribute will give wrong results, as outlined in the article. Unfortunately, since a lot of terminals simply use wcwidth, you often have to use that same flawed algorithm in your application to make it work in those terminals. [1] https://pkg.go.dev/github.com/rivo/uniseg#hdr-Monospace_Width https://pkg.go.dev/github.com/rivo/uniseg#hdr-Monospace_Widt...
Tyriar 3y ago
I'm on the VS Code team and maintain xterm.js which is what Hyper's frontend is based on. There are actually multiple developments happening in this area. First, there's a contribution from the author of DomTerm which adds grapheme cluster support to xterm.js, which will correctly merge and size things like emoji that are called out in the post. This is currently based on Unicode 15. See https://github.com/xtermjs/xterm.js/pull/4519 https://github.com/xtermjs/xterm.js/pull/4519 Second, while Windows Terminal does seem to work with emoji sometimes, it doesn't all the time. I'm not 100% sure, but I think it may only work on Windows ptys, not in WSL for example. Last time I spoke with the team they said they're working on a rewrite which could lead to proper emoji support.
- rivo 3y ago
  I spent a lot of time on this topic (see https://news.ycombinator.com/item?id=37048434 https://news.ycombinator.com/item?id=37048434). Feel free to get in touch and discuss. I'm happy to help.
  - Per_Bothner 3y ago
    I'm the author of DomTerm and the above-mentioned xterm.js PR. Both use the full UnicodeGrapheme Cluster Boundaries algorithm (https://unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries https://unicode.org/reports/tr29/#Grapheme_Cluster_Boundarie...). However, I haven't seen any specifications for how wide the resulting clusters should be in a mono-space context. So unless we enhance terminals to handle variable-width fonts (which I've been thinking about), we need to took at other terminals and make judgement calls. Generally, the width of a cluster is the width of the widest codepoint. Also, I decided that a "normal" character followed by emoji-presentation makes it 2 columns wide. In your linked-to article, you suggest 2-em dash and 3-em dash should be 3 and 4 columns respectively. That might be reasonable, but it is explicitly contrary to the EastAsianWidths specification. You also suggest that Pictographics fullowed by text-presentation should have width 1. That seems reasonable, though I don't implement that.
    - rivo 3y ago
zanellato19 3y ago
Wezterm rendered all of these correctly :)
ChrisArchitect 3y ago
(2021)
- ChrisArchitect 3y ago
  Some previous discussion: https://news.ycombinator.com/item?id=30113521 https://news.ycombinator.com/item?id=30113521
bitsandboots 3y ago
While this is a real issue, that effects terminals in real-world use, I took issue with this quote: > This issue affects every terminal I've tested: Visual Studio Code, iTerm2, Alacritty, and Hyper. I don't think those terminals are particularly known for being feature-complete with regards to VT/ANSI codes or new developments such as emoji. The quantity of things a VT-ish terminal must do is massive and not standardized, so many terminals just cover the common cases and nobody notices that they're incomplete. If the author noted that emojis don't work properly in xterm (not xterm.js) or konsole, then that'd be something to write home about. For reference, I've written a terminal emulator fairly recently, and found xterm to be most faithful and also a great source of documentation in itself. In personal use though, konsole is pleasant and seems to do everything I've seen right. On the other hand, currently writing this from mac and iterm2 is kind of terrible, send help.
- bestouff 3y ago
  GNOME terminal works well for me. And from experience I'm not surprised the terminals cited in the blog post don't work.

80 points by willm 3y ago

It still pains me that Unicode decided to get in the business of curating an ever-growing clip art collection. It’s like the Oxford English Dictionary decided that they’re actually poets; their main job is suddenly to invent brand new words that let people write with an exciting level of density and poetic license; and those new dictionary words would also be multi-color because everybody owns a pack of colored pencils, right.

avgcorrection 3y ago
Unicode needed to include written-on-paper glyphs that existed in the world. That makes sense to most people. But a lot, lot, lot of communication these days happen digitally (digital first). You can't scribble things as freely in a textbox like this one. You can certainly make suggestive combinations like `:)`, but you are pretty limited. (How do you scribble "family of three: mom, father, daughter" without any emojis?) How do you propose that Unicode include glyph innovations that are digital-first?
- JellyYelly 3y ago
  > How do you scribble "family of three: mom, father, daughter" without any emojis? How often do people need to do that; with that level of fidelity, right down to the composition of the family? My suspicion is not often. A short sentence does this description just as good and is much less ambiguous, as in written, concise communication that is "emoji-rich" is filled with so many details (such as the composition of the family and their skintone) it's often not clear which details are important for the message and which are not. In addition because emojis are rendered differently on different devices the meaning may be lost (e.g. when apple changed the gun emoji to a watergun, many messages took on different meanings depending on if a revolver was used or a watergun)
  - chii 3y ago
    > A short sentence does this description just as good and is much less ambiguous so you're now dictating what can be said, rather than designing a system for which _anything_ that could be said is possible? You're missing the goal of a system of writing and representation.
    - 0xBA5ED 3y ago
      The system exists. It's text.
      - chii 3y ago
        how would you write something that's not ascii character based, such as hieroglyphs?
        0xBA5ED 3y ago
        A set of hieroglyphs for a formally designed and recognized language that is in use to communicate is one thing. Endless bits of arbitrary clip art is another.
  - TeMPOraL 3y ago
    > How often do people need to do that; Surprisingly, this happens. I recently started to use a lot of emojis, including ones like these, to name calendar events - because I use a watch face on my smartwatch that renders the next 12 hours worth of events on the clock face, and given the small space, many of the events can fit three or four letters of description. Emojis work as great workaround, because I can encode things like "takeoff, gate A11" in 4 visual characters, or "doctor's visit" in one. (I prefix the event titles with emojis rather than replace the longer form completely, because some of those events are shared, and sometimes I forget what an emoji stands for anyway...) Now, this is perhaps an unique use case, but I found myself doing this in other scenarios too, like task planners - the common thread is, "not enough space to fit full label".
- pavlov 3y ago
  Are they glyphs or just vector graphics clip art? Unicode doesn’t include ligature code points for every English word either. Somehow we manage to keep glyphs and words separate. Why couldn’t we do the same for inline graphics? The copyright situation around emoji is actually worse now than if they’d been kept out of the standard. People expect emojis to look like on iPhone, but those specific graphics are owned by Apple, leaving everybody else scrambling for emojis that look close enough without infringing. If emojis had originally been released as an open source library of SVG graphics together with some kind of standard shorthand way to refer to them without embedding an entire inline URL, we could have truly open clip art instead of this weird semi-proprietary mess.
  - amelius 3y ago
    In my ideal world, people would have a standard library of SVG emoji on their phones, they can send them around, but they can also save the emojis others use to their own library. Of course, artists can add new SVG emoji and send them around. This way, we get an evolving library of emoji, just like how languages evolve. Unicode is stiffling this evolution.
    - biztos 3y ago
      Interestingly (but commercially) the very popular Line app, which IMO has the best "stickers," lets you do something like this. https://en.wikipedia.org/wiki/Line_(software) https://en.wikipedia.org/wiki/Line_(software) If I buy a set of stickers (little animations, much more expressive than emojis) I can send them to anyone in chats, or I can send the set as a gift to that user; in the latter case, the user can add them to their library. Your library is the standard stickers, any free stickers you've added, anything you've bought, and anything you've been gifted. The end result is that some people have a fairly distinct sticker-communication style because they have in their library relatively obscure ones, or just like to use the less-popular free ones. In theory anyone can make their own stickers but it's kind of a pain, and I haven't gotten around to it: https://creator.line.me/en/stickermaker/ https://creator.line.me/en/stickermaker/ So in my ideal world, we'd not have any emojis that can't be reduced to expressive ASCII, but we'd have a massive public library of freely available SVG Stickers with accessibility features, and all the chat apps would use them.
- zokier 3y ago
  > How do you propose that Unicode include glyph innovations that are digital-first? Why does Unicode specifically need to include novel "innovations"? It's arguably failure of the community that we have not been able to standardize interoperable higher level rich text formats, so now everything and kitchen sink needs to be bolted on Unicode instead in the name of interop.
  - Sesse__ 3y ago
    > Why does Unicode specifically need to include novel "innovations"? Unicode chose to get into emoji because Japanese carriers were already making their own private character sets with emoji, and Unicode has a goal of being a superset of all other (relevant) character sets. (Arguably emoji support in Unicode has been a resounding success. Look at the world-wide enthusiasm around each new Unicode version that gets announced now!)
    - ygra 3y ago
      It also brought broken handling of code points beyond the BMP to the forefront and dealing with Unicode text is correct in much more places by now. Previously the only people who noticed were those who needed obscure Han ideagraphs, hieroglyphs, and other things that are not terribly useful or interesting to the vast majority of people.
- ilyt 3y ago
  > (How do you scribble "family of three: mom, father, daughter" without any emojis?) Why you need to? We dealt without hieroglyphs just fine in communication.
  - Closi 3y ago
    The tahitans can deal with communication just fine with only 13 letters, so I guess we could probably trim the letters b, c, d and g out of unicode too. With enough optimisation we might just be able to communicate with just a boolean-based language. (i.e. why limit written communication rather than make it richer)
    - zamadatix 3y ago
      That you can take something too far in one direction doesn't give reason or explanation for anything except "don't take it too far in that direction". There are probably good explanations why not to take things infinitely far in either direction but they don't really help explain where the line should be drawn. I think the actual argument here is encoding of seemingly arbitrary colored pictorial combinations overcomplicates character encoding. If you want to display a colored drawing of an arbitrary family SVG is already a thing but if you want to textually encode an arbitrary family you should use characters in your language not expect the text encoding to pick up more arbitrary drawings. To me, using Emoji was probably a great way to force developer's hands on supporting certain encoding features more complicated languages use even in cases they only wanted to support latin text. That said, this job is already done. We don't need to continue putting everything into ever more complex pictorial encodings via Unicode for the rest of time.
      - Closi 3y ago
        That’s true, but if your argument can be extended to absurdity it probably can't be used to justify where the 'correct' place on the scale is to land.
      - avgcorrection 3y ago
        > That you can take something too far in one direction doesn't give reason or explanation for anything except "don't take it too far in that direction". Why or how does the Tahitian language take things too far?
- phito 3y ago
  > How do you scribble "family of three: mom, father, daughter" without any emojis You don't. You use text.
  - ungamedplayer 3y ago
    I still was trying to figure out how an emoji represents a family.
    - aendruk 3y ago
      With the scheme demonstrated at the beginning of the article. 1F468 man 200D zero width joiner 1F469 woman 200D zero width joiner 1F467 girl
      - worthless-trash 3y ago
        Thats just three people close together, could be a Torball team.
    - yomlica8 3y ago
      You use the pregnant man emoji, which represents mother, father and child together as one.
- tambourine_man 3y ago
  You write them down. With letters and words. See, we invented this thing called the alphabet. You memorize some 20 symbols and combine them to express every possible emotion, instead of memorizing a gazillion symbols. It’s no wonder hieroglyphs have fallen out of favour in the last 5k years.
  - avgcorrection 3y ago
    Thank you for your contribution. :)
- Cthulhu_ 3y ago
  I don't speak good emoji though; I understand the smiley face, but what does posting the family emojis mean? What is it used for? Nobody I know or have seen online replaces random words with emoji for brevity. That's annoying. At worst I've seen ironic posts or trying to be hip outlets adding the emoji for a word after the word itself, or use them as bullet point markers. It reminds me of a thing where it was like, "elephants are so smart because they can use this trumpet to communicate danger", and someone pointed out that humans can do that too by using words. Mind blown.
- perryizgr8 3y ago
  > How do you scribble "family of three: mom, father, daughter" without any emojis? Using nifty little things called jpegs.
  - Timon3 3y ago
    How do you propose they get transmitted? In-band in the middle of text (good luck reading that with an unsupported editor), or through servers, necessitating NAT and all that good fun? Since the images can be arbitrary you also open yourself up to bugs in the parser. All around a much, much worse idea compared to text emojis.
    - perryizgr8 3y ago
      > good luck reading that with an unsupported editor You need a "supported editor" to display Unicode emojis too, as TFA shows. And supposedly it is quite hard. OTOH displaying jpegs is a solved problem.
      - Timon3 3y ago
        > You need a "supported editor" to display Unicode emojis too, as TFA shows. Let's give this a try. This is roughly what a sentence with an emoji looks like in an unsupported editor: hello [] world (with [] being the well-known "placeholder" square). Now let's try the same with an emoji, I went with 16x16 to make this kind of bearable, but realistically you'd want at least 64x64: hello data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEASABIAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQH/2wBDAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQH/wAARCAAQABADAREAAhEBAxEB/8QAFwAAAwEAAAAAAAAAAAAAAAAAAAIICv/EABwQAAIDAQEBAQAAAAAAAAAAAAQFAgMGBwEIFP/EABoBAAEFAQAAAAAAAAAAAAAAAAACAwUGCAn/xAAhEQACAwEBAAICAwAAAAAAAAADBAECBQYHCBITFAAVFv/aAAwDAQACEQMRAD8A1xbv6+xizYBjaeOtvSMunT5XmVeTDaMAP1waXJyNft7FFlPoub9OHjK5gylJQmBICnfTMgi+6PJ31n5YubHpnY4S5H1eV89d3MumSj0TmCPdLhNmR0dDQNmtpM65HTrHjKxrGlSiY4JdczZL3ppUPljuLz2OdUaRNLYxkt119xdcp1qvKjcXzcyGqFquRYJq1YKGsNMNQSsFqCg6SYX6+xjPYGDZivWjpVnTYcr06vWBtFy/0qbSlPRsMTNtbb6VnPDSJSoYrfYqHIQxsqKIkUUX+nlHyybyPS+OwjkeZ5b0F3DzL5LvQubwsMu44FFB/PNotus45UjtB/tMehoUIpeb1XC0OhJDeWu7XPa5mqJD0cjGd3En011xHZ/RVK4fN04VGKpyMADaoSHiWlmfxRJLAuQf8hP6/wCB/SXGOkaDZ8hzrbo/P9m4cPaglrLqYUMzY1bMHRKs2rm7sSauQJTQgcNqyXkL2CyC+uZwRYNw5E5634bPnnYdjtF8WzPYuM7vo9DrQPL4mrodHyWlpkufUyT2wzhfpi2Nf9hMn1MvWZvFq0Na/wBrpyHYZHoGDgotdn/juj5vJTwzpuuZS+duLJCoBTRWtrhuuV2wxxRoNCCNF5mYggvrML8fcA+kuydHRbPryFxz3BZFumdXAs2PUzB9D4qbrnlCoCrpLwyxrewKVjCmN1q8ZetVTY0wNNKYVD0NeTeF39G7Pjdsfi+Z45xXBdCj1bDh8TUz+i67TziiPl5Qp3DnfJjjMKTtkn8K8zMfWtzVH9V9h2GN59z+8ir2Udl0nSZTeKuok7lsZ+Eu2Ei7WgxbICNcLlRkmq4bXIebxWZgYvvaf//Z world Now, which one is more readable?
- GuB-42 3y ago
  > Unicode needed to include written-on-paper glyphs that existed in the world. If it was the case, Emoji is seriously lacking a penis. I mean, seriously, give men a way to draw and this is what you will get. There are already existing "subjective combinations" like 8===D and the eggplant emoji for which it is its most common use. There is a proper penis in the hieroglyphics, but just because there is a hieroglyphic doesn't mean there is no corresponding emoji (ex: eye). On the other hand, many emoji pass even though I have never seen anything remotely similar being scribbled ever, it is the addition to the emoji block that drove its use. Plus, because inclusivity, all its equivalents in different genders, skin tones, cultures, etc... Clearly, Unicode acts as an arbiter here. It decides on what it thinks it is "good" rather than what is really in use. And it is not just about penises, the hangman is also absent, as is the "gun pointed on head" sign (suggesting suicide) that is commonly used in real life. Worth noting that the first emoji were included because they were already in common use before Unicode, in Japanese phones specifically. Unicode just included them so that Japanese users could switch to Unicode without loss of functionality.
  - cnity 3y ago
    Unicode actually does include a penis: 𓂸 There is even a "phallus with emission" variant.
    - noAnswer 3y ago
      Interesting: U+130BA EGYPTIAN HIEROGLYPH D053 (phallus with liquid issuing from it) 𓂺 Not even BabelMap or BabelPad have a font showing more than a square.
      - GuB-42 3y ago
        I just discovered that the default Windows 10 font censors phalluses except when they are next to another hieroglyph. 𓂲𓂺
- khiqxj 3y ago
  [dead]
- terracottalite 3y ago
  I think the emoji combining isn't that useful, at least in the family example. It is enough putting them side by side without merging them. That way you have even more possible combinations, like a family of an ogre and a human and a kid. Sorry if this doesn't make sense.
yorwba 3y ago
Unicode is only curating the clip-art collection, inventing brand-new entries is left to interested third parties. So a closer analogy would be the Oxford English Dictionary adding newly-coined words, which—of course—they do https://www.oed.com/discover/the-oed-september-2021-update/ https://www.oed.com/discover/the-oed-september-2021-update/ in addition to expanding coverage of older words (analogous to the Unicode Consortium working on historical scripts). NB if you don't like multi-color emoji, you can also use a monochrome font for them.
- pavlov 3y ago
  It’s not the same thing because Unicode is the only gatekeeper for emoji adoption. You can’t use an emoji that isn’t yet in the standard and deployed by the OS vendors. Whereas OED only adds words that have substantial real world usage. Re: monochrome emoji rendering — an impossible proposition if you need to render any user-generated text. People simply can’t understand that their emoji might look different than when they picked it on their iPhone keyboard. The supposed rendering latitude on emojis is completely imaginary; in practice you need to get an emoji set that’s as close as possible to Apple’s without infringing on their design copyrights.
  - yorwba 3y ago
    > You can’t use an emoji that isn’t yet in the standard and deployed by the OS vendors. You can, as long as you control the font. Pick any codepoint in the Private Use Area and have your font define a picture for it. That's the whole idea behind icon fonts. > People simply can’t understand that their emoji might look different than when they picked it on their iPhone keyboard. Android users who pick an emoji on their Google keyboard and then have something different show up in the message they sent seem to be able to cope somehow. But if you can't get away from imitating the Apple look because of wrong users, you could still try converting it to monochrome. Maybe users will forgive the deviation if it makes sense in context, e.g. for a terminal emulator.
    - ElectricalUnion 3y ago
      The terminal emulator is one of the places where I actually expect coloured emoji support. Granted, that it's supposed to emulate a teletypewriter but I think that ship sailed a long time ago.
Sesse__ 3y ago
The Unicode Consortium's stated direction is that they want protocols to support stickers (arbitrary images) instead of emoji. But protocols don't seem to be doing that, so…
- pavlov 3y ago
  If you’ve dug yourself into a hole and nobody is throwing you the rope, at least stop digging.
amelius 3y ago
My main problem with emoji is that they look different everywhere. For example "big grin" looks like that on native Android, but looks like an angry face on other platforms. Quite a difference in meaning. Another problem is that some emojis are horribly missing. For example during COVID it would have been nice to have a cotton swab thing. :face-palm:
- Cthulhu_ 3y ago
  Yeah but... why? So social media outlets can make coming in for a test c00l and hip and down with the kids? Does a swab emoji improve the message or is it just colorful seasoning?
- whywhywhywhy 3y ago
  > For example during COVID it would have been nice to have a cotton swab thing Why? It would be next to meaningless now but have to be supported forever. This actually exemplifies why they need to be more strict about what goes in.
  - amelius 3y ago
    No, it exemplifies that we don't need a central authority like Unicode to control what we can/cannot say in emojis.
    - Joker_vD 3y ago
      Funny how alphabetic text allows you to write whatever by just combining ~30 symbols in different order and quantities but for hieroglyphics you need thousands of symbols and it still is not enough, and never will be.
      - amelius 3y ago
        A picture can say more than a thousand words ;)
deleted 3y ago
[deleted]
Macha 3y ago
Honestly I'm happy enough that emoji are there as a carrot to upgrade their platform support for newer unicode revisions which the US otherwise might not care about. It's only been 10 years since you couldn't reliably use the € symbol in web forms for example. Never mind things even further afield from the US
pfp 3y ago
Agreed. The whole emoji phenomenon is a kind of infantilizing cultural rot -- it makes serious, static documentation and tooling resemble a children's book and hinders live communication by encouraging vague single-pictogram messages and the expression of raw emotions (genuine or not) instead of mature and balanced thoughts.
- giraffe_lady 3y ago
  None of these things are mutually exclusive the way you're implying. This is a "cultural degeneracy" argument, which are always suspect imo. You are certainly entitled to dislike the aesthetics of shifts in communication, but you're basically just assuming that the changes are inherently negative and there's no reason to think they are. "Single-pictogram messages and expression of raw emotion" are simply not mutually exclusive with mature, well-considered, intentional communication.
khiqxj 3y ago
[dead]
deleted 3y ago
[deleted]

panki27 3y ago

Interestingly, Microsoft Terminal renders all of these correctly!

frizlab 3y ago
With the proper width and all?
- dr_zoidberg 3y ago
  I ran a few tests in Windows Terminal. The bomb emoji got width 2, while the motorboat got width 1 and correct aspect ratio, though I didn't quite get to see it properly until I zoomed in like 5x. The family was rendered all as one emoji, cells wide, but left 4 blank cells before it. So it was a bit better than the authors tests, but there's still room for improvement.
xioxox 3y ago
So does KDE's Konsole for me.
foepys 3y ago
Microsoft Terminal is actually pretty good. I was surprised when I configured git bash as my main shell there and when opening it via right-click in Explorer, it correctly translated the path and opened the shell in the correct folder.
- chii 3y ago
  Is this the same terminal that Casey Muratori was complaining about how there's basic performance issues? (see https://news.ycombinator.com/item?id=28743687 https://news.ycombinator.com/item?id=28743687)
- jlokier 3y ago
  I didn't like Windows Terminal when I used it over the last few months at the last $job because my choices of software were limited and it was better than the Windows Console. There seems to be no way to make the cursor clear and visible on all background colours that I used for CLI and text editing. I ended up having to use a difficult-to-see 50% grey cursor, as a compromise that was better than losing the cursor completely sometimes. It wouldn't send Control-Space or Control-@ from the keyboard (they are the same character, ^@ aka NUL), which is the key used to set the editing mark in Emacs so used a lot. No key combination is mapped to that character in Windows Terminal, and I couldn't find a way to customise it to do so either. Historical real terminals like the VT100, and of course xterm etc, always did so, which is why it's an essential key in some terminal applications. I compromised by binding Control-] to set-mark-command in Emacs but this was annoying when switching between devices. Finally, line drawing characters, boxes, progress bar characters and such had ugly gaps, as though line height was incorrect for them. All of these seem to work fine in every other terminal emulator I've used.

draw_down 3y ago

[dead]

LorenDB 3y ago

I never have understood the push to emojify everything. But commit messages, options in menus, status outputs, you name it: I think we'd be better off leaving it in plain ASCII (or Unicode if your language needs that, just as long as you don't dip into the emoji). Emoji are quite nice for messaging people and occasionally some of the more generic ones can be useful for status indicators (think the red and green circles), but I don't want to be presented with a little green worm every time somebody submits a bug fix on GitHub.

avgcorrection 3y ago
I think emojis in commit messages are very overrated. But emojis are great in informal communication when you want to compensate for the fact that your words are completely divorced from inflection, tone, and body language.
- petepete 3y ago
  I once had a colleague who started every commit message with an emoji. It became tiring very quickly. Still, the odd facepalm, bug or bomb once in a while doesn't really do any harm. I use the robot emoji in my ~/.mailmap so I can easily identify bot commits (dependabot, snykbot etc) when scanning through commit logs locally.
  - LorenDB 3y ago
    Absolutely, emoji used spontaneously are great. When they are used as qualifiers or attributes constantly, it becomes just as bad as typing "feat:" or "chore:" or something similar in front of every commit message.
Joker_vD 3y ago
> or Unicode if your language needs that Благодаря ти много, добър непознат.
the_gipsy 3y ago
There are many languages that have multi-width symbols, it's not only emojis.

38 3y ago

works fine with Windows Terminal: http://0x0.st/H_S1.png http://0x0.st/H_S1.png but yes to be fair, the width on those ridiculous 5 codepoint emoji is off.

maleldil 3y ago
Kitty on macOS also works fine with all cases, including the correct width.
- Sesse__ 3y ago
  Kitty has the wrong width: https://github.com/kovidgoyal/kitty/issues/3810 https://github.com/kovidgoyal/kitty/issues/3810 It doesn't immediately overwrite the emoji with the next character, but the cursor position is off, which leads to wrapping problems down the line.
bazzargh 3y ago
Is it windows or is it golang making the difference there? Golang's implementation of wcwidth is different, and a commonly used lib has an 'emoji' table which covers that bomb (1F4A3) https://github.com/mattn/go-runewidth/blob/master/runewidth_table.go#L433 https://github.com/mattn/go-runewidth/blob/master/runewidth_... The family emoji looks to have the wrong width, but it does resemble the behaviour supposed to be fixed by https://github.com/mattn/go-runewidth/pull/63 https://github.com/mattn/go-runewidth/pull/63 (see https://github.com/mattn/go-runewidth/issues/59 https://github.com/mattn/go-runewidth/issues/59 - like the flag example, that's multiple emojis with zwj right?)
- 38 3y ago
  I just used the standard library, not even any imports: https://godocs.io/builtin#println https://godocs.io/builtin#println

frizlab 3y ago

Haaa yes the eternal emoji width problem when using monospaced fonts… I added an explicit workaround for this in my logger lib, with an environment variable to be able to change the behavior. https://github.com/xcode-actions/clt-logger/blob/12a5ebc1b00ec0d3588fe168cef29bad96e0c7f6/Sources/CLTLogger.swift#L135 https://github.com/xcode-actions/clt-logger/blob/12a5ebc1b00...

jansan 3y ago

IMO we should have left emojis out of everything that is rendered by convention with monospaced fonts. But of course I respect the desire for displaying rainbow farting unicorns even in consoles, so let's find a solution for this.

Joker_vD 3y ago

Honestly, seems like the problem with wcwidth's implementations. Most of them have been reporting almost all emojis as having width 1 (and some very non-invisible symbols as having with 0) for years, to this very day, for some unexplained reason despite regular web rants of how this is wrong. Check out this one: [0]. It's 8 years old! The problems are still there. [0] https://eev.ee/blog/2015/09/12/dark-corners-of-unicode/#combining-characters-and-character-width https://eev.ee/blog/2015/09/12/dark-corners-of-unicode/#comb...

appplication 3y ago
I cannot even begin to fathom how you would of about fixing this more broadly. I’m sure there are many who have come to count on the current behavior for some critical use case, and would vehemently oppose any attempts to change of correct. Hyrum’s law, etc.

Sesse__ 3y ago

I don't know of a single terminal that actually gets these ZWJ emoji sequences right. There are several terminals who will render them, but I haven't seen any actually get the width calculations correct. (E.g., https://github.com/kovidgoyal/kitty/issues/3810 https://github.com/kovidgoyal/kitty/issues/3810.) Part of the problem is that it really messes up the idea of a rectangular grid of base characters in memory, so you'll need some sort of indirection. Obviously I haven't tested all terminal emulators in existence, though :-) I patched a similar bug in screen earlier this year (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1039503 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1039503), which indicates the kind of stuff you have to go through.

hnlmorg 3y ago
I found wide character support to be surprisingly good across the terminal emulators I tried on macOS and Linux. This testing was done earlier this year when I was building support for Unicode variable names into my $SHELL (more to support foreign languages than glyphs but the end result is the same) It’s worth noting that wider character support needs to be implemented in both the terminal emulator AND and the console applications that run on the terminal emulator too.
- Sesse__ 3y ago
  So which ones support the ZWJ sequences correctly and fully?
  - hnlmorg 3y ago
    I cannot claim to have tested every ZWJ character on every terminal emulator but if you find any ZWJ related bugs in murex shell https://GitHub.com/lmorg/murex https://GitHub.com/lmorg/murex then it will be treated as a serious bug because Murex does have international users so aims for greater compatibility with other language writing systems (albeit the documentation is presently only available in English).
3np 3y ago
Have you checked mlterm? Its supported for other otherwise-neglected encodings and fonts is otherwise unparalleled AFAIK. https://wiki.archlinux.org/title/Mlterm https://wiki.archlinux.org/title/Mlterm

RedNifre 3y ago

Are there some well defined Unicode subsets, e.g. one without Emoji? (I'm only vaguely aware of XML specifying "Unicode without control characters" in the beginning).

Joker_vD 3y ago
Yes. ASCII is one. Latin-1 is another.
- ElectricalUnion 3y ago
  Depending on what degenerate system you are in, U+00A9 © COPYRIGHT SIGN might render like a emoji, so ISO/IEC 8859-1 isn't safe.
avgcorrection 3y ago
Basic Multilingual Plane should be it.
- deathanatos 3y ago
  While most emoji are outside the BMP, not all are. E.g., {U+2622, U+FE0F} (radiation symbol + variation selector) or even just U+2728 (sparkles). See https://unicode.org/reports/tr51/#Presentation_Style https://unicode.org/reports/tr51/#Presentation_Style
- ElectricalUnion 3y ago
  Unicode Character 'HOURGLASS' ⌛ (U+231B) says hello.
  - avgcorrection 3y ago
    Hello.

joshka 3y ago

Some deep context on this is at: https://gitlab.freedesktop.org/terminal-wg/specifications/-/issues/9 https://gitlab.freedesktop.org/terminal-wg/specifications/-/...

mcluck 3y ago

I'm dying to know what the point of N is if it should be treated the same as W

wodenokoto 3y ago
I think it is because those are unicode points that can be displayed as either emoji or text. (e.g., https://www.compart.com/en/unicode/U+262F https://www.compart.com/en/unicode/U+262F). so when displayed as text they are N

tambourine_man 3y ago

I don’t want emoji in my terminal. They are already everywhere. I chose my color scheme carefully for some candy color to pop up. It is useful to be able to see them on database dumps, to be able to clean them with sed, etc. But it should be a toggle, IMO.

rivo 3y ago

I spent quite a bit of time coming up with an algorithm that gives me the correct width for any Unicode character, including emojis [1]. The reality is much more complicated than what wcwidth does and using the East Asian Width attribute will give wrong results, as outlined in the article. Unfortunately, since a lot of terminals simply use wcwidth, you often have to use that same flawed algorithm in your application to make it work in those terminals. [1] https://pkg.go.dev/github.com/rivo/uniseg#hdr-Monospace_Width https://pkg.go.dev/github.com/rivo/uniseg#hdr-Monospace_Widt...

Tyriar 3y ago

I'm on the VS Code team and maintain xterm.js which is what Hyper's frontend is based on. There are actually multiple developments happening in this area. First, there's a contribution from the author of DomTerm which adds grapheme cluster support to xterm.js, which will correctly merge and size things like emoji that are called out in the post. This is currently based on Unicode 15. See https://github.com/xtermjs/xterm.js/pull/4519 https://github.com/xtermjs/xterm.js/pull/4519 Second, while Windows Terminal does seem to work with emoji sometimes, it doesn't all the time. I'm not 100% sure, but I think it may only work on Windows ptys, not in WSL for example. Last time I spoke with the team they said they're working on a rewrite which could lead to proper emoji support.

rivo 3y ago
I spent a lot of time on this topic (see https://news.ycombinator.com/item?id=37048434 https://news.ycombinator.com/item?id=37048434). Feel free to get in touch and discuss. I'm happy to help.
- Per_Bothner 3y ago
  I'm the author of DomTerm and the above-mentioned xterm.js PR. Both use the full UnicodeGrapheme Cluster Boundaries algorithm (https://unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries https://unicode.org/reports/tr29/#Grapheme_Cluster_Boundarie...). However, I haven't seen any specifications for how wide the resulting clusters should be in a mono-space context. So unless we enhance terminals to handle variable-width fonts (which I've been thinking about), we need to took at other terminals and make judgement calls. Generally, the width of a cluster is the width of the widest codepoint. Also, I decided that a "normal" character followed by emoji-presentation makes it 2 columns wide. In your linked-to article, you suggest 2-em dash and 3-em dash should be 3 and 4 columns respectively. That might be reasonable, but it is explicitly contrary to the EastAsianWidths specification. You also suggest that Pictographics fullowed by text-presentation should have width 1. That seems reasonable, though I don't implement that.
  - rivo 3y ago

zanellato19 3y ago

Wezterm rendered all of these correctly :)

ChrisArchitect 3y ago

(2021)

ChrisArchitect 3y ago
Some previous discussion: https://news.ycombinator.com/item?id=30113521 https://news.ycombinator.com/item?id=30113521

bitsandboots 3y ago

While this is a real issue, that effects terminals in real-world use, I took issue with this quote: > This issue affects every terminal I've tested: Visual Studio Code, iTerm2, Alacritty, and Hyper. I don't think those terminals are particularly known for being feature-complete with regards to VT/ANSI codes or new developments such as emoji. The quantity of things a VT-ish terminal must do is massive and not standardized, so many terminals just cover the common cases and nobody notices that they're incomplete. If the author noted that emojis don't work properly in xterm (not xterm.js) or konsole, then that'd be something to write home about. For reference, I've written a terminal emulator fairly recently, and found xterm to be most faithful and also a great source of documentation in itself. In personal use though, konsole is pleasant and seems to do everything I've seen right. On the other hand, currently writing this from mac and iterm2 is kind of terrible, send help.

bestouff 3y ago
GNOME terminal works well for me. And from experience I'm not surprised the terminals cited in the blog post don't work.