20 ms·
The bottom emoji breaks rust-analyzer
- kerkeslager 4y agoThat's an awful lot of time spent talking about your Emacs config...
- mi_lk 4y ago+1 appreciate the effort, but just get to the point thanks...
- dmm 4y agoI liked the emacs config discussion.
- owenmarshall 4y agoMe too! Fixing Emacs + this whole `rls` strangeness was a yak whose shave I've been putting off for a while, and this just gave me quite a head start. Plus, one of my biggest gripes with Emacs documentation that it is very hard to find good articles that are contemporary + shows the full configuration + comes from a perspective of a user using a tool, rather than a programmer building their environment from scratch. Yes, I know, Emacs is one of the quintessential "environments built from scratch", and I engage in that too – sometimes to my detriment. But some days I just need to get python-mode / Poetry.el / eglot+Pyright to all play nice together and an article like this would go a long way.
- deleted 4y ago[deleted]
- Ygg2 4y agoIt's fasterthanlime. It always takes ages to get to the point.
- deathanatos 4y agoI love the author's style. Not every blog need do it, but I appreciate that fasterthanli.me does it. To me, it is intellectually honest: this shows a reader every step along the way, every painful trail that must be overcome from point A to point B. Nothing is omitted. And I think the sooner we all did this, as an industry, the sooner the very many problems and bugs that exist (that get hit before we can even "get to the point", as you say) would get dragged into the light, and maybe we'd progress, as a society, towards having computers that weren't shit.
- Ygg2 4y ago> Nothing is omitted To quote internet reviewer: Brevity is the soul of wit. That means stop wasting my time. Keep it nice and simple. Look. Like what you want, I'm free to prefer a shorter form, and to point out this is part of author's style.
- kelnos 4y agoAgreed, but I personally enjoyed the detour into "wow, our tools have terrible UX" territory. Totally get that some people would find it superfluous, but I thought it was fun. I'm a vim user, and find the landscape to be pretty bad there too; it's nice to see that emacs is no better (and IMO worse, based at least on this one example).
- andreareina 4y agoThe perils of combining executive config with long-lived sessions. I frequently try something out by evaling it first to see if I want to put it in my permanent config then forget to actually add it. Mickens almost had it right about "perfect window placement", it's just perfect editore state (including open files and caret position)
- kerkeslager 4y agoDo they have terrible UX, though? I'm also a vim user as my primary editor. My .vimrc does very little beyond the default .vimrc. I don't want to spend any time configuring my tools, I just want them to work. And that's what I get with vim, on any Linux system out there: a vim that works pretty much exactly the same as the config I use on my own machine and know well. If it's not already installed (and it often is), vim is just a package manager call away. If you really think your tools have terrible UX, find/build better tools? I'm not convinced time spent in my dotfiles isn't just time wasted. I don't want to hack my tools, I just want my tools to work. And vim does. Most of the tools I've come across for vim, are already in vim with good enough UX. The exception is language-specific tooling for stuff like highlighting compiler/linter errors or failing tests, but it's a massive amount of work to get that stuff working, and I'm not sure that pays off when I can just run that stuff from the shell. Sure, it takes a second to switch to another command line tab to run the tool from shell, but how many 1-second switches does it take to add up to 4 hours spent debugging my .vimrc? It's not worth it. And, by the way, I'm not throwing any shade here. If you like fiddling around with your config files, that's an entirely valid reason to spend any amount of time that you want, fiddling around with your config files. Do what you enjoy--you don't need my blessing, but you have my blessing. Just be honest with yourself about why you're doing it, and do it on your own time. If you need to complete a task in a timely manner, it's extremely unlikely that any step in completing that task involves your editor configuration. There's absolutely no way that all that Emacs configuration was the fastest way to reproduce that bug in the OP.
- bhaney 4y agoFirst time I've heard "pleading face" referred to as "the bottom emoji"
- fasterthanlime 4y ago[flagged]
- decremental 4y ago[flagged]
- bhaney 4y agoI'm not really sure why I should take it as a sign of that. What crowd does this indicate I don't hang out with?
- tullo_x86 4y agoQueer people. This is an easily-inferred reference for anyone who's decently-good friends with at least one gay guy or lesbian.
- bhaney 4y agoI have a handful of very close friends who are gay, but I've never heard of this bottom-face emoji thing before today. And I'm a little concerned why not knowing about something that seems like an esoteric piece of fetish-related in-group communication implies that I don't hang out with queer people? Is there an expectation that in order to hang out with people belonging to some group, I need to learn specific lingo regarding that group's sexual power dynamics?
- justincredible 4y ago[flagged]
- mplewis 4y ago
- pnathan 4y agoSo this all justifies my general stance of "find a Unicode expert" when questions of unicode come up. But that's pretty wilfully blind (although practical). I would like to understand why UTF-32 didn't catch on as The Standard Unicode for the modern world. it seems that - albeit memory wasteful - it would sidestep a lot of these issues.
- layer8 4y agoUTF-32 doesn’t completely solve this either, because grapheme clusters.
- pnathan 4y agooh, _bother_. :-/
- bsder 4y agoAnd endianness.
- layer8 4y agoSince Unicode only uses 21 bits, it would be possible to define an endianness-safe 32-bit encoding. E.g. shift left by one and set the LSB.
- a1369209993 4y agoTo be fair, UTF-32 is actually much better about this than most other encodings for which endianness issues are even a thing. The top byte of any UTF-32 code unit has to zero, and NUL isn't a valid (ie text) character, so any ASCII characters anywhere in the file make the endianness unambiguous. More generally, every code unit in the file has to have the form 00xxyy00, and the possible values for xx and yy must be in the range 0 thru 16, so there are only 17*17-1 = 288 unicode code points that can possibly occur in a endianness-ambiguous string/file. And out of those, you have confusions like "Ā" (U+0100 LATIN CAPITAL LETTER A WITH MACRON) versus "𐀀" (U+10000 LINEAR B SYLLABLE B008 A) or "𠀀" (U+20000) versus "Ȁ" (U+200), most of which can only happen if you have no idea what language you're expecting. (Some, like "𐄀" (U+10100 AEGEAN WORD SEPARATOR LINE), are the same regardless.) Whereas here's a valid bash script: ⌀℀ ⼀戀椀渀⼀戀愀猀栀猀甀搀漀 椀搀⌀ 2>/dev/null || true echo "Hello, World!" # (You can almost do this with C/C++ as well, but something needs to #define "⼀⼀" as "int" for it to work properly, and then only if the compiler accepts non-ascii characters (namely "⼀" aka U+2F00 KANGXI RADICAL ONE) in identifiers.)
- lacrosse_tannin 4y agostory of my life with emacs.
- zamalek 4y agoTo be fair, other operating systems took a while to get unicode right.
- secondcoming 4y agoI gave up trying to read this. Where was the bug?
- yoru-sulfur 4y agolsp-mode: https://github.com/emacs-lsp/lsp-mode/issues/2080 https://github.com/emacs-lsp/lsp-mode/issues/2080
- deathanatos 4y agoThe LSP protocol sends indexes. Insanely, those indexes are in terms of UTF-16 code units. Emacs's LSP client implementation here is sending the wrong index: 8 for the emoji's index, but 9 for the index of the next "character". But an emoji spans two UTF-16 code units, so the next index is 10. Rust-analyzer simply crashes here, but it's been fed hot garbage by the editor. One might argue it shouldn't crash. TFA digs into the details around that, too, because Amos leaves no stone unturned.
- zerocrates 4y agoI imagine this "UTF-16 code unit indexing" decision is just an artifact of the fact that that's how JavaScript works with strings, and LSP comes from VSCode.
- deathanatos 4y agoOh, TFA mentions exactly that. … but still, encoding your language's idiosyncrasies into the protocol is … poor design. This bug was inevitable with such a choice (although even a UTF-8 byte offset, or a scalar value offset would probably be similarly fraught with error, but UTF-16 seems like begging the universe for it), though the actual conclusion here was a bit different than I thought it was going to be. (And yes, I know other JS UTF-16 idiosyncrasies made their way into the very fabric of JSON … and those are ugly too.)
- teddyh 4y agoNitpick: lsp-mode is not “Emacs's LSP client”. Emacs recently chose to include “eglot-mode” as part of Emacs itself, and eglot-mode must therefore be considered to be Emacs’ official LSP client, not “lsp-mode”.
- aendruk 4y agoMisleading headline. > the actual bug: let's add to our code.. an emoji! Any emoji. > rust-analyzer adheres to the LSP spec. And lsp-mode doesn't. So “emojis break an Emacs extension”. Though for me the real takeaway is that LSP specifies UTF-16 offsets. That sounds unpleasant to work with.
- deleted 4y ago[deleted]
- Ygg2 4y agoThanks, for people looking for bug, see https://github.com/emacs-lsp/lsp-mode/issues/2080 https://github.com/emacs-lsp/lsp-mode/issues/2080
- pja 4y ago> Though for me the real takeaway is that LSP specifies UTF-16 offsets. That sounds unpleasant to work with. Speaking as someone who has written an lsp server: yes, it is indeed unpleasant to work with.
- stouset 4y ago> Though for me the real takeaway is that LSP specifies UTF-16 offsets. That sounds unpleasant to work with. It is beyond time for UTF-16 to die. And yet it looks like we're stuck with it for the inevitable foreseeable future.
- puffoflogic 4y agoWe'll be stuck with UTF-16 as long as we have Unicode, it would seem. The character set itself has to have a hole in the middle of it for the sole purpose of working around UTF-16 limitations.
- deleted 4y ago[deleted]
- fasterthanlime 4y agoThis submission went from #5 to #42 in a minute so I'm assuming it's been deranked. An explanation would be nice, in case a moderator is around. edit: now it's no longer there at all. Welp.
- Macha 4y agoCould be automated, there's the flamebait detector that deranks if it gets a high comments/vote ratio.
- steveklabnik 4y ago(not a mod just been on this website forever) At this exact moment, there's 49 points, but 45 comments. If comments >= points, you get a significant ranking penalty (the "flamebait detector", iirc). I can't say for sure that that's happened, but given how close the two numbers are, I bet at some point that was true.
- fasterthanlime 4y agoAhhh, that sounds likely, thanks!
- deleted 4y ago[deleted]
- andreareina 4y agoGiven the controversy on-thread about the length of the intro, calling it the bottom emoji, and the title making it sound like the problem is specific to that emoji, I wouldn't be surprised if it has collected a number of flags as well as tripping the overheated discursion detector. FWIW I enjoy your articles in general and this was not an exception, although I could have done with a shorter exposition myself ;)
- recursive 4y agoSmall mistake. > High surrogates are D800-DB7F Akshually, high surrogates extend all the way to DBFF. https://unicode-table.com/en/blocks/high-surrogates/ https://unicode-table.com/en/blocks/high-surrogates/
- KingLancelot 4y ago[dead]
- bitwize 4y agoNext time just use Visual Studio Code for Rust development.
- ParetoOptimal 4y agoA non-core lsp library having a bug doesn't justify moving from emacs to vscode. Especially if you use org-mode, org-roam, or customize your workflow with elisp.
- deleted 4y ago[deleted]
- teddyh 4y agoThis is a bug in lsp-mode, a third-party Emacs package which is an LSP client. Since Emacs version 29, Emacs will include in its standard distribution a different LSP client package, eglot, which does not seem to have this bug: https://github.com/joaotavora/eglot/blob/e501275e06952889056268dabe08ccd0dbaf23e5/eglot.el#L1415 https://github.com/joaotavora/eglot/blob/e501275e06952889056...
- IshKebab 4y agoHa as soon as he mentioned LSP I knew it was going to be about the insane fact that it sends all text as UTF-8 (and Rust uses UTF-8) but it uses UTF-16 code points for column indexes. (I assume that is what it is anyway; I did not make it to the end, sorry.) They've had an open bug about it for years but I think most people just ignore the issue because to handle it correctly you have to convert to UTF-16 and back (again after VSCode has already done it) and it's just not really worth dealing with until Microsoft fixes their end. Actually I just went back and double checked and Microsoft did actually fix it last year! You can now negotiate to get positions in Unicode code points instead. Rejoice!
- puffoflogic 4y agoExcept, as you'll find people in these very comments arguing, if you don't implement utf16 anyways then you're not a compliant LSP. Don't forget that utf16/UCS2 is Embrance, Extend, Extinguish at its finest. Literally the whole point of utf16 - not necessarily the explicit one, but the point behind the motivations people had for choosing what they chose - was incompatibility with other software supporting unicode back in the day.
- some_random 4y agoOf all the languages to have a problem with the bottom emoji, Rust definitely ought to have the largest blast radius.
- MiddleEndian 4y agoSomewhat related, Dropbox fails to sync with certain emojis.