10 ms·

How to open a file in Emacs: a story about Lisp, technology, and human progress

262 points by mpereira 6y ago

bmitc 6y ago
Articles like this really trigger the imposter syndrome for me. I just have no patience for these type of things, and seeing others' both ability and patience to power through issues like this one, I just feel like an imposter. But these types of issues are why I've always given up on learning Emacs. When I can just install Visual Studio Code and the Remote - SSH extension and whatever else extension that I want to work remotely and get on with life, it's really hard to justify the time one needs to spend with Emacs to bend it to your will.
- PaulDavisThe1st 6y ago
  You appear utterly convinced that VSCode will never demonstrate an obscure bug like the one discussed in TFA, or alternatively that if it does you'll be able to jump ship to some other tool. I don't share your confidence, and would prefer to know that such issues can be fixed with or without vendor participation.
  - macintux 6y ago
    That seems an ungenerous and quite broad interpretation of a fairly simple assertion. Even as much as I love Emacs, I would never fault someone for choosing a more productive tool now even if there’s a non-zero risk of something breaking in the future...because there’s always something that will break on any platform. Time spent being productive now is usually worth that risk.
    - PaulDavisThe1st 6y ago
      The counter-point to that is that the world benefits more by someone choosing to actually fix whatever was wrong, rather than just switching platforms (or never choosing it in the first place because they came across the post on HN). The claim in the comment I was responding to appeared to me to be "hmm, Emacs has a problem, VSCode probably doesn't, I would rather use VSCode". It didn't seem to me to be written with much awareness of what the actual problem in TFA really was, nor with any real knowledge that VSCode actually would be more productive. It seemed to me to be more like "Ewww! Emacs has a bug, someone fixed it with a lot of obscure stuff, no thanks, VSCode here I come". I certainly respect that for many, VSCode will be a more productive platform when measured by the metrics that matter to them.
      - forrestthewoods 6y ago
        > the world benefits more by someone choosing to actually fix whatever was wrong Maybe. I imagine at some point someone wrote ffap-guess-file-name-at-point to fix a problem. And the blog author solved a problem by disabling it without knowing what problem it maybe solved. Problems eventually become complicated enough it’s not a binary fixed/broken, but rather a matter of trade-offs. Open source and shared solutions are great. But they aren’t a silver bullet either.
      - bmitc 6y ago
        You shouldn't make so many assumptions. If you're confused, ask a clarifying question instead of going off of incorrect assumptions.
  - bmitc 6y ago
    There were two points in my comment. One was of feeling like an imposter, in that even as a software engineer I lack both the patience and skill to pull the debugging off the article showcased. It's both admiration and lament at the same time. The second point was that I personally prefer to altogether avoid issues like this if I can by selecting what I feel is the best available tool at the time. I have tried to use Emacs multiple times, and while this exact issue is not one I have encountered (I did say "these types of issues"), I have come across similar issues, especially with having Emacs talk to another computer or VM. Every time I have tried to get into Emacs, I end up hitting some barrier akin to these things, and so I resign myself to using Visual Studio Code, where then I get off to coding rather than trying to bend the environment to what I'd like. And it always seems that with Visual Studio Code it both does what I'd like and works without a million gotchas. Visual Studio Code is not perfect, and I have had issues with it and expect more issues. However, they don't hit me as early as they seem to with Emacs, and that's an important distinction. The remoting extension is extremely seamless, and I have not seen an Emacs example of matching anything close to it. It really comes down to personality. If I was a carpenter and had a broken hammer, I wouldn't go off rebuilding the hammer or designing one from scratch. I'd do my best to find someone else who's already built a hammer that works. That doesn't imply I never expect the hammer to never break. > would prefer to know that such issues can be fixed with or without vendor participation Visual Studio Code is open source and well-managed. I'd prefer that a solution be merged into the tool for everyone, rather than being buried in someone's .emacs file.
    - grzm 6y ago
      > "the use of TFA" Read it as the fine article or the featured article.
    - PaulDavisThe1st 6y ago
      Your explanation involving containers/VMs makes a lot more sense to me than avoiding specific bugs when using Emacs' equivalent to VSCode's remoting extension. As a purely native developer, I have zero use for such things, but I can certainly appreciate that VSCode may have done a much better (or certainly, shall we say, a less historically indebted) version of this sort of functionality than Emacs. [ Edit: and yes, the featured article ]
- grzm 6y ago
  I think it's great that we have such a variety of editors and IDEs to choose from and find one that meets best meets our individual needs. Over the years I've used BBEdit, TextMate, IntelliJ, and Emacs for development. For the past couple of years it's been switching between the last two. I'm glad you've found VS Code to be a match for you. I've spent time configuring all of them to aid in development. ust like there's more than one hammer on the market, or more than one drill, each has its strengths and weaknesses. There have definitely been frustrating times configuring Emacs, but one of the reasons it's paid off over the years is that when I run into an issue, I can always dig into it and potentially fix it myself; and when I want to extend it to do sometime it can't do yet, I can do that, too. It's an investment, but it's one that I think has paid off, and will continue to pay off. Emacs has been applicable in more situations and environments than those other tools. (IntelliJ's tooling has been useful, but I've also run into bugs that have impeded my work, which I can't fix on my own because the source isn't open.) I also do a lot of work in Clojure, so Emacs Lisp isn't much of a jump. Admittedly this might not be the case for everyone. I think another big reason I've stuck with Emacs is that the cost of switching editors as they come along (I hear Atom is really nice, and people have been talking positively about Sublime for years) has a cost as well. Emacs is long in the tooth, which is a curse (Emacs Lisp is a really old lisp) and a blessing (it's unlikely to go away any time soon, and that time spent learning Emacs Lisp is going to continue to pay off). Please don't read this as apologia for Emacs or as a criticism of your choice, as it's neither. I'm just trying to add some context to how others may justify the time they've spent customizing Emacs.
  - grzm 6y ago
    And I'll add that you don't need to limit yourself to just one. I do most of my text editing and development in Emacs, but when I'm working in Java, I'll use IntelliJ. I miss out on some of the key bindings and workflow I'm used to in Emacs, but for me the other features IntelliJ affords me when working with Java are worth it.
    - bmitc 6y ago
      I actually prefer to use the de facto IDEs for most languages, only deferring to something like Visual Studio Code or Emacs when those don’t exist.
- legends2k 6y ago
  It isn't as complex as it seems. You can choose to be at any point in the spectrum of yak shaving; keep doing it vs skip it altogether. I did the latter with vanilla Emacs. It works just like a regular GUI text editor e.g. Notepad, gedit, etc. Only when I wanted more power I dropped into customizing my .emacs. I balance myself between doing it often vs not doing it at all. Doing the above for 5 years I've a decent personal .emacs I take with me in a Git repo. Setting up takes less than 10 mins on any new machine.
bambataa 6y ago
I always wonder how people evolve from fiddling with package configurations to using emacs at this level. My stunbling block is not so much knowing Elisp (I’m sure I can stumble through that) but just knowing the internal architecture and how emacs works overall. Has anyone come across a good guide for hacking emacs? I know there is the manual. Perhaps I just need to look at that again and get going.
- brokenwallet 6y ago
  The Elisp reference manual describes Elisp the language, and Emacs's Elisp API: https://www.gnu.org/software/emacs/manual/html_node/elisp/index.html https://www.gnu.org/software/emacs/manual/html_node/elisp/in... e.g. here is describes keymaps: https://www.gnu.org/software/emacs/manual/html_node/elisp/Keymap-Basics.html#Keymap-Basics https://www.gnu.org/software/emacs/manual/html_node/elisp/Ke...
- ernst_klim 6y ago
  Emacs Lisp Intro and Elisp reference manual, which come with Emacs. Former is concise and more suited for configuration, latter is for package developers.
- mpereira 6y ago
  > I always wonder how people evolve from fiddling with package configurations to using emacs at this level. One step at a time. If getting better at Elisp is something you want, just keep at it :)
guenthert 6y ago
"I’ve recently joined a company that for security reasons doesn’t allow their source code on laptops. Development happens strictly on workstations inside their private networks, with some using their text editor’s support for remote file editing and others running editors on those machines via SSH." I think using TRAMP or similar technologies is bypassing, if not the letter, then the spirit of that companies' security policy. Sure, not the whole repository is lost when the laptop is, but having an (ephemeral) copy of the files one is working on outside the security perimeter is kind of defeating its purpose. Why is that company granting access to remote file transfer? I'd think they would be better off restricting access to some form of desktop virtualization (the old SunRays were nice) if they must offer remote access.
deleted 6y ago
[deleted]
macintux 6y ago
It’s a shame that most of the discussion is about the first part of the essay, impressive as the troubleshooting is. His discussion of core values is quite interesting, in that it’s both relatively obvious in retrospect yet something I’ve not given conscious thought to. I’ve certainly thought about programming languages as expressions of intent, and how some attempt to be what I call “kitchen sink” languages, like C++, Java, or JavaScript, while others are much more opinionated about their purpose, like Erlang, Forth or APL. Also reminds me of the old description of UNIX as user-friendly, just picky about who its friends are.
- mpereira 6y ago
  I'm glad you persevered through it. Thanks for reading!
- akaralar 6y ago
  Exactly my feelings, the first part just shows some good technical skills and flexibility of Emacs but it was the second part that was much more interesting. I too enjoyed the part about values and it gave me some food for thought. I also can tell the author read Sapiens as there are quite a few points that were discussed in the book. Also some of the links in the article are great rabbit holes as well, like the 80000 hours website. A+ writing overall!
mlang23 6y ago
Being able to open remote files with Tramp is convenient. However, whats even cooler is that Tramp can execute remote commands. IOW, if you edit a file in a remote directory, you can also use M-x compile RET to run a remote build process and see the output in your Emacs Compilation buffer. If the paths pritned are relative, you can even use M-x next-error RET to jump to a build error location.
- macintux 6y ago
  Running remote commands is almost entirely what that part of the article is about.
brokenwallet 6y ago
This is a great post showing effective performance analysis. It's worth noting that Emacs's built-in way to find files (project-find-file) in a project does not suffer these performance issues. Try it, C-x p f. * project-find-file doesn't call expand-file-name for all 70k files (each of which involves a network roundtrip). * project-find-file will use "git ls-files" (see project--vc-list-files) rather than find (or expecting the user to install and enable fd) * project-find-file uses completing-read, which wouldn't by default call Ivy/Ivy prescient. I can see how the author has ended up here: projectile predates the built-in project support, so it must only seem natural to use projectile as a base, and apply performance fixes on top. Emacs's built in functionality tends to work better with other Emacs functionality, hence why project-find-file works well with TRAMP. It's worth trimming your config occasionally, and rebasing onto built-in functionality. The author's fast-project-find-file has a more responsive UI, however. It'd be great to have that in core.
- omaranto 6y ago
  > * project-find-file uses completing-read, which wouldn't by default call Ivy/Ivy prescient. Not by default, that's true, but for users that have installed ivy and activated ivy-mode, ivy does take over completing-read.
- omaranto 6y ago
  > It's worth noting that Emacs's built-in way to find files (project-find-file) in a project does not suffer these performance issues. Try it, C-x p f. That is your personal key binding, project-find-file does not seem to have a default key binding. (This, by I which I mean telling people that some built-in command is bound to some key while forgetting that is in my configuration and not by default, has happened to me several times.)
  - catern 6y ago
    Actually it's a default key binding in the (yet unreleased) Emacs 28. project.el has a lot of nice improvements in the upcoming version of Emacs. (I imagine the grandparent comment author just checked the version tag in `C-h k C-x p f' to see when it was released, but currently on Emacs HEAD the keybinding doesn't have a proper version tag - that's a big, albeit minor)
tgbugs 6y ago
The author quickly moves on after finding that expand-file-name is the issue and routes around the problem. The last time we hear from it is when it is patched out. But what is really going on? The problem is that many parts of Emacs and libraries built on top of them have not been made fully TRAMP aware. There is an abstraction boundary that is being crossed, assumptions are being violated, etc. because all paths are treated equally, despite the fact that most parts of Emacs can identify that a path is remote (or could be remote). Fixing the underlying issue and providing latency aware variants of commands is not easy (you usually can't just run elisp on a remote machine), but when we get there Emacs will be even more powerful! Here is an excerpt from some notes I made back in October. Consider these cases. I have 10k files that I want to move on a remote server. I have 10k files that I want to move on a local server. Where do I put the for loop? I want to put it so that I only have to call ssh once. Similar issue with using find and subshells. If I have to invoke a subshell there is significant overhead. Having even a little bit of knowledge about the latency of running a single command, or the knowledge that certain commands commute under loop reordering would make it much easier to produce efficient code, basically #+begin_src elisp (do-for-values value '(a b c d e f g) (please-do-this-on-the-remote #'command value)) #+end_src should be rewritten to #+begin_src elisp (please-do-this-on-the-remote (do-for-values value '(a b c d e f g) (command value))) #+end_src However, this is only safe if the values are from the local and not the remote and are not functions that are dependent on local state (that doesn't really matter in strict languages though). The best way to approach this might be to detect and warn before running. Or not, sometimes you do just want to be more efficient, and if you don't want the optimization then disable it some other way. The reason is because sometimes it is much easier to write the latency multiplying version, and the conversion to the latency efficient form is trivial for a computer but extremely easy to screw up for a human being. EDIT: Hah. I though I recognized Richard's reply. It was to me.
- mpereira 6y ago
  Yeah it's tricky. The reason why packages like Eshell and Magit work pretty much out of the box with Tramp is that they rely on functionality like `expand-file-name`, which will handle both the local and remote cases. It's convenient, but as you said, "assumptions are being violated". > Hah. I though I recognized Richard's reply. It was to me. Thanks for being the cause of that really nice reply :)
clircle 6y ago
I do not use projectile, but I do use rg via counsel-rg in Emacs. I wonder how that would compare speed-wise.
justaj 6y ago
> TUI Emacs has the terminal emulator (and also commonly tmux) limiting and conflicting with the clipboard, keybindings, colors, and more. None of these limitations are present in GUI Emacs. Is it me, or is this telling me that the terminal Emacs experience is just severely crippled compared to the GUI one?
- kagevf 6y ago
  sorry for the late reply ... I have been using emacs for org mode and common lisp coding for about 1 year ... In my (very limited) experience I've noticed these differences: - TUI can't display images - Some keystroke combinations are "absorbed" by the terminal so are not available. I believe they are shift+alt key combinations. A little annoying, but not a huge deal-breaker. ... and that's all I've encountered so far. I don't think TUI is severely crippled compared to the GUI version, but the GUI version is nicer to use.
abhinav22 6y ago
Wowsers! That was a long read, I skimmed through a bit of it. Kudos for dedicating time and effort to high quality long form articles - this is something that online needs to revert to. That said, can you give us a TL;DR if you have a moment :D
- lproven 6y ago
  Can it be summarised? No. Can it be described? Well, this is how I did it in a tweet, and the author like it and thanked me. « Fascinating, very long essay which starts with a discussion of text editors, wanders through software development models & the issues of modernising very old software, & turns into a discussion of intelligence, language & computing. »
radarsat1 6y ago
I use emacs but tend to just find another way to do things (e.g. grep, & other shell things) when I run into a stumbling block like this; believe it or not, not everyone wants to start debugging their text editor when they run into trouble. Admittedly I wish I would spend a bit of time in emacs internals sometimes just so I could learn to approach similar issues, but it's just never much of a priority when I'm trying to work on what I'm actually working on. I rarely get distracted by trying to optimize my config, as a result I use almost the vanilla defaults and I'm mostly happy with that. TRAMP is pretty great, I wish it would not make a new connection for every network interaction though. Setting ControlMaster on ssh helps a bit but it's still not as fast/transparent as I'd like. There are far too many "blocking" operations in emacs, as much as I love working in it, there is the non-zero times that it locks up on me and it drives me crazy when that happens. If it was ever the case that I had to wait 30 seconds to open a file, I'm sure I would abandon emacs instead of trying to debug it. That said I can't imagine working on a 70,000-file repo. That sounds like a terrible idea.
- mpereira 6y ago
  > TRAMP is pretty great, I wish it would not make a new connection for every network interaction though Same. One of my wishes for Emacs in "From caching up to getting ahead" is for Tramp improvements. These days I went back to TUI Emacs via SSH for work.
  - db48x 6y ago
    TRAMP doesn't open any network connections at all. Instead it relies on ssh (or ftp if you're really old-school). Open up your ~/.ssh/config file and set "ControlMaster yes", "ControlPersist yes" and "ControlPath ~/.ssh/cp-%C" and ssh will multiplex all of your ssh sessions across as few unique tcp connections as possible. This will speed up scp and sftp connections as well.
    - mpereira 6y ago
      I do that. Performance is still pretty bad compared to VSCode remote or TUI Emacs via SSH.
humanistbot 6y ago
When I was first getting into programming and computer science as a student, I was so into these kinds of super-customizable rabbit holes. My whole life pretty much centered around the keyboard and terminal. I was all in on knowing everything about the entire stack and making it do whatever it was I wanted. I'd spend hours to save a few keystrokes. That wasn't so much about the productivity trade-off as it was about an almost ideological drive I had to make the machine do exactly whatever it was I wanted. Nothing commercial off the shelf, only things I could customize down to the source code. A couple decades later, I just find it all so exhausting.
- ernst_klim 6y ago
  > I just find it all so exhausting. So do I, and yet I use Emacs. I think there is a middle-ground. There are things worth customizing and improving, in particular the tools I work with, and there are things that I expect to "Just work"™. Hence I use Gnome and Emacs for important workflow, such as programming, TeX, email, git, shell etc. It's and IDE, if you wish, but with more stuff since email is also part of the workflow.
- brundolf 6y ago
  I've gone through a similar transition with technology. I think the key is to separate the yak-shaving you do for fun, from the yak-shaving you actually need to do for productivity. Minimize the latter, and admit that the former really is just for fun, and don't do any more of it than you actually feel like doing (which can be zero).
- Zelphyr 6y ago
  This describes why I have had a Mac for the last 20 years. I used to build my own machines (before laptops really became commonplace, of course) and then install Linux. I loved the learning experience and the almost total control over my system that such an exercise provided. After awhile I realized I just wanted to write code and not have maintaining my system take time away from that. The fact that the Mac provided easy access to Unix was a major plus. Also, at the time, none of the Linux desktop environments could match the polish that the OS X GUI provided. I've been saddened and frustrated by the state of macOS of late (hopefully Big Sur is better) but I'm still not ready to go back to the old days. As you say; it just sounds exhausting.
opan 6y ago
>I'm not aware of similar functionality in Vim, Vim can open remote files by putting scp:// at the start. Check the vim wiki or similar for exact syntax. Works with your ssh config aliases and keys and such, so can be pretty smooth. What I don't know is if he knows of this feature and declares it not-equivalent, or if he's never seen it. TRAMP is likely more powerful, in part due to emacs itself. Like, you can view a remote dir in dired and open an image and see it inside your emacs. I don't know if gvim can view images, I only use regular vim or neovim, but I typically use a graphical emacs. Mostly unrelated, you can open a web url in neovim like it's a file. I've done this to read a page with a bad layout before. A bit like w3m, but then you can easily delete chunks or save parts to another file if you want. You are just seeing the raw html, but usually the paragraph tags are fairly readable.
- coldtea 6y ago
  >What I don't know is if he knows of this feature and declares it not-equivalent, or if he's never seen it. I know it, but it's not even close. Almost every editor can do something like that over SCP/SSH/SFTP.
oblio 6y ago
Cute. If I'd have to guess, that's at least 200 hours put into learning Emacs, probably more.
- guenthert 6y ago
  200h is a lot if you think of Emacs as a tool to turn keystrokes into source code of your current project. I don't think it's a lot if you think of Emacs as a tool to help realize the ideas of your lifetime.
  - oblio 6y ago
    Summarizing: everything you do takes time and this time has an associated opportunity cost. If you can configure another tool in 2-5-10 hours to achieve 90% of the efficiency Emacs gives you and you can use the other 190-195-198 hours to actually realize those "ideas of your lifetime", it is a lot of time spent (and maybe wasted). Also: https://xkcd.com/1205/ https://xkcd.com/1205/ + tinkering isn't necessarily productivity. If anything, it's more often procrastination. Finally, I'm not sure how you could prove that Emacs it the only thing that allows you to achieve those "ideas of your lifetime", I don't think that's even a falsifiable statement, it's just your personal opinion.
    - BeetleB 6y ago
      > If you can configure another tool in 2-5-10 hours to achieve 90% of the efficiency Emacs gives you There's no way any other program will give you 90% of what Emacs does in a mere 10 hours. Not. A. Chance. See this list of things people do in Emacs from an earlier comment. https://news.ycombinator.com/item?id=25627763 https://news.ycombinator.com/item?id=25627763 These are what the author mentioned. From my own usage, I can easily make the list 3x longer. Emacs is an application platform for Elisp apps. Your statement is the equivalent of saying "Just spend 10 hours learning VBA and you'll get 90% of the benefits you got from years of practicing C."
    - alpaca128 6y ago
      You take that linked xkcd far too seriously. Life is not about maximising productivity. We can also save time by not caring about the color of our clothes, getting cheap fast food instead of preparing our own meals and so on, yet in the end most people don't do that because the difference is more than hours of their life. I got better ergonomics, fewer distractions and higher comfort out of my customisation. Yes, I sunk many hours to get to that point. But on the way I learned many fascinating concepts, bash scripting, a deeper understanding of software and computers and more. Without this kind of tinkering I wouldn't even have started programming in the first place.
mark_l_watson 6y ago
Nice write up, thanks. I often mosh + tmux on remote servers slso, and even though I do nothing fancy, with Emacs it all works for me.
- mpereira 6y ago
  I've been doing that too recently, for work. The performance of TUI Emacs via SSH is much better than Tramp.
helmut_hed 6y ago
This really captures the power of Emacs for people with Lisp skills. For coders with strong habits of high-level thinking it's a real force multiplier. Why shouldn't we apply the same analysis and effort to our productivity tools we do to our "output"?

262 points by mpereira 6y ago

Articles like this really trigger the imposter syndrome for me. I just have no patience for these type of things, and seeing others' both ability and patience to power through issues like this one, I just feel like an imposter. But these types of issues are why I've always given up on learning Emacs. When I can just install Visual Studio Code and the Remote - SSH extension and whatever else extension that I want to work remotely and get on with life, it's really hard to justify the time one needs to spend with Emacs to bend it to your will.

PaulDavisThe1st 6y ago
You appear utterly convinced that VSCode will never demonstrate an obscure bug like the one discussed in TFA, or alternatively that if it does you'll be able to jump ship to some other tool. I don't share your confidence, and would prefer to know that such issues can be fixed with or without vendor participation.
- macintux 6y ago
  That seems an ungenerous and quite broad interpretation of a fairly simple assertion. Even as much as I love Emacs, I would never fault someone for choosing a more productive tool now even if there’s a non-zero risk of something breaking in the future...because there’s always something that will break on any platform. Time spent being productive now is usually worth that risk.
  - PaulDavisThe1st 6y ago
    The counter-point to that is that the world benefits more by someone choosing to actually fix whatever was wrong, rather than just switching platforms (or never choosing it in the first place because they came across the post on HN). The claim in the comment I was responding to appeared to me to be "hmm, Emacs has a problem, VSCode probably doesn't, I would rather use VSCode". It didn't seem to me to be written with much awareness of what the actual problem in TFA really was, nor with any real knowledge that VSCode actually would be more productive. It seemed to me to be more like "Ewww! Emacs has a bug, someone fixed it with a lot of obscure stuff, no thanks, VSCode here I come". I certainly respect that for many, VSCode will be a more productive platform when measured by the metrics that matter to them.
    - forrestthewoods 6y ago
      > the world benefits more by someone choosing to actually fix whatever was wrong Maybe. I imagine at some point someone wrote ffap-guess-file-name-at-point to fix a problem. And the blog author solved a problem by disabling it without knowing what problem it maybe solved. Problems eventually become complicated enough it’s not a binary fixed/broken, but rather a matter of trade-offs. Open source and shared solutions are great. But they aren’t a silver bullet either.
    - bmitc 6y ago
      You shouldn't make so many assumptions. If you're confused, ask a clarifying question instead of going off of incorrect assumptions.
- bmitc 6y ago
  There were two points in my comment. One was of feeling like an imposter, in that even as a software engineer I lack both the patience and skill to pull the debugging off the article showcased. It's both admiration and lament at the same time. The second point was that I personally prefer to altogether avoid issues like this if I can by selecting what I feel is the best available tool at the time. I have tried to use Emacs multiple times, and while this exact issue is not one I have encountered (I did say "these types of issues"), I have come across similar issues, especially with having Emacs talk to another computer or VM. Every time I have tried to get into Emacs, I end up hitting some barrier akin to these things, and so I resign myself to using Visual Studio Code, where then I get off to coding rather than trying to bend the environment to what I'd like. And it always seems that with Visual Studio Code it both does what I'd like and works without a million gotchas. Visual Studio Code is not perfect, and I have had issues with it and expect more issues. However, they don't hit me as early as they seem to with Emacs, and that's an important distinction. The remoting extension is extremely seamless, and I have not seen an Emacs example of matching anything close to it. It really comes down to personality. If I was a carpenter and had a broken hammer, I wouldn't go off rebuilding the hammer or designing one from scratch. I'd do my best to find someone else who's already built a hammer that works. That doesn't imply I never expect the hammer to never break. > would prefer to know that such issues can be fixed with or without vendor participation Visual Studio Code is open source and well-managed. I'd prefer that a solution be merged into the tool for everyone, rather than being buried in someone's .emacs file.
  - grzm 6y ago
    > "the use of TFA" Read it as the fine article or the featured article.
  - PaulDavisThe1st 6y ago
    Your explanation involving containers/VMs makes a lot more sense to me than avoiding specific bugs when using Emacs' equivalent to VSCode's remoting extension. As a purely native developer, I have zero use for such things, but I can certainly appreciate that VSCode may have done a much better (or certainly, shall we say, a less historically indebted) version of this sort of functionality than Emacs. [ Edit: and yes, the featured article ]
grzm 6y ago
I think it's great that we have such a variety of editors and IDEs to choose from and find one that meets best meets our individual needs. Over the years I've used BBEdit, TextMate, IntelliJ, and Emacs for development. For the past couple of years it's been switching between the last two. I'm glad you've found VS Code to be a match for you. I've spent time configuring all of them to aid in development. ust like there's more than one hammer on the market, or more than one drill, each has its strengths and weaknesses. There have definitely been frustrating times configuring Emacs, but one of the reasons it's paid off over the years is that when I run into an issue, I can always dig into it and potentially fix it myself; and when I want to extend it to do sometime it can't do yet, I can do that, too. It's an investment, but it's one that I think has paid off, and will continue to pay off. Emacs has been applicable in more situations and environments than those other tools. (IntelliJ's tooling has been useful, but I've also run into bugs that have impeded my work, which I can't fix on my own because the source isn't open.) I also do a lot of work in Clojure, so Emacs Lisp isn't much of a jump. Admittedly this might not be the case for everyone. I think another big reason I've stuck with Emacs is that the cost of switching editors as they come along (I hear Atom is really nice, and people have been talking positively about Sublime for years) has a cost as well. Emacs is long in the tooth, which is a curse (Emacs Lisp is a really old lisp) and a blessing (it's unlikely to go away any time soon, and that time spent learning Emacs Lisp is going to continue to pay off). Please don't read this as apologia for Emacs or as a criticism of your choice, as it's neither. I'm just trying to add some context to how others may justify the time they've spent customizing Emacs.
- grzm 6y ago
  And I'll add that you don't need to limit yourself to just one. I do most of my text editing and development in Emacs, but when I'm working in Java, I'll use IntelliJ. I miss out on some of the key bindings and workflow I'm used to in Emacs, but for me the other features IntelliJ affords me when working with Java are worth it.
  - bmitc 6y ago
    I actually prefer to use the de facto IDEs for most languages, only deferring to something like Visual Studio Code or Emacs when those don’t exist.
legends2k 6y ago
It isn't as complex as it seems. You can choose to be at any point in the spectrum of yak shaving; keep doing it vs skip it altogether. I did the latter with vanilla Emacs. It works just like a regular GUI text editor e.g. Notepad, gedit, etc. Only when I wanted more power I dropped into customizing my .emacs. I balance myself between doing it often vs not doing it at all. Doing the above for 5 years I've a decent personal .emacs I take with me in a Git repo. Setting up takes less than 10 mins on any new machine.

bambataa 6y ago

I always wonder how people evolve from fiddling with package configurations to using emacs at this level. My stunbling block is not so much knowing Elisp (I’m sure I can stumble through that) but just knowing the internal architecture and how emacs works overall. Has anyone come across a good guide for hacking emacs? I know there is the manual. Perhaps I just need to look at that again and get going.

brokenwallet 6y ago
The Elisp reference manual describes Elisp the language, and Emacs's Elisp API: https://www.gnu.org/software/emacs/manual/html_node/elisp/index.html https://www.gnu.org/software/emacs/manual/html_node/elisp/in... e.g. here is describes keymaps: https://www.gnu.org/software/emacs/manual/html_node/elisp/Keymap-Basics.html#Keymap-Basics https://www.gnu.org/software/emacs/manual/html_node/elisp/Ke...
ernst_klim 6y ago
Emacs Lisp Intro and Elisp reference manual, which come with Emacs. Former is concise and more suited for configuration, latter is for package developers.
mpereira 6y ago
> I always wonder how people evolve from fiddling with package configurations to using emacs at this level. One step at a time. If getting better at Elisp is something you want, just keep at it :)

guenthert 6y ago

"I’ve recently joined a company that for security reasons doesn’t allow their source code on laptops. Development happens strictly on workstations inside their private networks, with some using their text editor’s support for remote file editing and others running editors on those machines via SSH." I think using TRAMP or similar technologies is bypassing, if not the letter, then the spirit of that companies' security policy. Sure, not the whole repository is lost when the laptop is, but having an (ephemeral) copy of the files one is working on outside the security perimeter is kind of defeating its purpose. Why is that company granting access to remote file transfer? I'd think they would be better off restricting access to some form of desktop virtualization (the old SunRays were nice) if they must offer remote access.

deleted 6y ago

[deleted]

macintux 6y ago

It’s a shame that most of the discussion is about the first part of the essay, impressive as the troubleshooting is. His discussion of core values is quite interesting, in that it’s both relatively obvious in retrospect yet something I’ve not given conscious thought to. I’ve certainly thought about programming languages as expressions of intent, and how some attempt to be what I call “kitchen sink” languages, like C++, Java, or JavaScript, while others are much more opinionated about their purpose, like Erlang, Forth or APL. Also reminds me of the old description of UNIX as user-friendly, just picky about who its friends are.

mpereira 6y ago
I'm glad you persevered through it. Thanks for reading!
akaralar 6y ago
Exactly my feelings, the first part just shows some good technical skills and flexibility of Emacs but it was the second part that was much more interesting. I too enjoyed the part about values and it gave me some food for thought. I also can tell the author read Sapiens as there are quite a few points that were discussed in the book. Also some of the links in the article are great rabbit holes as well, like the 80000 hours website. A+ writing overall!

mlang23 6y ago

Being able to open remote files with Tramp is convenient. However, whats even cooler is that Tramp can execute remote commands. IOW, if you edit a file in a remote directory, you can also use M-x compile RET to run a remote build process and see the output in your Emacs Compilation buffer. If the paths pritned are relative, you can even use M-x next-error RET to jump to a build error location.

macintux 6y ago
Running remote commands is almost entirely what that part of the article is about.

brokenwallet 6y ago

This is a great post showing effective performance analysis. It's worth noting that Emacs's built-in way to find files (project-find-file) in a project does not suffer these performance issues. Try it, C-x p f. * project-find-file doesn't call expand-file-name for all 70k files (each of which involves a network roundtrip). * project-find-file will use "git ls-files" (see project--vc-list-files) rather than find (or expecting the user to install and enable fd) * project-find-file uses completing-read, which wouldn't by default call Ivy/Ivy prescient. I can see how the author has ended up here: projectile predates the built-in project support, so it must only seem natural to use projectile as a base, and apply performance fixes on top. Emacs's built in functionality tends to work better with other Emacs functionality, hence why project-find-file works well with TRAMP. It's worth trimming your config occasionally, and rebasing onto built-in functionality. The author's fast-project-find-file has a more responsive UI, however. It'd be great to have that in core.

omaranto 6y ago
> * project-find-file uses completing-read, which wouldn't by default call Ivy/Ivy prescient. Not by default, that's true, but for users that have installed ivy and activated ivy-mode, ivy does take over completing-read.
omaranto 6y ago
> It's worth noting that Emacs's built-in way to find files (project-find-file) in a project does not suffer these performance issues. Try it, C-x p f. That is your personal key binding, project-find-file does not seem to have a default key binding. (This, by I which I mean telling people that some built-in command is bound to some key while forgetting that is in my configuration and not by default, has happened to me several times.)
- catern 6y ago
  Actually it's a default key binding in the (yet unreleased) Emacs 28. project.el has a lot of nice improvements in the upcoming version of Emacs. (I imagine the grandparent comment author just checked the version tag in `C-h k C-x p f' to see when it was released, but currently on Emacs HEAD the keybinding doesn't have a proper version tag - that's a big, albeit minor)

tgbugs 6y ago

The author quickly moves on after finding that expand-file-name is the issue and routes around the problem. The last time we hear from it is when it is patched out. But what is really going on? The problem is that many parts of Emacs and libraries built on top of them have not been made fully TRAMP aware. There is an abstraction boundary that is being crossed, assumptions are being violated, etc. because all paths are treated equally, despite the fact that most parts of Emacs can identify that a path is remote (or could be remote). Fixing the underlying issue and providing latency aware variants of commands is not easy (you usually can't just run elisp on a remote machine), but when we get there Emacs will be even more powerful! Here is an excerpt from some notes I made back in October. Consider these cases. I have 10k files that I want to move on a remote server. I have 10k files that I want to move on a local server. Where do I put the for loop? I want to put it so that I only have to call ssh once. Similar issue with using find and subshells. If I have to invoke a subshell there is significant overhead. Having even a little bit of knowledge about the latency of running a single command, or the knowledge that certain commands commute under loop reordering would make it much easier to produce efficient code, basically #+begin_src elisp (do-for-values value '(a b c d e f g) (please-do-this-on-the-remote #'command value)) #+end_src should be rewritten to #+begin_src elisp (please-do-this-on-the-remote (do-for-values value '(a b c d e f g) (command value))) #+end_src However, this is only safe if the values are from the local and not the remote and are not functions that are dependent on local state (that doesn't really matter in strict languages though). The best way to approach this might be to detect and warn before running. Or not, sometimes you do just want to be more efficient, and if you don't want the optimization then disable it some other way. The reason is because sometimes it is much easier to write the latency multiplying version, and the conversion to the latency efficient form is trivial for a computer but extremely easy to screw up for a human being. EDIT: Hah. I though I recognized Richard's reply. It was to me.

mpereira 6y ago
Yeah it's tricky. The reason why packages like Eshell and Magit work pretty much out of the box with Tramp is that they rely on functionality like `expand-file-name`, which will handle both the local and remote cases. It's convenient, but as you said, "assumptions are being violated". > Hah. I though I recognized Richard's reply. It was to me. Thanks for being the cause of that really nice reply :)

clircle 6y ago

I do not use projectile, but I do use rg via counsel-rg in Emacs. I wonder how that would compare speed-wise.

justaj 6y ago

> TUI Emacs has the terminal emulator (and also commonly tmux) limiting and conflicting with the clipboard, keybindings, colors, and more. None of these limitations are present in GUI Emacs. Is it me, or is this telling me that the terminal Emacs experience is just severely crippled compared to the GUI one?

kagevf 6y ago
sorry for the late reply ... I have been using emacs for org mode and common lisp coding for about 1 year ... In my (very limited) experience I've noticed these differences: - TUI can't display images - Some keystroke combinations are "absorbed" by the terminal so are not available. I believe they are shift+alt key combinations. A little annoying, but not a huge deal-breaker. ... and that's all I've encountered so far. I don't think TUI is severely crippled compared to the GUI version, but the GUI version is nicer to use.

abhinav22 6y ago

Wowsers! That was a long read, I skimmed through a bit of it. Kudos for dedicating time and effort to high quality long form articles - this is something that online needs to revert to. That said, can you give us a TL;DR if you have a moment :D

lproven 6y ago
Can it be summarised? No. Can it be described? Well, this is how I did it in a tweet, and the author like it and thanked me. « Fascinating, very long essay which starts with a discussion of text editors, wanders through software development models & the issues of modernising very old software, & turns into a discussion of intelligence, language & computing. »

radarsat1 6y ago

I use emacs but tend to just find another way to do things (e.g. grep, & other shell things) when I run into a stumbling block like this; believe it or not, not everyone wants to start debugging their text editor when they run into trouble. Admittedly I wish I would spend a bit of time in emacs internals sometimes just so I could learn to approach similar issues, but it's just never much of a priority when I'm trying to work on what I'm actually working on. I rarely get distracted by trying to optimize my config, as a result I use almost the vanilla defaults and I'm mostly happy with that. TRAMP is pretty great, I wish it would not make a new connection for every network interaction though. Setting ControlMaster on ssh helps a bit but it's still not as fast/transparent as I'd like. There are far too many "blocking" operations in emacs, as much as I love working in it, there is the non-zero times that it locks up on me and it drives me crazy when that happens. If it was ever the case that I had to wait 30 seconds to open a file, I'm sure I would abandon emacs instead of trying to debug it. That said I can't imagine working on a 70,000-file repo. That sounds like a terrible idea.

mpereira 6y ago
> TRAMP is pretty great, I wish it would not make a new connection for every network interaction though Same. One of my wishes for Emacs in "From caching up to getting ahead" is for Tramp improvements. These days I went back to TUI Emacs via SSH for work.
- db48x 6y ago
  TRAMP doesn't open any network connections at all. Instead it relies on ssh (or ftp if you're really old-school). Open up your ~/.ssh/config file and set "ControlMaster yes", "ControlPersist yes" and "ControlPath ~/.ssh/cp-%C" and ssh will multiplex all of your ssh sessions across as few unique tcp connections as possible. This will speed up scp and sftp connections as well.
  - mpereira 6y ago
    I do that. Performance is still pretty bad compared to VSCode remote or TUI Emacs via SSH.

humanistbot 6y ago

When I was first getting into programming and computer science as a student, I was so into these kinds of super-customizable rabbit holes. My whole life pretty much centered around the keyboard and terminal. I was all in on knowing everything about the entire stack and making it do whatever it was I wanted. I'd spend hours to save a few keystrokes. That wasn't so much about the productivity trade-off as it was about an almost ideological drive I had to make the machine do exactly whatever it was I wanted. Nothing commercial off the shelf, only things I could customize down to the source code. A couple decades later, I just find it all so exhausting.

ernst_klim 6y ago
> I just find it all so exhausting. So do I, and yet I use Emacs. I think there is a middle-ground. There are things worth customizing and improving, in particular the tools I work with, and there are things that I expect to "Just work"™. Hence I use Gnome and Emacs for important workflow, such as programming, TeX, email, git, shell etc. It's and IDE, if you wish, but with more stuff since email is also part of the workflow.
brundolf 6y ago
I've gone through a similar transition with technology. I think the key is to separate the yak-shaving you do for fun, from the yak-shaving you actually need to do for productivity. Minimize the latter, and admit that the former really is just for fun, and don't do any more of it than you actually feel like doing (which can be zero).
Zelphyr 6y ago
This describes why I have had a Mac for the last 20 years. I used to build my own machines (before laptops really became commonplace, of course) and then install Linux. I loved the learning experience and the almost total control over my system that such an exercise provided. After awhile I realized I just wanted to write code and not have maintaining my system take time away from that. The fact that the Mac provided easy access to Unix was a major plus. Also, at the time, none of the Linux desktop environments could match the polish that the OS X GUI provided. I've been saddened and frustrated by the state of macOS of late (hopefully Big Sur is better) but I'm still not ready to go back to the old days. As you say; it just sounds exhausting.

opan 6y ago

>I'm not aware of similar functionality in Vim, Vim can open remote files by putting scp:// at the start. Check the vim wiki or similar for exact syntax. Works with your ssh config aliases and keys and such, so can be pretty smooth. What I don't know is if he knows of this feature and declares it not-equivalent, or if he's never seen it. TRAMP is likely more powerful, in part due to emacs itself. Like, you can view a remote dir in dired and open an image and see it inside your emacs. I don't know if gvim can view images, I only use regular vim or neovim, but I typically use a graphical emacs. Mostly unrelated, you can open a web url in neovim like it's a file. I've done this to read a page with a bad layout before. A bit like w3m, but then you can easily delete chunks or save parts to another file if you want. You are just seeing the raw html, but usually the paragraph tags are fairly readable.

coldtea 6y ago
>What I don't know is if he knows of this feature and declares it not-equivalent, or if he's never seen it. I know it, but it's not even close. Almost every editor can do something like that over SCP/SSH/SFTP.

oblio 6y ago

Cute. If I'd have to guess, that's at least 200 hours put into learning Emacs, probably more.

guenthert 6y ago
200h is a lot if you think of Emacs as a tool to turn keystrokes into source code of your current project. I don't think it's a lot if you think of Emacs as a tool to help realize the ideas of your lifetime.
- oblio 6y ago
  Summarizing: everything you do takes time and this time has an associated opportunity cost. If you can configure another tool in 2-5-10 hours to achieve 90% of the efficiency Emacs gives you and you can use the other 190-195-198 hours to actually realize those "ideas of your lifetime", it is a lot of time spent (and maybe wasted). Also: https://xkcd.com/1205/ https://xkcd.com/1205/ + tinkering isn't necessarily productivity. If anything, it's more often procrastination. Finally, I'm not sure how you could prove that Emacs it the only thing that allows you to achieve those "ideas of your lifetime", I don't think that's even a falsifiable statement, it's just your personal opinion.
  - BeetleB 6y ago
    > If you can configure another tool in 2-5-10 hours to achieve 90% of the efficiency Emacs gives you There's no way any other program will give you 90% of what Emacs does in a mere 10 hours. Not. A. Chance. See this list of things people do in Emacs from an earlier comment. https://news.ycombinator.com/item?id=25627763 https://news.ycombinator.com/item?id=25627763 These are what the author mentioned. From my own usage, I can easily make the list 3x longer. Emacs is an application platform for Elisp apps. Your statement is the equivalent of saying "Just spend 10 hours learning VBA and you'll get 90% of the benefits you got from years of practicing C."
  - alpaca128 6y ago
    You take that linked xkcd far too seriously. Life is not about maximising productivity. We can also save time by not caring about the color of our clothes, getting cheap fast food instead of preparing our own meals and so on, yet in the end most people don't do that because the difference is more than hours of their life. I got better ergonomics, fewer distractions and higher comfort out of my customisation. Yes, I sunk many hours to get to that point. But on the way I learned many fascinating concepts, bash scripting, a deeper understanding of software and computers and more. Without this kind of tinkering I wouldn't even have started programming in the first place.

mark_l_watson 6y ago

Nice write up, thanks. I often mosh + tmux on remote servers slso, and even though I do nothing fancy, with Emacs it all works for me.

mpereira 6y ago
I've been doing that too recently, for work. The performance of TUI Emacs via SSH is much better than Tramp.

helmut_hed 6y ago

This really captures the power of Emacs for people with Lisp skills. For coders with strong habits of high-level thinking it's a real force multiplier. Why shouldn't we apply the same analysis and effort to our productivity tools we do to our "output"?