3 ms·

I honestly don't think IO-blocking-UI is such a big issue. And if you try to solve it using workers you introduce an annoying programming paradigm of async/awa

I honestly don't think IO-blocking-UI is such a big issue. And if you try to solve it using workers you introduce an annoying programming paradigm of async/await and IPC. Copying data over IPC can be slow, and having to set up IPC to share the data you need to run in a worker is also annoying. The great thing about the single threaded shared memory model of elisp is that it's a lot easier to code against. And yeah sometimes someone blocks the UI doing synchronous I/O, but that comes at the benefit of making it really easy for people to write addons that do all sorts of things. A lot of the I/O blocking UI problems could be alleviated by just moving to async I/O primitives without incurring the runtime and developer experience problems of a multi process architecture

danShumway 3y ago
Hrm. I don't know if I agree with this or not. Your last sentence is 100% spot on. But at the same time, I've used Emacs as a window manager before, and the reason I stopped was because of IO blocking the UI. I could not handle having a process bug out and having it freeze my entire window manager, and it happened frequently. I don't want to dismiss the argument; maybe it's just that Emacs packages are bad at avoiding that behavior? But I'm not sure that async I/O primitives are enough for a text editor, even though I would generally agree with you. On the subject of that last sentence: > A lot of the I/O blocking UI problems could be alleviated by just moving to async I/O primitives without incurring the runtime and developer experience problems of a multi process architecture This sounds like an advertisement for Javascript to me? This is one of the reasons I think why NodeJS became so popular when it was first introduced, literally because it was single-threaded with good async I/O primitives (or good enough, this was before Promises were in the language), and that led to huge performance improvements over other server implementations that prioritized workers. So while I still don't like the kitchen sink approach, I'm still left feeling like a lot of the problems and solutions being proposed here are kind of things that would be addressed by using Deno in Emacs? At least, I'm seeing the appeal. Single-threaded memory pool to help make developer experience better and to improve performance, coupled with decent async I/O primitives -- that's Javascript, that's one of the things the language is sort of good at. You have workers for larger tasks, and that's definitely handy, but most of your programming is happening in a single-threaded event loop using promises. If I looked at this sentence in isolation, I would think that you were arguing in favor of adding Deno.
- samus 3y ago
  Adding a new language still doesn't solve the blocking problems with the current extensions. It still seems way less complicated to just patch the extensions that are not behaving well. If their maintainers don't want to cooperate, there is always the possibility to fork these extensions, but that's also way easier than writing these from scratch or treating them in Typescript.
  - danShumway 3y ago
    > Adding a new language still doesn't solve the blocking problems with the current extensions. That is a pretty strong argument which I agree with. The somewhat dismissive answer I could give around JS in Emacs is that if I wanted to write JS so badly that I was willing to rewrite the ecosystem in order to do it, I'm not sure why I shouldn't just use VS Code. I'm floating the idea that JS would be a good fit for a lot of these problems, but it is correct to ground that with the idea that having 2 separate coding languages in a text editor just feels weird and cumbersome and I don't know, isn't the ecosystem fragmented enough? I can't actually avoid Elisp in Emacs and I wouldn't be able to just because Javascript was added.
chlorion 3y ago
I think having a single-threaded async/await style interface for doing non-blocking things like creating and reading output from subprocesses could be pretty nice. As far as I can tell Emacs already supports the primitives required to implement an async/await style interface. The way it works now, is that the editor is one big event loop. The event loop handles reading keyboard input and eval'ing the elisp functions bound to those keys, and while that's not happening, non-blocking things like subprocesses are being progressed. This happens outside of elisp, inside of the C part of Emacs somewhere. Right now subprocesses support using custom "sentinels", these are an elisp function that gets called by the event loop whenever the subprocess progresses in some way. I'm not really a concurrency expert but I think you could work these already existing things into a nice interface. Having built in support for this would be a huge upgrade! There are libraries like emacs-aio that try to do this, but I don't know very much about them right now, but that should at the least serve as a proof of concept.
- kazinator 3y ago
  > async/await style interface for doing non-blocking things like creating and reading output from subprocesses could be pretty nice. That's literally fork/wait + pipes.
  - chlorion 3y ago
    The point is to have a nicer API on top of these primitives, not to introduce new features into Emacs. async/await APIs are just a fancy abstraction over an event loop and callbacks and whatnot. Even just having "promises" would be a nice upgrade. Maybe it's just me but I find the current functions for doing non-blocking stuff very clunky and awkward.

danShumway 3y ago

Hrm. I don't know if I agree with this or not. Your last sentence is 100% spot on. But at the same time, I've used Emacs as a window manager before, and the reason I stopped was because of IO blocking the UI. I could not handle having a process bug out and having it freeze my entire window manager, and it happened frequently. I don't want to dismiss the argument; maybe it's just that Emacs packages are bad at avoiding that behavior? But I'm not sure that async I/O primitives are enough for a text editor, even though I would generally agree with you. On the subject of that last sentence: > A lot of the I/O blocking UI problems could be alleviated by just moving to async I/O primitives without incurring the runtime and developer experience problems of a multi process architecture This sounds like an advertisement for Javascript to me? This is one of the reasons I think why NodeJS became so popular when it was first introduced, literally because it was single-threaded with good async I/O primitives (or good enough, this was before Promises were in the language), and that led to huge performance improvements over other server implementations that prioritized workers. So while I still don't like the kitchen sink approach, I'm still left feeling like a lot of the problems and solutions being proposed here are kind of things that would be addressed by using Deno in Emacs? At least, I'm seeing the appeal. Single-threaded memory pool to help make developer experience better and to improve performance, coupled with decent async I/O primitives -- that's Javascript, that's one of the things the language is sort of good at. You have workers for larger tasks, and that's definitely handy, but most of your programming is happening in a single-threaded event loop using promises. If I looked at this sentence in isolation, I would think that you were arguing in favor of adding Deno.

samus 3y ago
Adding a new language still doesn't solve the blocking problems with the current extensions. It still seems way less complicated to just patch the extensions that are not behaving well. If their maintainers don't want to cooperate, there is always the possibility to fork these extensions, but that's also way easier than writing these from scratch or treating them in Typescript.
- danShumway 3y ago
  > Adding a new language still doesn't solve the blocking problems with the current extensions. That is a pretty strong argument which I agree with. The somewhat dismissive answer I could give around JS in Emacs is that if I wanted to write JS so badly that I was willing to rewrite the ecosystem in order to do it, I'm not sure why I shouldn't just use VS Code. I'm floating the idea that JS would be a good fit for a lot of these problems, but it is correct to ground that with the idea that having 2 separate coding languages in a text editor just feels weird and cumbersome and I don't know, isn't the ecosystem fragmented enough? I can't actually avoid Elisp in Emacs and I wouldn't be able to just because Javascript was added.

chlorion 3y ago

I think having a single-threaded async/await style interface for doing non-blocking things like creating and reading output from subprocesses could be pretty nice. As far as I can tell Emacs already supports the primitives required to implement an async/await style interface. The way it works now, is that the editor is one big event loop. The event loop handles reading keyboard input and eval'ing the elisp functions bound to those keys, and while that's not happening, non-blocking things like subprocesses are being progressed. This happens outside of elisp, inside of the C part of Emacs somewhere. Right now subprocesses support using custom "sentinels", these are an elisp function that gets called by the event loop whenever the subprocess progresses in some way. I'm not really a concurrency expert but I think you could work these already existing things into a nice interface. Having built in support for this would be a huge upgrade! There are libraries like emacs-aio that try to do this, but I don't know very much about them right now, but that should at the least serve as a proof of concept.

kazinator 3y ago
> async/await style interface for doing non-blocking things like creating and reading output from subprocesses could be pretty nice. That's literally fork/wait + pipes.
- chlorion 3y ago
  The point is to have a nicer API on top of these primitives, not to introduce new features into Emacs. async/await APIs are just a fancy abstraction over an event loop and callbacks and whatnot. Even just having "promises" would be a nice upgrade. Maybe it's just me but I find the current functions for doing non-blocking stuff very clunky and awkward.