3 ms·
Some of these exist, and all are possible without threads. Non-blocking save on network drives; Non-blocking autosave This requires a memcpy and the aio_* fun
by jrockway 15y ago
Some of these exist, and all are possible without threads.
Non-blocking save on network drives; Non-blocking autosave
This requires a memcpy and the aio_* functions. When a save is requested, you make a copy of the buffer in memory (so that what ends up on disk is what the buffer looked like exactly when the save was requested), then you stream this data to disk with aio_write (perhaps making the write atomic by doing a synchronous rename after all the data is on disk). This would probably take no more than a few hours to implement, and most of the complexity is simple to handle with atomic writes and aio_cancel. I think the biggest problem is handling the "userspace" semantics; after-save hooks expect to run after the data is on disk but before the user has changed the buffer in any way. What do you do about this?
(I've done this before. I have an Emacs mode at work that saves to a special centralized database in addition to the local disk at every modification. We just queue the write in before-save hook and tell some other process to read the data from a temp file that we create. It works perfectly, even on Windows!)
Rebuilding tag tables in the background
This is easy and eproject already does it. You run etags in a background process and when it's done, point Emacs at the new tags tables. Since the tags tables don't do anything until a lookup is requested, this is completely asynchronous.
Try it today:
https://github.com/jrockway/eproject/blob/master/contrib/eproject-tags.el https://github.com/jrockway/eproject/blob/master/contrib/epr...
Loading and fontifying large files in the background (i.e. concurrent fontification, not just jit-lock)
This you can't do unless you can guarantee that matchers are bounded and generate correct results no matter what the boundaries are. I don't know of any programming mode that could be made to meet this invariant easily; consider the case:
/* this code is bad:
int i = 0; */
If we had two threads fontifying this block, and the bounds were set to thread 1 = line 1, thread 2 = line 2, the results are going to be wrong. The first line will be treated as a comment and the second line will look like a normal variable declaration because getting the correct syntax information requires analyzing the entire first line. Any language with multiline constructs (string literals, preprocessor directives, comments, etc.) will have this problem, thus making parallelization Not Easy. If font-locking is slow, profile it and fix the algorithm, but don't expect speedups for free. Parsing programs that are invalid (as most are while being typed; you type characters, not tokens) and guessing what the user "means" is hard.
Concurrent url fetching and downloads (e.g. updating el-get without blocking).
Fork curl or wget. Display the result when done. Or, implement HTTP from scratch; network IO is already nonblocking.
Concurrent spellcheck
This is already implemented this way; a pipe is opened to aspell and words are piped to aspell as necessary. If parsing the results is too expensive and you don't care about interactivity, wait for bigger batches of data back from aspell. All IO to other processes is asynchronous and can be made synchronous with special calls like accept-process-output.
Fast scrolling in large buffers (especially complex XML files opened in nxml-mode)
How will threads solve this problem?
- nyellin 15y agoThank you, especially for your notes on aio_. As promised, I sent you something by email, but know that your comment is worth more than that. Fast scrolling in large buffers (especially complex XML files opened in nxml-mode) I think nxml-where.el is guilty for the slow scrolling in large xml files. I wanted to offload that to another thread, but a simpler solution is to only update the path post-scroll.
- jrockway 15y agoThanks! It's also worth pointing out that Emacs and Emacs Lisp need not be multithreaded in order to implement specific asynchronous operations. If aio did not exist, there's nothing stopping you from creating a thread (from C), doing your blocking save there (and nothing else), and sending a note back to the lisp side when it's done. It's generic threading that adds complexity, because that would require all state-changing operations to be threadsafe, which can be very difficult to "bolt on" to a complex system like Emacs. (Not to mention locking overhead; see Python's GIL debate.) A good example of this approach is the 0MQ messaging library. While it is internally threaded, it does not care what the application does about thread safety; there is no way that you can ever "see" its threads. It simply requires that you pass ownership of blocks of memory to the library, thus avoiding locking and issues of thread safety while still using threads.