13 ms·

Continued progress porting Emacs to Rust

163 points by ealhad 8y ago

nazri1 8y ago
DEFUN ("atan", Fatan, Satan, 1, 2, 0, Hah I couldn't read that with a straight face!
- db48x 8y ago
  Agreed! It's a great example.
- martinlofgren 8y ago
  That's beautiful!
OskarS 8y ago
When I first heard of this project I thought "Well, that's insane, it will never be finished, and will surely be abandoned in six months". Glad to hear that it's making good progress! I love it when I'm wrong about stuff.
- bjornedstrom 8y ago
  Remember that most of Emacs is written in Emacs Lisp. If you take the entire source code distribution, around 15% of it is written in C. Those 15% are of course fairly complex, however it's at least less code than you may think.
  - db48x 8y ago
    Yes. The last time I looked, it was about a million lines of elisp, and only about a hundred thousand of C. We'll have it cleaned up in no time.
- mschaef 8y ago
  Agreed... this seemed like it might run afoul of CADT [1], and I'm glad it hasn't. 1] https://www.jwz.org/doc/cadt.html https://www.jwz.org/doc/cadt.html
ealhad 8y ago
To anyone wondering why this might be a good idea, there are some interesting comments in a GitHub issue (https://github.com/Wilfred/remacs/issues/305 https://github.com/Wilfred/remacs/issues/305): > For users, I hope Remacs will be faster (it's easier to optimise), more robust (we have stronger type checks and the ability to add unit tests), better documented (see #262) and somewhat more featureful (we have some ideas for solving the dumper problem that would allow users to dump their instance as an image) than GNU Emacs. > I think we will get problems when the current generation of emacs devs retire and there's a better chance that emacs will survive if we try to make it more appealing to young developers. Rust seems to be a good option.
- flurrything 8y ago
  > For users, I hope Remacs will be faster The main reason my emacs is painfully slow is that most elisp code blocks. Stuff like hitting tab for auto-completion might perform a blocking query from a language server hanging emacs for multiple seconds. Opening a file tries to start a language server and blocks the editor in the process. Etc. Making emacs itself faster won't make the experience of working with emacs any better.
  - simias 8y ago
    Some things like indenting a large file or even searching in a huge buffer can be quite slow. For instance I just tried indenting a ~1000line C buffer and it took about 2 seconds. When dealing with very large files (hundreds of MB) some basic movement and editing commands can be quite slow. Dealing with very large TAGS file (for the linux kernel for instance) is annoyingly laggy as well. I agree that it's not the panacea though, Emacs would benefit from using a more threaded or event-oriented programming model for many things.
  - ams6110 8y ago
    Emacs does block on many operations, but I've never had it be really noticeable on tab-completion type stuff. Though I only use auto-complete on emacs commands, so maybe I've just missed it.
wscott 8y ago
And there are related efforts to upgrade the lisp used in emacs to a more modern and standardized version. I think it is GuileEmacs I am remembering. I hope to two efforts are compatible.
- nerdponx 8y ago
  My thinking as well. This is a golden opportunity to replace Elisp. Edit: Guile won't be used, because of the burden of contributing. https://github.com/Wilfred/remacs/issues/66#issuecomment-273333925 https://github.com/Wilfred/remacs/issues/66#issuecomment-273...
  - rekado 8y ago
    This is very unfortunate. Emacs itself also requires copyright assignment, though, so if Remacs doesn't then it's virtually impossible for GNU to switch code bases (not that it would have been likely anyway).
- simias 8y ago
  That's the first thing that crossed my mind when I saw this project but I think it might be a mistake. Convincing people (and contributors) to ditch C-emacs for "remacs" will be hard, convincing people to ditch elisp-emacs for guilemacs will be hard but doing both at the same time might be just too much. If remacs manages to reach the point where you can use it to replace emacs in-place without any breakage while bringing better performance and more stability it will be a tremendous success IMO.
- rauhl 8y ago
  Although Scheme has a standard, I wouldn’t really call Guile ‘more modern and standardised’: Guile has no standard (it extends the Scheme standard in incompatible ways), and ‘modern’ is hard to define. Emacs Lisp added lexical scope in the past couple of years, probably more recently than Guile added a new language (as opposed to library) feature — is that modern enough? I wouldn’t imagine that the two efforts are compatible, because they are both efforts to rewrite the Emacs Lisp engine in another language, one in Rust & one in Guile Scheme. It’s possible that one might be able to call over to the other, through its CFFI. I’d like to see Emacs rewritten in Common Lisp, and so I’d prefer a Rust engine to a Guile engine: Rust is a static language, and a Rust engine is unlikely to lead to Rust-extensible Emacs code, while a Guile engine is likelier to lead to Guile-extensible Emacs code, which is completely the wrong direction IMHO.
mrec 8y ago
Not an Emacs user, so forgive me if I'm being dense, but is this (from the README) really true? Emacs will change how you think about programming. Emacs is an incremental programming environment. There's no edit-compile-run cycle. There isn't even an edit-run cycle. You can execute snippets of code and gradually turn them into a finished project. There's no distinction between your editor and your interpreter. I'm assuming what it means is "Emacs will change how you think about programming Emacs Lisp". Which is a rather narrower proposition, and much less of an incentive to switch, since I doubt many non-Emacs users are particularly interested in programming Emacs Lisp.
- bachmeier 8y ago
  My interpretation is that it's not a statement about writing code but running it. It's accurate IMO but probably not terribly important in the grand scheme of things. > I doubt many non-Emacs users are particularly interested in programming Emacs Lisp By definition, if you're programming Emacs Lisp, you're an Emacs user. Emacs is the interpreter.
  - serhart 8y ago
    > By definition, if you're programming Emacs Lisp, you're an Emacs user. Emacs is the interpreter. That's not necessarily true. Guile supports Elisp as a language. You can write Elisp sans Emacs.
- another-cuppa 8y ago
  It is true. Learning any Lisp changes how you think about programming. Learning Emacs Lisp will make you wish all development was like that.
- tmountain 8y ago
  Clojure + Cider inside of "Spacemacs" is the most enjoyable development environment I've used in over 20+ years of programming. It's amazing how quickly you can get stuff done. Lisp changes how you think about programming. Emacs gives you the shortest path to a code/test feedback loop.
- 8y ago
ams6110 8y ago
Emacs rather famously has a build process that involves first compiling a minimal elisp environment (written in C and maybe some assembly) that then loads all the editing routines (which are written in elisp) and then dumps itself out as a binary (to save the load time, which is substantial, on subsequent start-ups). I assume the Rust-based emacs would do the same thing?
- wiz21c 8y ago
  not sure, but I guess that process is to make sure Emacs can be built from scratch, ie without relying on proprietary software.
  - db48x 8y ago
    The build process of Emacs doesn't rely on any proprietary software. But it is quite complex :)
- db48x 8y ago
  We still use the same unexec process as normal Lisp, not because we really like it but because it's easier to make small changes. It really is a defect attractor though (I even introduced one a few months back, and had to debug it), so we do want to simplify it.
tom_mellior 8y ago
The author misunderstands some parts of the C language, in particular related to static variables. "In fact, these aren't just file globals accessible to only one translation unit, these are static variables that are accessible across the whole program." gets the meaning of C's static exactly backwards: C file-scope static variables are only accessible (by name) in their translation unit; all other file-scope variables are extern by default. More importantly, they write: static struct Lisp_Objfwd o_fwd; "... creates an (uninitialized) static variable called o_fwd". No. C static variables are implicitly initialized to zero of the appropriate type, in this case, apparently a struct containing only a null pointer. This also means that their unsafe { #[allow(const_err)] static mut o_fwd: ::hacks::Hack<::data::Lisp_Objfwd> = unsafe { ::hacks::Hack::uninitialized() }; "a lot more typing to get a proper uninitialized value in a Rust program" is just useless voodoo.
- db48x 8y ago
  You are entirely correct, and I should perhaps have been more precise. However, because Lisp_Objfwd is basically a pointer, zero-initialized is pretty much as good as uninitialized. It's not a useful value.
  - danaliv 8y ago
    It fails a NULL check, where an uninitialized value (probably) won’t. That seems like a pretty important distinction, no?
    - db48x 8y ago
      In C, yes, that's a useful distinction. One of the goals of Rust is that you'll never have to check that manually. You'll never encounter a pointer that points to an invalid, uninitialized, or unallocated object. Since we're writing some unsafe code here, we can operate outside that restriction as long as we promise to put things right before the end of the unsafe block. This can be simplified somewhat once everything is written in Rust.
  - beagle3 8y ago
snazz 8y ago
Remacs’s README is really well written. More projects should try this hard to sell their ideas. This totally beats the GNU Emacs website at selling me on why I should use Emacs.
fangyuanziti 8y ago
Neovim vs Remacs now.

163 points by ealhad 8y ago

nazri1 8y ago

DEFUN ("atan", Fatan, Satan, 1, 2, 0, Hah I couldn't read that with a straight face!

db48x 8y ago
Agreed! It's a great example.
martinlofgren 8y ago
That's beautiful!

OskarS 8y ago

When I first heard of this project I thought "Well, that's insane, it will never be finished, and will surely be abandoned in six months". Glad to hear that it's making good progress! I love it when I'm wrong about stuff.

bjornedstrom 8y ago
Remember that most of Emacs is written in Emacs Lisp. If you take the entire source code distribution, around 15% of it is written in C. Those 15% are of course fairly complex, however it's at least less code than you may think.
- db48x 8y ago
  Yes. The last time I looked, it was about a million lines of elisp, and only about a hundred thousand of C. We'll have it cleaned up in no time.
mschaef 8y ago
Agreed... this seemed like it might run afoul of CADT [1], and I'm glad it hasn't. 1] https://www.jwz.org/doc/cadt.html https://www.jwz.org/doc/cadt.html

ealhad 8y ago

To anyone wondering why this might be a good idea, there are some interesting comments in a GitHub issue (https://github.com/Wilfred/remacs/issues/305 https://github.com/Wilfred/remacs/issues/305): > For users, I hope Remacs will be faster (it's easier to optimise), more robust (we have stronger type checks and the ability to add unit tests), better documented (see #262) and somewhat more featureful (we have some ideas for solving the dumper problem that would allow users to dump their instance as an image) than GNU Emacs. > I think we will get problems when the current generation of emacs devs retire and there's a better chance that emacs will survive if we try to make it more appealing to young developers. Rust seems to be a good option.

flurrything 8y ago
> For users, I hope Remacs will be faster The main reason my emacs is painfully slow is that most elisp code blocks. Stuff like hitting tab for auto-completion might perform a blocking query from a language server hanging emacs for multiple seconds. Opening a file tries to start a language server and blocks the editor in the process. Etc. Making emacs itself faster won't make the experience of working with emacs any better.
- simias 8y ago
  Some things like indenting a large file or even searching in a huge buffer can be quite slow. For instance I just tried indenting a ~1000line C buffer and it took about 2 seconds. When dealing with very large files (hundreds of MB) some basic movement and editing commands can be quite slow. Dealing with very large TAGS file (for the linux kernel for instance) is annoyingly laggy as well. I agree that it's not the panacea though, Emacs would benefit from using a more threaded or event-oriented programming model for many things.
- ams6110 8y ago
  Emacs does block on many operations, but I've never had it be really noticeable on tab-completion type stuff. Though I only use auto-complete on emacs commands, so maybe I've just missed it.

wscott 8y ago

And there are related efforts to upgrade the lisp used in emacs to a more modern and standardized version. I think it is GuileEmacs I am remembering. I hope to two efforts are compatible.

nerdponx 8y ago
My thinking as well. This is a golden opportunity to replace Elisp. Edit: Guile won't be used, because of the burden of contributing. https://github.com/Wilfred/remacs/issues/66#issuecomment-273333925 https://github.com/Wilfred/remacs/issues/66#issuecomment-273...
- rekado 8y ago
  This is very unfortunate. Emacs itself also requires copyright assignment, though, so if Remacs doesn't then it's virtually impossible for GNU to switch code bases (not that it would have been likely anyway).
simias 8y ago
That's the first thing that crossed my mind when I saw this project but I think it might be a mistake. Convincing people (and contributors) to ditch C-emacs for "remacs" will be hard, convincing people to ditch elisp-emacs for guilemacs will be hard but doing both at the same time might be just too much. If remacs manages to reach the point where you can use it to replace emacs in-place without any breakage while bringing better performance and more stability it will be a tremendous success IMO.
rauhl 8y ago
Although Scheme has a standard, I wouldn’t really call Guile ‘more modern and standardised’: Guile has no standard (it extends the Scheme standard in incompatible ways), and ‘modern’ is hard to define. Emacs Lisp added lexical scope in the past couple of years, probably more recently than Guile added a new language (as opposed to library) feature — is that modern enough? I wouldn’t imagine that the two efforts are compatible, because they are both efforts to rewrite the Emacs Lisp engine in another language, one in Rust & one in Guile Scheme. It’s possible that one might be able to call over to the other, through its CFFI. I’d like to see Emacs rewritten in Common Lisp, and so I’d prefer a Rust engine to a Guile engine: Rust is a static language, and a Rust engine is unlikely to lead to Rust-extensible Emacs code, while a Guile engine is likelier to lead to Guile-extensible Emacs code, which is completely the wrong direction IMHO.

mrec 8y ago

Not an Emacs user, so forgive me if I'm being dense, but is this (from the README) really true? Emacs will change how you think about programming. Emacs is an incremental programming environment. There's no edit-compile-run cycle. There isn't even an edit-run cycle. You can execute snippets of code and gradually turn them into a finished project. There's no distinction between your editor and your interpreter. I'm assuming what it means is "Emacs will change how you think about programming Emacs Lisp". Which is a rather narrower proposition, and much less of an incentive to switch, since I doubt many non-Emacs users are particularly interested in programming Emacs Lisp.

bachmeier 8y ago
My interpretation is that it's not a statement about writing code but running it. It's accurate IMO but probably not terribly important in the grand scheme of things. > I doubt many non-Emacs users are particularly interested in programming Emacs Lisp By definition, if you're programming Emacs Lisp, you're an Emacs user. Emacs is the interpreter.
- serhart 8y ago
  > By definition, if you're programming Emacs Lisp, you're an Emacs user. Emacs is the interpreter. That's not necessarily true. Guile supports Elisp as a language. You can write Elisp sans Emacs.
another-cuppa 8y ago
It is true. Learning any Lisp changes how you think about programming. Learning Emacs Lisp will make you wish all development was like that.
tmountain 8y ago
Clojure + Cider inside of "Spacemacs" is the most enjoyable development environment I've used in over 20+ years of programming. It's amazing how quickly you can get stuff done. Lisp changes how you think about programming. Emacs gives you the shortest path to a code/test feedback loop.
8y ago

ams6110 8y ago

Emacs rather famously has a build process that involves first compiling a minimal elisp environment (written in C and maybe some assembly) that then loads all the editing routines (which are written in elisp) and then dumps itself out as a binary (to save the load time, which is substantial, on subsequent start-ups). I assume the Rust-based emacs would do the same thing?

wiz21c 8y ago
not sure, but I guess that process is to make sure Emacs can be built from scratch, ie without relying on proprietary software.
- db48x 8y ago
  The build process of Emacs doesn't rely on any proprietary software. But it is quite complex :)
db48x 8y ago
We still use the same unexec process as normal Lisp, not because we really like it but because it's easier to make small changes. It really is a defect attractor though (I even introduced one a few months back, and had to debug it), so we do want to simplify it.

tom_mellior 8y ago

The author misunderstands some parts of the C language, in particular related to static variables. "In fact, these aren't just file globals accessible to only one translation unit, these are static variables that are accessible across the whole program." gets the meaning of C's static exactly backwards: C file-scope static variables are only accessible (by name) in their translation unit; all other file-scope variables are extern by default. More importantly, they write: static struct Lisp_Objfwd o_fwd; "... creates an (uninitialized) static variable called o_fwd". No. C static variables are implicitly initialized to zero of the appropriate type, in this case, apparently a struct containing only a null pointer. This also means that their unsafe { #[allow(const_err)] static mut o_fwd: ::hacks::Hack<::data::Lisp_Objfwd> = unsafe { ::hacks::Hack::uninitialized() }; "a lot more typing to get a proper uninitialized value in a Rust program" is just useless voodoo.

db48x 8y ago
You are entirely correct, and I should perhaps have been more precise. However, because Lisp_Objfwd is basically a pointer, zero-initialized is pretty much as good as uninitialized. It's not a useful value.
- danaliv 8y ago
  It fails a NULL check, where an uninitialized value (probably) won’t. That seems like a pretty important distinction, no?
  - db48x 8y ago
    In C, yes, that's a useful distinction. One of the goals of Rust is that you'll never have to check that manually. You'll never encounter a pointer that points to an invalid, uninitialized, or unallocated object. Since we're writing some unsafe code here, we can operate outside that restriction as long as we promise to put things right before the end of the unsafe block. This can be simplified somewhat once everything is written in Rust.
- beagle3 8y ago

snazz 8y ago

Remacs’s README is really well written. More projects should try this hard to sell their ideas. This totally beats the GNU Emacs website at selling me on why I should use Emacs.

fangyuanziti 8y ago

Neovim vs Remacs now.