24 ms·
Emacs is my primary editor, however Emacs dumper has always been a dumpster fire. Basically they coded up so much ill-concieved and inefficient Emacs Lisp that
by jmount 10y ago
Emacs is my primary editor, however Emacs dumper has always been a dumpster fire.
Basically they coded up so much ill-concieved and inefficient Emacs Lisp that the editor would never start up in an acceptable time. Instead of engineering around this (lazy loading services, fixing things, not doing things nobody needs) they had the great idea they could start the editor once and then core dump the in-memory state of a running editor. Then on later editor starts they would map-in the core dump and instantly be in a (somewhat) good state. Fails all kinds of smell tests and really speaks to bad taste having unbounded consequences. It is an idea that should not work and it is only happenstance that it ever worked (and it gets harder and harder as we have things like address layout randomization, file handles, and so on).
[edited "file" -> "fire", sorry! And yes I know lispers always dumped, but they are dumping the C memory environment here- not just their precious Lisp state. They should have had some appreciation for how the C environment actually worked since they decided to use it.]
- melling 10y agoYes, I remember building Emacs in the late 1980's and watching the final dump. I guess 30 years later hardware hasn't improved enough to avoid this? People are building editors in JavaScript today. Perhaps the Emacs Lisp engine needs to be tuned a little more? That Guile Emacs has been decades in the making...
- fatbird 10y agoSounds like it needs the neovim treatment: neomacs.
- sooheon 10y agoGod a neomacs written in a faster, more modern lisp would be a dream.
- bryanlarsen 10y agoIt's called guile emacs. https://lwn.net/Articles/615220/ https://lwn.net/Articles/615220/ https://www.emacswiki.org/emacs/GuileEmacs https://www.emacswiki.org/emacs/GuileEmacs
- fatbird 10y agoInteresting. Any idea on why it's not got more traction? Or is it actually on track to become the canonical emacs? Reading that LWN article, it looks like Guile wasn't a great choice from a community perspective, but more than that, by working internally to the emacs project it's subjected itself to internal standards of interoperability that it'll never really be able to hit (where a pseudo-hostile fork like xEmacs was might have sufficient momentum to force some accommodation).
- qwertyuiop924 10y agoIt sorta works, but it's hard to keep it working, so people don't really do much with it. Shockingly, the compatability was pretty good, last I checked. If you really want an emacs written in a sane Lisp, and don't mind losing gnuemacs compatability, check out Hemlock. or Edwin.
- draven 10y ago> check out Hemlock. or Edwin. Or Climacs https://www.common-lisp.net/project/climacs/ https://www.common-lisp.net/project/climacs/ It seems there's a rewrite as well: https://github.com/robert-strandh/Second-Climacs https://github.com/robert-strandh/Second-Climacs
- qwertyuiop924 10y agoOh. I forgot Climacs. Well, there you go. Another CL editor.
- lispm 10y agoHere is another one: https://github.com/cxxxr/lem https://github.com/cxxxr/lem
- sedachv 10y ago> Reading that LWN article, it looks like Guile wasn't a great choice from a community perspective I think going forward it will be. Between Guile-Emacs, Guix, and GNU Shepherd you have the best-supported Lisp Machine operating system analogue available right now: https://www.gnu.org/software/guix/ https://www.gnu.org/software/guix/ I am excited about GuixSD and I think a lot of other people will be.
- pjmlp 10y agoIt was called XEmacs.
- klocksib 10y agoOnce upon a time this basically happened: xemacs
- hyperhopper 10y agoAs for the more higher level stuff, spacemacs does a great job of that. It really makes configuration files, separation of concerns, vim bindings, and many other things sensible and user friendly. It doesn't touch the lower level code though, as it is just an elisp layer on top of emacs itself.
- pm215 10y agoI agree that dumping an image is not a great idea in the modern world, but the concept(1) dates right back to when emacs was still based on TECO on the PDP-10: http://pdp-10.trailing-edge.com/mit_emacs_170_teco_1220/01/info/mkdump.info.html http://pdp-10.trailing-edge.com/mit_emacs_170_teco_1220/01/i... (and I suspect the available CPU horsepower back then was such that you pretty much needed to do tricks like this to get quick startup even if you were careful about how much lisp code you ran at startup). (1) as a part of emacs, that is -- I'd guess that "language runtime takes an image of its pre-initialized state and then just loads that" was used in non-emacs contexts before that.
- lispm 10y agoThe first Lisp from the early 60s did already dump images.
- int_19h 10y agoDid it dump Lisp images, or the image of the entire process (including all the non-Lisp stuff)? It appears that all the complexity and instability here comes from the latter part. Lisp itself was always designed around this image dumping concept. But C and its standard library were not.
- qwertyuiop924 10y agoActually, many early unixes included undump, which let you build an executable from a coredump.
- int_19h 10y agoAnd what were the stability guarantees for such things? Could you expect to take such a dump, bring it to a different system (potentially with different hardware, different minor OS revision etc), and expect it to work?
- jrochkind1 10y ago
- btilly 10y agoIt is an idea that should not work and it is only happenstance that it ever worked In the 1980s it was a surprisingly common approach. For example most Unix variants had an undump command to turn a core dump into an executable. See https://www.ctan.org/tex-archive/obsolete/support/undump https://www.ctan.org/tex-archive/obsolete/support/undump for example. It was also how Microsoft Office file formats were implemented until about a decade ago. With the hindsight of experience, it is easy to say that it is a bad way to go and won't age well. But back when Emacs went this way, the problems were not as widely understood as they are now.
- sooheon 10y agoNow that we do have the hindsight of experience, and it is time to make that decision again, why are the maintainers going with the clearly smellier approach?
- jeremiep 10y agoBecause even with experience you don't refactor a 40+ years old piece of software easily without breaking half the ecosystem. I'm sure there's a whole world of tradeoffs we're not seeing that the maintainers have to deal with.
- MBCook 10y agoThey have an immediate problem (things are slow/broken on newer glibc) that needs fixing. Someone has posted sane code to fix the issue in a similar fashion to what they were doing before but without needing to reach into the memory manager. The maintainer who is mad wants it fixed right (replace the whole thing and make the elisp load fast) but so far neither they nor anyone else seems to have posted code to do that. Bird in the hand. I would hope they'd just remove the need for the mechanism totally in the future, but for now.... bird in the hand.
- btilly 10y agoWhat would fixing it right look like? Make elisp faster. Easier said than done. Any easy wins already happened. Introduce various lazy loading mechanisms and switch to that by default. That doesn't help any third party plugins that people might use. In fact any such work could create ways of breaking the expectations of said plugins! Identify/eliminate work spent on dead code paths in initial load. Again, easier said than done. And there are bound to be a lot of, "I don't see what that is useful for, but what third party plugins might need it?" In the meantime, end users faced with the choice of a slow new release versus a fast old one, will resist upgrading. And if there is a patch that simply makes things fast, that is generally acceptable. The current moment provides pressure to do things right. If the cheap fix is taken, there are no guarantees on when there will be another opportunity like this one. But pressures exist to take the easy out right now. And if taken, it won't be easier next time either.
- youdontknowtho 10y agodid you mean dumpster file or dumpster fire? I Kid! I KID!!!! EDIT: Seriously? Humor's not allowed? What if you read my comment with Triumph The Insult Comic Dog's voice? See? It's funny! EDIT EDIT: I'm actually an EMACS fan. It's weird that you have to propitiate yourself to a text editor over a joke.
- youdontknowtho 10y agoDude. That was comedy gold! Downvotes? Really? Come on! That was funny. Down vote all you like, that was funny.
- deleted 10y ago[deleted]
- tumba 10y agoThis may seem strange to you, but this is actually the way many lisp systems (and other image-based development systems, like Smalltalk) work. I build software using the commercial develop environment Allegro Common Lisp and an image dump is precisely the way I deliver software. The ultimate deliverable is an executable that loads the image and launches the main execution thread.
- weavie 10y agoThis works really well for us. To load up our web server takes a while due to having to load up caches etc.. so we tend to do all that once, dump the image and deploy it - cache already primed. It had an even bigger advantage for me today. Due to some git shenanigans I somehow managed to wipe out a few days of work. No idea how... Almost resigning myself to having to recode the whole lot I just remembered that I had built an image to test just prior to attempting to check in. Clozure CL saves the source code of each function in the image. I was able to probe the image and pull out all the source code that I had just lost, saving myself a few days of tedious recoding! Not a recommended use of images - but it sure saved my butt today!
- gkop 10y ago> Due to some git shenanigans I somehow managed to wipe out a few days of work. What happened??
- weavie 10y agoI actually have no idea. I had a lot of unchecked work - about a weeks worth. (I know, I should not have let it go that far.. it was just one of those tasks that just seemed to get bigger and bigger.) It was time to check in, so I went through file by file staging it all (using Magit). I then decided to do some more testing. Built a lisp image. Then I became distracted, started thinking about home time etc... so my recollection of what I did is a little hazy. I think I saw I had some changes that I wasn't wanting to commit and test. So I think I stashed them. Then probably surfed hacker news or something.. When I got back to things I suddenly noticed that none of my changes were there any more. Nothing was staged, nothing was stashed. It was all gone. I spent a few hours looking around for it but all to no avail. It is the second time I have lost work by misusing git. I think the other time I was messing with rebasing and branching and all my work disappeared. I do really need to learn how to use git properly. But in the meantime I am not taking any more chances. I have set up a zone on our internal SmartOS server which uses ZFS. An hourly cron job on my machine rsyncs my source directory to the zone and then creates a ZFS snapshot. So, hopefully no matter how badly I use git now I will never be able to lose more than about an hours worth of work.
- kazinator 10y agoThey didn't have this great idea; it came from the Lisp culture. Lisp implementations have the ability to save images. It's not described in ANSI CL, but all the "industrial strength" Lisp have this, and it is one of the means for application delivery. If you're working on an interpreted Lisp which is getting slow to load due to its growing library, image dumping and restarting is the obvious "off the shelf" approach you already know about from being a Lisp programmer. > Instead of engineering around this (lazy loading services I'd be surprised if Emacs didn't have lazy loading too. Lazy loading only goes so far. Sometimes lazy loading brings in a lot of dependencies, so it is slow for those users who use whatever is being lazily loaded. Lazy loading is a hack to work around the lack of a fast image restore. The thing that is hacky about Emacs image saving is that it saves the C state. It's not exactly a Lisp image saving mechanism. This is why I was abel to port this dump/undump very easily to GNU Make, which doesn't share any data structures with Emacs, let alone any Lisp stuff. Dumping the image worked just fine for the C data structures on GNU Make's heap. It's because it's such a low-level mechanism that they are having difficulties with its portability. I don't hear about, for instance, the CLISP project having issues with the EXT:SAVEINITMEM function due to some GLibc support changing; it doesn't rely on that.
- AceJohnny2 10y ago> I'd be surprised if Emacs didn't have lazy loading too. Not sure if this is exactly what you're referring to, but Emacs has the autoload [1] feature, which lets you declare some functions and in which elisp file to find them. Emacs won't load that file until the function is actually called. This is usually for user-called functions. Most packages use it. [1] https://www.gnu.org/software/emacs/manual/html_node/elisp/Autoload.html https://www.gnu.org/software/emacs/manual/html_node/elisp/Au...
- sedachv 10y ago> Instead of engineering around this (lazy loading services, fixing things, not doing things nobody needs) they had the great idea they could start the editor once and then core dump the in-memory state of a running editor. Then on later editor starts they would map-in the core dump and instantly be in a (somewhat) good state. Fails all kinds of smell tests and really speaks to bad taste having unbounded consequences. How else would you compile a dynamic programming language program into a binary executable?
- sedachv 10y agoBTW, something that just came to mind: this "bad taste having unbounded consequences" of copying the entire process memory is exactly how fork(2) works.
- firethief 10y agoDoing something that relies on OS internals is bad taste when done in userspace. Interface stability aside, what fork does is simpler: a copied process can rely on its environment matching its parent's enough for a large subset of the contents of the duplicated address space to retain their meaning. Serializing a process's address space's contents and then loading it into an "unrelated" process later makes it more complicated to know what memory contents can still be relied on (e.g. ASLR, file handles, etc.).
- armitron 10y agoNothing of what Emacs or Common Lisp systems or Smalltalk systems do when they dump images relies on OS internals. Emacs unexec relied on glibc internals but the new portable dumper does away with that and only relies on gasp Emacs internals. Where do you people come up with that stuff?
- sedachv 10y ago> Interface stability aside, what fork does is simpler: a copied process can rely on its environment matching its parent's enough for a large subset of the contents of the duplicated address space to retain their meaning. Serializing a process's address space's contents and then loading it into an "unrelated" process later makes it more complicated to know what memory contents can still be relied on (e.g. ASLR, file handles, etc.). Everything you mentioned except for ASLR is a problem for forked processes, and it boils down to kernel state (file descriptors and signals being two that I can think of right now).
- gkafkg8y8 10y ago> Emacs dumper has always been a dumpster fire. This made me laugh. Thanks!