8 ms·

Lisp Operating System (2015)

173 points by antipropaganda 7y ago

ryanolsonx 7y ago
They spelt emacs wrong
- sagartewari01 7y ago
  They didn't name anything, they just provided a description.
  - hnlmorg 7y ago
    I assumed his comment was a reference to the old "joke": > "Emacs is a great OS, but it lacks a decent text editor." The comment presumably made because emacs is extended with LISP.
    - somatic 7y ago
      Emacs is written in Emacs LISP, which is its own language. The really performance-critical stuff is written in C, as is the interpreter, but that’s it.
      - hnlmorg 7y ago
        That's ostensibly what I said wasn't it? Sorry but I can't tell if you were agreeing or disagreeing with me.
        somatic 7y ago
        I interpreted your comment to mean something like “Emacs is configurable with third-party LISP code”, in much the same way as Vim is configurable with third-party Vimscript. But Vim is written in C, Vimscript being just a bolt on, whereas Emacs LISP is its own language entirely, a distinct variant of LISP, and much of Emacs implemented in it.
        hnlmorg 7y ago
        LISP is a family of languages (much like BASIC). So Emacs Lisp is still a LISP. I don't even know why you're bring Vim into the discussion when it has nothing to do with the topic being discussed. It doesn't invalidate the statement I made; I just makes your reply look a little snobby because you seem concerned that someone might compare Emacs and Emacs Lisp to Vim and Vim Script (I clearly wasn't given the complete lack of reference to Vim in my post). To be honest I thought we had gotten past those tedious flamewars in the 90s.
    - snaky 7y ago
      This joke 2.0 would be about ChromeOS and a decent web browser.
- dan-robertson 7y ago
  There’s lots of great stuff about emacs but it doesn’t really fit the description in the OP.
neilv 7y ago
For the incremental development the author describes, you can do that atop any GNU/Linux host platform on which your CL implementation runs. To play with a bootable user experience, before you've replaced the host kernel and device support, you can strip down something like Debian Live. I previously did this for a Racket-based living room appliance project. A simpler example, without Racketisms, that uses only 2 shell scripts (you only need 1; the other is to play partition table tricks), is my earlier Debian Live variant: https://www.neilvandyke.org/lildeb/ https://www.neilvandyke.org/lildeb/ I'm also a fan of a variation on the author's approach: have your better/different language get a foothold on a working GNU/Linux system, alongside traditional apps, in a system the user could use as their daily driver, and incrementally replace all of userland. This was my plan for PostmarketOS (though I've suspended my open source work on "non-employable" things): https://www.neilvandyke.org/postmarketos/ https://www.neilvandyke.org/postmarketos/ The author's approach also seems valid, and worth playing with.
- TeMPOraL 7y ago
  > I'm also a fan of a variation on the author's approach: have your better/different language get a foothold on a working GNU/Linux system, alongside traditional apps, in a system the user could use as their daily driver, and incrementally replace all of userland. That's incidentally what Emacs is to many of its users (myself included).
  - neilv 7y ago
    I used to do this with Emacs, too (VM, Gnus, w3m, and more besides all the programming tools), and some of my Emacs packages are in support of that: https://www.neilvandyke.org/emacs/ https://www.neilvandyke.org/emacs/ For a (mostly) single host process, without preemptive multitasking/threading, Emacs as an operating system works surprisingly well.
ngcc_hk 7y ago
Whilst difference is good, I am not sure many of the “attack” on the process etc is right. Whilst you can bring anything on old hardware (car and cdr come to mind and those strange intel register format), some gem can be argued instead of rubbish. Process is an isolation, small address is easier to debug if you do not need 64bit address etc. And driver issue ... lisp driver. Really not sure the whole game.
pushpop 7y ago
I’m all for thinking outside the box but many of the complaints described in that are blamed on UNIX when in fact they were also adopted by a multitude of other non-UNIX platforms and reason they became common was because it simply made more sense for general computing. For example their “file system” (for want a better description) approach was tried on an IBM mainframe and it proved more cumbersome to maintain than the typical file system approach we have currently. Also when talking about address spaces he acknowledges traditional memory management exists for security reasons then says that’s not needed in his OS without actually proving how his method either prevents the same attacks nor new ones that effectively result in the same kind of vulnerability (I could think of a few very easy attacks against his OS design). Similarly the way he combines physical storage and system memory into one device completely overlooks technical limitations of the hardware, the risk of data loss if you don’t manage things correctly, the vastly different storage capacities between the two, nor the fact that basically every OS does this to some extent already via caching. While it’s nice to go back to the drawing board and rethink certain idioms, this felt more like a half baked rant at UNIX than an objective look at the past and current OS designs.
- hnlmorg 7y ago
  That was my take away as well. Some interesting ideas - most of which aren't new - but the ideology felt very biased. To a fault even. I would still love to see a working version of this. Or even just a more traditional general computing OS but one which used FP (not necessarily LISP) as its primitive.
  - TuringTest 7y ago
    This article I read yesterday has one interesting take on what an OS designed on modern programming principles could look like. It has a loose treatment of technical FP terms, but the insights behind it are at least thought-provoking. https://medium.com/@reinman/monoids-to-groupoids-492c35105113 https://medium.com/@reinman/monoids-to-groupoids-492c3510511... I believe we're in for a redesign of OS architectures, now that the 50's assumption of a single machine with a single OS kernel running all programs (or the 60's equivalent of several few mainframes communicating through a slow network) is no longer the main paradigm.
heisig 7y ago
Summary: - We don't need separate processes if the language we use is inherently safe (e.g., modern Common Lisp). - Without process isolation, it is possible to share large, complicated and possibly mutable data structures (graphs, arrays, user-defined objects) system-wide. - Once it is possible to cheaply share and communicate arbitrary data structures, it is pointless to maintain designated 'file system'. - Not having a file system raises the question of what data should be persistent and what shouldn't. But on a modern computer, it is feasible to just treat all data as persistent (maybe excluding the youngest GC generation). The best news is that the author is actually working hard to implement this operating system. The first part - the Lisp implementation - is already in a pretty good shape and could be finished within the next two years: https://github.com/robert-strandh/SICL https://github.com/robert-strandh/SICL
- varjag 7y ago
  Should mention that Mezzano OS satisfies most of these design points and already runs. https://github.com/froggey/Mezzano https://github.com/froggey/Mezzano
  - mark_l_watson 7y ago
    I wanted to also mention Mezzano - really easy to play with using VirtualBox. A bit off topic, but when I miss the environment of the Xerox 1108 Lisp Machine that I had from 1982 to about 1987, I find the closest thing today that offers a similar experience is Pharo Smalltalk.
    - 0x445442 7y ago
      I've had some user interface ideas I've wanted to explore floating around in my head for more than a decade. I' m now in a position to start exploring these ideas and after an extensive survey I decided to use Squeak. The deeper I get into this system the more appreciation I gain for the entire model.
      - gambler 7y ago
rakoo 7y ago
The criticisms may or may not be correct, but I fail to see why a Lisp (and not another language) would be the answer to all of the issues here.
- heisig 7y ago
  The points for Common Lisp are: 1. It is ANSI standardized and mature. 2. Common Lisp programs are safe. Some implementations violate safety for performance reasons, but this one certainly won't. 3. Common Lisp has the best error handling mechanism that I know of (Conditions, handlers and restarts). 4. The language is programmable, both at the syntax level (Macros) and at the object level (CLOS with the metaobject protocol). So new ideas from other programming languages can often just be ported. 5. The language has been designed for interactivity and safe incremental redefinition. 6. Lisp has already been used very successfully for operating system development (https://en.wikipedia.org/wiki/Lisp_machine https://en.wikipedia.org/wiki/Lisp_machine). This makes it pretty much the ideal choice for such a project.
effie 7y ago
This document talks a lot about deficiencies of the unix way, yet fails to give any concrete examples of advantage of the proposed new way where everything runs in the same address space, uses object storage, and is allowing users to run (!) "only code that has been produced from the high-level notation of the language". I agree that there are technical problems with interoperability of programs and partially this may be due to the unix way. But the article does not analyze this and it is not clear that such a radical change is a good way to solve those problems. Even if theoretically yes, there are practical things to consider, such as the insane amount of work needed to switch. Perhaps the author just likes Lisp and has a solution in need of a problem.
shalabhc 7y ago
Great write-up and I agree with many points. I don't think the 'Lisp language' is the key point here though, but the other OS properties like a single level store and composition methods richer than byte arrays. Would be nice if the author referenced historical systems that overlap these ideas. One is the Symbolics Genera system with 'data level integration' so apps shared rich data structures instead of uninterpreted bits. https://www.ifis.uni-luebeck.de/~moeller/symbolics-info/genera/genera.html https://www.ifis.uni-luebeck.de/~moeller/symbolics-info/gene... Another is Appolo Domain/OS which had a single level store: http://bitsavers.org/pdf/apollo/014962-A00_Domain_OS_Design_Principles_Jan89.pdf http://bitsavers.org/pdf/apollo/014962-A00_Domain_OS_Design_...
- twic 7y ago
  Yes, absolutely astounding that this article doesn't mention Genera. If we're going to proceed to list every single address space operating system we've ever heard of, i'll mention SPIN, written in Modula 3 (although i don't know if it did anything clever about sharing data between applications): http://www-spin.cs.washington.edu/external/overview.html http://www-spin.cs.washington.edu/external/overview.html
GorgeRonde 7y ago
Makes me think of loper os: http://www.loper-os.org/?p=4 http://www.loper-os.org/?p=4
hardtoreason 7y ago
I literally stalled on the first sentence where he asserts without any rationale that an OS written in Lisp "would be a good thing in itself". WHY WHY WOULD ANYONE THINK THIS MATTERS
- gooseus 7y ago
  I would, and it's based on a pretty simple philosophical/moral/spiritual belief: Anything that increases and improves our knowledge of the universe - including knowledge of how to best use existing tools, and which tools to use for which jobs - is a good thing in and of itself (barring exceptions related to negative externalities which don't apply here). I don't know if that's the author's rationale, but it's the one I would use.
- GrinningFool 7y ago
  Conversely, why would anyone think that because a thing doesn't matter to them, it shouldn't matter to anyone else either?
  - deleted 7y ago
    [deleted]
  - AnimalMuppet 7y ago
    I think that if you were to poll the users of an OS about what matters to them, the language it was implemented in would be at the absolute bottom of the list. It may matter to the author of the article. That's fine. To the users? I'm pretty sure that hardtoreason correctly reflects their attitude. And if you're writing an OS, but don't care about the users? Well, write it however matters to you, because you aren't likely to have users who care, because you aren't likely to have users at all.
    - yellowapple 7y ago
      > And if you're writing an OS, but don't care about the users? Well, write it however matters to you, because you aren't likely to have users who care, because you aren't likely to have users at all. Counterpoint: Windows 10.
mwkaufma 7y ago
How does failure-recovery (software bugs, hardware, or otherwise) or resilience-engineering more generally work on a system that demurs isolation and redundancy (honest question)?
newsoul2019 7y ago
Does anyone remember the TI Explorer? (https://en.wikipedia.org/wiki/TI_Explorer https://en.wikipedia.org/wiki/TI_Explorer). They were sold in the 1980s, I first saw one in person in 1993. It was astonishing back then that it had an optical mouse and an optical cable connecting the keyboard to the rest of the unit. My understanding is that it was a full stack LISP machine, down to CAR/CDR support in hardware.

They spelt emacs wrong

sagartewari01 7y ago
They didn't name anything, they just provided a description.
- hnlmorg 7y ago
  I assumed his comment was a reference to the old "joke": > "Emacs is a great OS, but it lacks a decent text editor." The comment presumably made because emacs is extended with LISP.
  - somatic 7y ago
    Emacs is written in Emacs LISP, which is its own language. The really performance-critical stuff is written in C, as is the interpreter, but that’s it.
    - hnlmorg 7y ago
      That's ostensibly what I said wasn't it? Sorry but I can't tell if you were agreeing or disagreeing with me.
      - somatic 7y ago
        I interpreted your comment to mean something like “Emacs is configurable with third-party LISP code”, in much the same way as Vim is configurable with third-party Vimscript. But Vim is written in C, Vimscript being just a bolt on, whereas Emacs LISP is its own language entirely, a distinct variant of LISP, and much of Emacs implemented in it.
        hnlmorg 7y ago
        LISP is a family of languages (much like BASIC). So Emacs Lisp is still a LISP. I don't even know why you're bring Vim into the discussion when it has nothing to do with the topic being discussed. It doesn't invalidate the statement I made; I just makes your reply look a little snobby because you seem concerned that someone might compare Emacs and Emacs Lisp to Vim and Vim Script (I clearly wasn't given the complete lack of reference to Vim in my post). To be honest I thought we had gotten past those tedious flamewars in the 90s.
  - snaky 7y ago
    This joke 2.0 would be about ChromeOS and a decent web browser.
dan-robertson 7y ago
There’s lots of great stuff about emacs but it doesn’t really fit the description in the OP.

For the incremental development the author describes, you can do that atop any GNU/Linux host platform on which your CL implementation runs. To play with a bootable user experience, before you've replaced the host kernel and device support, you can strip down something like Debian Live. I previously did this for a Racket-based living room appliance project. A simpler example, without Racketisms, that uses only 2 shell scripts (you only need 1; the other is to play partition table tricks), is my earlier Debian Live variant: https://www.neilvandyke.org/lildeb/ https://www.neilvandyke.org/lildeb/ I'm also a fan of a variation on the author's approach: have your better/different language get a foothold on a working GNU/Linux system, alongside traditional apps, in a system the user could use as their daily driver, and incrementally replace all of userland. This was my plan for PostmarketOS (though I've suspended my open source work on "non-employable" things): https://www.neilvandyke.org/postmarketos/ https://www.neilvandyke.org/postmarketos/ The author's approach also seems valid, and worth playing with.

TeMPOraL 7y ago
> I'm also a fan of a variation on the author's approach: have your better/different language get a foothold on a working GNU/Linux system, alongside traditional apps, in a system the user could use as their daily driver, and incrementally replace all of userland. That's incidentally what Emacs is to many of its users (myself included).
- neilv 7y ago
  I used to do this with Emacs, too (VM, Gnus, w3m, and more besides all the programming tools), and some of my Emacs packages are in support of that: https://www.neilvandyke.org/emacs/ https://www.neilvandyke.org/emacs/ For a (mostly) single host process, without preemptive multitasking/threading, Emacs as an operating system works surprisingly well.

ngcc_hk 7y ago

Whilst difference is good, I am not sure many of the “attack” on the process etc is right. Whilst you can bring anything on old hardware (car and cdr come to mind and those strange intel register format), some gem can be argued instead of rubbish. Process is an isolation, small address is easier to debug if you do not need 64bit address etc. And driver issue ... lisp driver. Really not sure the whole game.

pushpop 7y ago

I’m all for thinking outside the box but many of the complaints described in that are blamed on UNIX when in fact they were also adopted by a multitude of other non-UNIX platforms and reason they became common was because it simply made more sense for general computing. For example their “file system” (for want a better description) approach was tried on an IBM mainframe and it proved more cumbersome to maintain than the typical file system approach we have currently. Also when talking about address spaces he acknowledges traditional memory management exists for security reasons then says that’s not needed in his OS without actually proving how his method either prevents the same attacks nor new ones that effectively result in the same kind of vulnerability (I could think of a few very easy attacks against his OS design). Similarly the way he combines physical storage and system memory into one device completely overlooks technical limitations of the hardware, the risk of data loss if you don’t manage things correctly, the vastly different storage capacities between the two, nor the fact that basically every OS does this to some extent already via caching. While it’s nice to go back to the drawing board and rethink certain idioms, this felt more like a half baked rant at UNIX than an objective look at the past and current OS designs.

hnlmorg 7y ago
That was my take away as well. Some interesting ideas - most of which aren't new - but the ideology felt very biased. To a fault even. I would still love to see a working version of this. Or even just a more traditional general computing OS but one which used FP (not necessarily LISP) as its primitive.
- TuringTest 7y ago
  This article I read yesterday has one interesting take on what an OS designed on modern programming principles could look like. It has a loose treatment of technical FP terms, but the insights behind it are at least thought-provoking. https://medium.com/@reinman/monoids-to-groupoids-492c35105113 https://medium.com/@reinman/monoids-to-groupoids-492c3510511... I believe we're in for a redesign of OS architectures, now that the 50's assumption of a single machine with a single OS kernel running all programs (or the 60's equivalent of several few mainframes communicating through a slow network) is no longer the main paradigm.

heisig 7y ago

Summary: - We don't need separate processes if the language we use is inherently safe (e.g., modern Common Lisp). - Without process isolation, it is possible to share large, complicated and possibly mutable data structures (graphs, arrays, user-defined objects) system-wide. - Once it is possible to cheaply share and communicate arbitrary data structures, it is pointless to maintain designated 'file system'. - Not having a file system raises the question of what data should be persistent and what shouldn't. But on a modern computer, it is feasible to just treat all data as persistent (maybe excluding the youngest GC generation). The best news is that the author is actually working hard to implement this operating system. The first part - the Lisp implementation - is already in a pretty good shape and could be finished within the next two years: https://github.com/robert-strandh/SICL https://github.com/robert-strandh/SICL

varjag 7y ago
Should mention that Mezzano OS satisfies most of these design points and already runs. https://github.com/froggey/Mezzano https://github.com/froggey/Mezzano
- mark_l_watson 7y ago
  I wanted to also mention Mezzano - really easy to play with using VirtualBox. A bit off topic, but when I miss the environment of the Xerox 1108 Lisp Machine that I had from 1982 to about 1987, I find the closest thing today that offers a similar experience is Pharo Smalltalk.
  - 0x445442 7y ago
    I've had some user interface ideas I've wanted to explore floating around in my head for more than a decade. I' m now in a position to start exploring these ideas and after an extensive survey I decided to use Squeak. The deeper I get into this system the more appreciation I gain for the entire model.
    - gambler 7y ago

rakoo 7y ago

The criticisms may or may not be correct, but I fail to see why a Lisp (and not another language) would be the answer to all of the issues here.

heisig 7y ago
The points for Common Lisp are: 1. It is ANSI standardized and mature. 2. Common Lisp programs are safe. Some implementations violate safety for performance reasons, but this one certainly won't. 3. Common Lisp has the best error handling mechanism that I know of (Conditions, handlers and restarts). 4. The language is programmable, both at the syntax level (Macros) and at the object level (CLOS with the metaobject protocol). So new ideas from other programming languages can often just be ported. 5. The language has been designed for interactivity and safe incremental redefinition. 6. Lisp has already been used very successfully for operating system development (https://en.wikipedia.org/wiki/Lisp_machine https://en.wikipedia.org/wiki/Lisp_machine). This makes it pretty much the ideal choice for such a project.

effie 7y ago

This document talks a lot about deficiencies of the unix way, yet fails to give any concrete examples of advantage of the proposed new way where everything runs in the same address space, uses object storage, and is allowing users to run (!) "only code that has been produced from the high-level notation of the language". I agree that there are technical problems with interoperability of programs and partially this may be due to the unix way. But the article does not analyze this and it is not clear that such a radical change is a good way to solve those problems. Even if theoretically yes, there are practical things to consider, such as the insane amount of work needed to switch. Perhaps the author just likes Lisp and has a solution in need of a problem.

shalabhc 7y ago

Great write-up and I agree with many points. I don't think the 'Lisp language' is the key point here though, but the other OS properties like a single level store and composition methods richer than byte arrays. Would be nice if the author referenced historical systems that overlap these ideas. One is the Symbolics Genera system with 'data level integration' so apps shared rich data structures instead of uninterpreted bits. https://www.ifis.uni-luebeck.de/~moeller/symbolics-info/genera/genera.html https://www.ifis.uni-luebeck.de/~moeller/symbolics-info/gene... Another is Appolo Domain/OS which had a single level store: http://bitsavers.org/pdf/apollo/014962-A00_Domain_OS_Design_Principles_Jan89.pdf http://bitsavers.org/pdf/apollo/014962-A00_Domain_OS_Design_...

twic 7y ago
Yes, absolutely astounding that this article doesn't mention Genera. If we're going to proceed to list every single address space operating system we've ever heard of, i'll mention SPIN, written in Modula 3 (although i don't know if it did anything clever about sharing data between applications): http://www-spin.cs.washington.edu/external/overview.html http://www-spin.cs.washington.edu/external/overview.html

GorgeRonde 7y ago

Makes me think of loper os: http://www.loper-os.org/?p=4 http://www.loper-os.org/?p=4

hardtoreason 7y ago

I literally stalled on the first sentence where he asserts without any rationale that an OS written in Lisp "would be a good thing in itself". WHY WHY WOULD ANYONE THINK THIS MATTERS

gooseus 7y ago
I would, and it's based on a pretty simple philosophical/moral/spiritual belief: Anything that increases and improves our knowledge of the universe - including knowledge of how to best use existing tools, and which tools to use for which jobs - is a good thing in and of itself (barring exceptions related to negative externalities which don't apply here). I don't know if that's the author's rationale, but it's the one I would use.
GrinningFool 7y ago
Conversely, why would anyone think that because a thing doesn't matter to them, it shouldn't matter to anyone else either?
- deleted 7y ago
  [deleted]
- AnimalMuppet 7y ago
  I think that if you were to poll the users of an OS about what matters to them, the language it was implemented in would be at the absolute bottom of the list. It may matter to the author of the article. That's fine. To the users? I'm pretty sure that hardtoreason correctly reflects their attitude. And if you're writing an OS, but don't care about the users? Well, write it however matters to you, because you aren't likely to have users who care, because you aren't likely to have users at all.
  - yellowapple 7y ago
    > And if you're writing an OS, but don't care about the users? Well, write it however matters to you, because you aren't likely to have users who care, because you aren't likely to have users at all. Counterpoint: Windows 10.

mwkaufma 7y ago

How does failure-recovery (software bugs, hardware, or otherwise) or resilience-engineering more generally work on a system that demurs isolation and redundancy (honest question)?

newsoul2019 7y ago

Does anyone remember the TI Explorer? (https://en.wikipedia.org/wiki/TI_Explorer https://en.wikipedia.org/wiki/TI_Explorer). They were sold in the 1980s, I first saw one in person in 1993. It was astonishing back then that it had an optical mouse and an optical cable connecting the keyboard to the rest of the unit. My understanding is that it was a full stack LISP machine, down to CAR/CDR support in hardware.