6 ms·

Why do we need a new OS?

84 points by thecombjelly 11y ago

alexandrerond 11y ago
When I read this I raised my eyebrow. I hardly think that Lisp (1958) can be the base of the operating system of the future. Their texts are full of bold statements about how easy and natural lisp is and how everything else sucks. Nevertheless it's interesting to look at their references: http://3lproject.org/references http://3lproject.org/references I understand than in short, the security model is about how to make sure a process does not do more than what it's supposed to. In this case it's lisp environments that provide the necessary sandboxing.
- na85 11y ago
  That Lisp began in 1958 is mostly irrelevant. It (Common Lisp) is a fully-modern, Turing-complete language with a first-rate object system and supports all programming paradigms. On top of that the implementations have all been regularly updated over the years and many are extremely mature and well-optimized. Do you equate newness with quality, as seems to be common on HN? Perhaps an OS written entirely in this month's JavaScript framework, deployed on Docker, using monads-as-a-service or whatever the hipsters have come up with lately would be more your cup of tea.
  - rodgerd 11y ago
    > Do you equate newness with quality, as seems to be common on HN? When the link's thesis is "these OSes design years ago are old and therefore bad" it is hardly unreasonable to point out the language the author prefers is even older.
    - pjmlp 11y ago
      Because the UNIX based OS are based on PDP-11 architecture with the compromises required to fit in such systems. Whereas already in the 60s, we had hardware systems like the Burroughs b5000 that could be fully programmed in Extended Algol and Espol, with zero Assembly. The idea, as I see it, to base modern OSes in better older architectures that were lost to the mists of time. Actually the mobile OSes, with their memory safe languages seem to already be moving into this direction.
    - na85 11y ago
      >these OSes design years ago are old and therefore bad That's not the link's thesis. Equally flippant but more correct would be to say "these OSes were written in memory-unsafe languages because hardware limitations demanded it back then, therefore they are bad"
  - anentropic 11y ago
    a Turing-complete programming language? How novel!
    - na85 11y ago
      My point was that there isn't anything you can do in Golang or Rust or any other FOTM language that you can't do in Lisp.
- Animats 11y ago
  If they came up with a good security model for applications, that would be a big step forward. Capabilities are a mechanism, not a model. Somebody has to decide who gets which capabilities. That mechanism needs to be understandable by the user. For example, Privileges for a game: - Can use full screen. - Can read own content installed with app. - Can read/write persistent storage associated with app. - Can communicate with domain "thisgame.ea.com". - Cannot run in background, access address book, ... Word processor: - Can access files mentioned in user file dialogs (file dialogs being run from a separate process). - Can transmit requests to send mail to app "mail". For any app which accesses information outside its own content and own persistent storage, the user should have the options "allow", "deny", and "fake". If a program asks for address book access and won't run without it, you can give it the "fake" option, which gives it access to a dummy address book. (I've seen something like this demoed for jailbroken Android phones.)
  - arthurfm 11y ago
    > I've seen something like this demoed for rooted Android phones The amount of data XPrivacy [1] can fake is impressive. It's a shame it isn't a built-in feature. [1] https://github.com/M66B/XPrivacy#features https://github.com/M66B/XPrivacy#features
    - Animats 11y ago
      The biggest problem is the UNIX and Windows tradition that programs have all the rights of the user running them. Today, they should start out with the rights to read their own installed stuff, access their own private read/write prefs area, and use temp files. Anything more than that is at the discretion of the user.
- nickpsecurity 11y ago
  LISP side of things already has a bit of work toward a security model and components for robust implementation: Security Kernel https://www.kyleisom.net/downloads/security_kernel.pdf https://www.kyleisom.net/downloads/security_kernel.pdf Formally-verified Scheme http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=881F8DEAC1EE14DC88EAB49ED026CB05?doi=10.1.1.36.9989&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=881... The features could certainly be worth it. Some aspects of the Genera experience, esp productivity & holistic debugging, still aren't in modern flows: http://www.symbolics-dks.com/Genera-why-1.htm http://www.symbolics-dks.com/Genera-why-1.htm So, who knows where it will go. Betting on a semi-functional language with prior high-security and high-productivity work is a better start than most projects like this, though.
abstractbeliefs 11y ago
Fascinating project. I'm glad that alongside the philosophy of this being a FOSS endeavor, there's the practical aspect of this being adjustable on the fly by users in the old Lisp machine tradition. Though I'm sure it's too early for outsiders to shape the PoC code, is it available anywhere for us all to follow along?
- thecombjelly 11y ago
  > is it available anywhere for us all to follow along? A mailing list to follow development will be available soon unfortunately I haven't had time to get it up and running yet. edit: but check back to 3lproject.org soon or the kickstarter page. If you have kickstarted it I'll also make sure to send out an update.
- mchahn 11y ago
  > there's the practical aspect of this being adjustable on the fly by users in the old Lisp machine tradition. Is there the possibility that the OS code will be so mutable that it is impossible to predict if an application will run on any one installation? Will there be any posix-like standard to guarantee compatibility?
panic 11y ago
Does the language-based security model of 3L mean that all programs must be written in Lisp? Single-language systems tend to miss out on a lot of cross-platform software (which is typically written in or bootstrapped using C).
- thecombjelly 11y ago
  (creator of 3L) No. Almost any language we use today can run on 3L and transpilers are planned for the future of 3L. The security features will still be provided via the underlying lisp though. Technically the same security mechanism could work for most languages but it is much simpler, easier, and more natural to implement in lisp. Even languages like C can be supported safely via memory emulation (although right now that would mean it runs quite slowly but likely, although no guarantee, it will run well enough in the future as hardware becomes more powerful).
eklavya 11y ago
I have always been curious about Lisps but the lack of static typing turns me off. Sure you can have a type system (typed racket) but it doesn't feel organic, also then why not use any other statically typed language (haskell, scala, ..). With all the hype (very likely well founded) surrounding Lisps, I feel like I am missing out.
- yarrel 11y ago
  http://clhs.lisp.se/Body/d_type.htm http://clhs.lisp.se/Body/d_type.htm
- efnx 11y ago
  House, an experimental Haskell operating system is exactly that. If I had a lot of time on my hands I would love to revive that project. https://en.m.wikipedia.org/wiki/House_(operating_system) https://en.m.wikipedia.org/wiki/House_(operating_system)
  - tathougies 11y ago
    (Shameless plug follows) Take a look at my own Haskell operating system project Hos (https://github.com/tathougies/hos https://github.com/tathougies/hos). Written in Haskell 98 using the JHC compiler (a bit outdated, but makes it easy to generate ANSI C from Haskell sources). Currently supports multi-tasking, interrupts, paging, garbage collection. Has the beginnings of a simple user-space ATA driver. Patches welcome. E-mail me at travis@athougies.net if you want help setting up a development environment!
    - iso-8859-1 11y ago
      Would be nice if you could get Hos listed on https://wiki.haskell.org/Applications_and_libraries/Operating_system https://wiki.haskell.org/Applications_and_libraries/Operatin... ...
- cle 11y ago
  http://3lproject.org/features http://3lproject.org/features Some of that is vastly simpler to do with a dynamic language. "You can boot into a REPL (Read Evaluate Print Loop) and can inspect or rewrite any part of the OS as you are using it." As far as I know, nobody has figured out a way to have your cake and eat it too with static typing. Some of the simple things that lisps have been doing for decades are very difficult if not impossible to do with statically compiled languages.
unsatchmo 11y ago
"Windows, Linux, Mac OS X, and the BSDs we have today are still built using the architecture developed when computing resources were limited even though it is no longer technically necessary." Nonsense! Why would you waste any resources on infrastructure when you could use them for applications. Every cycle that your garbage collector runs is a cycle that the cpu could have been sleeping and saving battery life. This concept that we don't need to care about resources anymore is absolute poppycock.
- chipsy 11y ago
  The advantage isn't really in that resources can be wasted by adding a feature like garbage collection, but that if you are going to have features like that in your programs, building it in at the OS level is a better way to go about it, as it aggregates the resulting issues in one place, where it can be better managed(at a whole-system level).
- nlawalker 11y ago
  >> Why would you waste any resources on infrastructure when you could use them for applications. For the same reason we waste them on application frameworks instead of more interesting computations - because applications are written by humans, who take a long time to do it and are notoriously bad at it. Every cycle your garbage collector runs supports the existence of software that might not exist yet, or ever, without it.
  - unsatchmo 11y ago
    I am all for advanced memory management in application frameworks. The OS however isn't something you're going to rewrite very often so work harder and squeeze out efficiency so that your apps get them.
    - rickycook 11y ago
      why not? if it's all hot swappable and development at that level is as easy as in user space
    - 11y ago
peterburkimsher 11y ago
There is a market for a new OS, but security alone is not enough reason to make people switch. Supporting Unicode would affect users a lot more. I wish I saw an OS with good support for Chinese.
- DigitalJack 11y ago
  I would have thought the Chinese would have built one.
- thecombjelly 11y ago
  3L will support Unicode from the very bottom up.
- astrange 11y ago
  Are the Chinese not happy with the ones they have?
onre 11y ago
"Another change since the original OSes were initially being built is the need for security" I do not know what's referred to as "original OSes" but many old operating systems had actually pretty decent security architecture. MULTICS and VMS, for example, and many others too.
- thecombjelly 11y ago
  It's true that some did. I was more referring to the history of most of the major OSes we use today and specifically when it came to the first microcomputers. Generally when they were first being used (and the OS was being expanded) security was not much of a concern.
rodgerd 11y ago
So OSes designed in the 70s are bad[1] but we'll use a language designed in the 50s because it's good. Old stuff is bad except when we like it, I guess. [1] And quite dishonest to lump WinNT in with that...
- nickpsecurity 11y ago
  Architecture designed in the 70's to run common stuff on terrible computers was necessarily bad. There was solid architecture from 1960's-1980's that was great for management, reliability, security, and so on. HW was too expensive. The alternatives flourished on minicomputers and personal computers with tons of software built on them. Legacy effects took over from there with nobody able or willing to re-engineer everything from scratch or even existing, complex code. So, yes they suck. Yes superior architectures or languages of the time are worth copying or adopting over the knockoffs created due to hardware deficiencies. This is often even provable where one can look at problems that crop up in technology X caused by design point Y that's prevented by alternative design Z. And one of those alternatives was from 1961. ;)
- thecombjelly 11y ago
  OSes designed in the 70s aren't and weren't necessarily bad. It just depends on the circumstances. When they were written they fit the times quite well. But things have changed since then. Lisps have also changed since it was first created. Unlike many languages lisp has evolved tremendously over the years and has benefited from a tons of language research. The lisps we have today are much more than we had in the 60s.
  - TheCondor 11y ago
    Do you have any concrete examples of things that no longer fit well?
nickpsecurity 11y ago
Very interesting project, Thomas. Here's one for inspiration: http://www.symbolics-dks.com/Genera-why-1.htm http://www.symbolics-dks.com/Genera-why-1.htm About time we have something modern that can do everything in that ad that's relevant today. I want my live-updating, live-fixing, per-function compiling, whole-system debugging, crash-resistant OS. If they could do it in the 80's then... you know... ;)
- unsatchmo 11y ago
  This is the best argument for a High Level Language OS that I've seen yet.
  - nickpsecurity 11y ago
    Thank you!
- thecombjelly 11y ago
  Thanks. :) Genera (and other lisp OSes of the time) proved a lot of what 3L is building on. Unfortunately at the time the hardware required to run genera was expensive. Today the atmosphere is different.
thegenius2000 11y ago
I know (with software) to never say never, but to be quite frank, I'm content with the current Unix paradigm, and I haven't seen a compelling reason why any new OS would be fundamentally superior. But I would absolutely love to be proven wrong... http://www.catb.org/esr/writings/taoup/html/plan9.html http://www.catb.org/esr/writings/taoup/html/plan9.html
Yaggo 11y ago
Entirely new OS would be a waste of resources, but some big player could donate few millions to boost the development of Wayland. What the markets lacks is a system with openness / hardware support of Linux combined with solid graphical layer of OS X. Or maybe AOSP could transform into true open source (culturally) project?
jorangreef 11y ago
Too much network and filesystem data moves through an OS before the user application can work with it. As data bandwidth increases, this becomes a bottleneck. It would be great if an OS could push this out to userspace and focus on zero-copy stacks for network and disk.
- thecombjelly 11y ago
  3L does do this. You can share as much data between the OS and user applications as you want. The OS doesn't really mediate anything so there is no overhead from copying data or messages between processes or enforcing some security mechanism. The language runtime enforces the references that are available to some code that gets run but that's it and that's all that is needed. You really don't need a separate OS program facilitating everything to serve as an OS.
  - jorangreef 11y ago
    Thanks, I looked through the 3L website and features section but didn't realize that from there.
    - thecombjelly 11y ago
      Yeah sorry my fault really. I'm working on improving the organization of the site and I haven't had near enough time to write everything down in an intelligible fashion. Most things are still in my notes or waiting to be extracted from various papers.
sedatk 11y ago
The article is full of gold: - OS's were designed for limited resources back in time. Now it's time to design something for unlimited resources. - Attempting to retrofit a new security model to a current OS would require everything would be rewritten so we decided to rewrite an OS, which will require everything to be rewritten for it, including the OS this time. - Sandboxing, local-scope FS access, user controlled permissions have never been done before, so we're doing those. - Runtime analysis will give out better performance - Compilers have always been hindered by OS design, so our new OS will break the barriers of compilers (finally! ) (and more!)
- eloisant 11y ago
  When they say we no longer have to design for limited resources, I really want to know what we can gain from dropping that constraint. Do I, as a user of the OS gain anything, or is it just easier for the kernel developers to not have to care about that?
  - thecombjelly 11y ago
    > Do I, as a user of the OS gain anything, or is it just easier for the kernel developers to not have to care about that? Even if you don't gain anything directly you still gain significantly indirectly. Improvements in the speed and quality of kernel development means end users see new features quicker and bugs fixed sooner. It also makes it easier for CS researchers to try things out and make large advancements that can be easily applied to the OS.
    - capitalsigma 11y ago
      > Improvements in the speed...of kernel development means end users see new features quicker When is the last time you needed a feature that could only be provided by your kernel? > It also makes it easier for CS researchers to try things out How are they going to try things out if your OS doesn't support any of the techniques they're interested in (i.e. novel ways of getting at hardware, novel ways of doing virtual memory, etc)? Your premise is that life will be better because we won't have to deal with the ugliness of talking directly to hardware in C/C++/assembler, so what room does that leave for research? Also I just want to tag on a question based on one of your reddit posts: > One example is that 3L can save the entire system as you are using it. This means that if the power gets cut or the battery dies you can (instantly) reboot the system and it will be like you never lost power. You realize that this sort of thing is a huge area of current research, right? Because it's a really, really hard problem?
hasenj 11y ago
Resources aren't really that cheap. Just try running a video game on a Macbook Air. It doesn't work.
- quartzli 11y ago
  @thecombjelly Can programmer "touch" PreScheme in 3L?
  - thecombjelly 11y ago
    Right now, not really. PreScheme currently compiles to C. Eventually the goal is to create an assembly backend and if that were combined with incremental compilation features you could manipulate it at runtime to some degree.
thecombjelly 11y ago
I'm the creator of 3L so feel free to ask me anything or point out how it will never work and is a terrible idea and is nothing new. :)
- quartzli 11y ago
  What is the reason you decide the "Fully Source Mapped" feature? What's the advantage and disadvantage? Thanks.
  - thecombjelly 11y ago
    The only major disadvantage is it takes time to develop. The advantage is it makes development of the OS easier and application debugging easier. It is also makes future proofing the OS more robust. A program that can rewrite or inspect other programs is very powerful, in the same way lisp macros are. One concrete advantage of this is you could write a debugger as a user application, it wouldn't have to be integrated in to the language runtime.
    - hvidgaard 11y ago
      > A program that can rewrite or inspect other programs is very powerful And is a security nightmare. If I have an executeable, I have verified, I don't want the instructions to be altered in any way. In fact I prefer if it is impossible from a hardware level by marking them as executeable bits.
      - thecombjelly 11y ago
        This still falls within the 3L security model though. A program can only do that if you, the user, allows it. Also, 3L runs in two modes: development or secure. In secure mode you can completely disable manipulation of source code (or strip it out) and mark all code as read only and have the system verify the code hasn't been modified when it is run.
        11y ago
bronlund 11y ago
Yes! Please make it so you don't have to be time travelling cyborg in order to sort files the way you like. You have files right?
- thecombjelly 11y ago
  The "In Depth"[0] page addresses that a tiny bit. In summary, 3L drops explicit support for filesystems. Instead the system just provides a mapping of labels to storage blocks. From there those blocks can be utilized in any fashion you like. A filesystem library can be written to manage its set of blocks as it pleases. A database can skip the overhead of a filesystem and directly access the blocks to improve performance. But yes, assuming you have imported the filesystem library (which has probably already been done assuming you are at the REPL) then you can process files as regular lists of lisp data (or in whatever language you want really). Not sure that is really that special though since you can already do that. [0] http://3lproject.org/in-depth http://3lproject.org/in-depth
deleted 11y ago
[deleted]
bradley_long 11y ago
Keep going! To make a new OS successful, there are too many difficulties. Existing competitors, acceptance of consumers and cost of shifting from the old to the new. We are happy to see a new OS. Hope it can go well!

84 points by thecombjelly 11y ago

When I read this I raised my eyebrow. I hardly think that Lisp (1958) can be the base of the operating system of the future. Their texts are full of bold statements about how easy and natural lisp is and how everything else sucks. Nevertheless it's interesting to look at their references: http://3lproject.org/references http://3lproject.org/references I understand than in short, the security model is about how to make sure a process does not do more than what it's supposed to. In this case it's lisp environments that provide the necessary sandboxing.

na85 11y ago
That Lisp began in 1958 is mostly irrelevant. It (Common Lisp) is a fully-modern, Turing-complete language with a first-rate object system and supports all programming paradigms. On top of that the implementations have all been regularly updated over the years and many are extremely mature and well-optimized. Do you equate newness with quality, as seems to be common on HN? Perhaps an OS written entirely in this month's JavaScript framework, deployed on Docker, using monads-as-a-service or whatever the hipsters have come up with lately would be more your cup of tea.
- rodgerd 11y ago
  > Do you equate newness with quality, as seems to be common on HN? When the link's thesis is "these OSes design years ago are old and therefore bad" it is hardly unreasonable to point out the language the author prefers is even older.
  - pjmlp 11y ago
    Because the UNIX based OS are based on PDP-11 architecture with the compromises required to fit in such systems. Whereas already in the 60s, we had hardware systems like the Burroughs b5000 that could be fully programmed in Extended Algol and Espol, with zero Assembly. The idea, as I see it, to base modern OSes in better older architectures that were lost to the mists of time. Actually the mobile OSes, with their memory safe languages seem to already be moving into this direction.
  - na85 11y ago
    >these OSes design years ago are old and therefore bad That's not the link's thesis. Equally flippant but more correct would be to say "these OSes were written in memory-unsafe languages because hardware limitations demanded it back then, therefore they are bad"
- anentropic 11y ago
  a Turing-complete programming language? How novel!
  - na85 11y ago
    My point was that there isn't anything you can do in Golang or Rust or any other FOTM language that you can't do in Lisp.
Animats 11y ago
If they came up with a good security model for applications, that would be a big step forward. Capabilities are a mechanism, not a model. Somebody has to decide who gets which capabilities. That mechanism needs to be understandable by the user. For example, Privileges for a game: - Can use full screen. - Can read own content installed with app. - Can read/write persistent storage associated with app. - Can communicate with domain "thisgame.ea.com". - Cannot run in background, access address book, ... Word processor: - Can access files mentioned in user file dialogs (file dialogs being run from a separate process). - Can transmit requests to send mail to app "mail". For any app which accesses information outside its own content and own persistent storage, the user should have the options "allow", "deny", and "fake". If a program asks for address book access and won't run without it, you can give it the "fake" option, which gives it access to a dummy address book. (I've seen something like this demoed for jailbroken Android phones.)
- arthurfm 11y ago
  > I've seen something like this demoed for rooted Android phones The amount of data XPrivacy [1] can fake is impressive. It's a shame it isn't a built-in feature. [1] https://github.com/M66B/XPrivacy#features https://github.com/M66B/XPrivacy#features
  - Animats 11y ago
    The biggest problem is the UNIX and Windows tradition that programs have all the rights of the user running them. Today, they should start out with the rights to read their own installed stuff, access their own private read/write prefs area, and use temp files. Anything more than that is at the discretion of the user.
nickpsecurity 11y ago
LISP side of things already has a bit of work toward a security model and components for robust implementation: Security Kernel https://www.kyleisom.net/downloads/security_kernel.pdf https://www.kyleisom.net/downloads/security_kernel.pdf Formally-verified Scheme http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=881F8DEAC1EE14DC88EAB49ED026CB05?doi=10.1.1.36.9989&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=881... The features could certainly be worth it. Some aspects of the Genera experience, esp productivity & holistic debugging, still aren't in modern flows: http://www.symbolics-dks.com/Genera-why-1.htm http://www.symbolics-dks.com/Genera-why-1.htm So, who knows where it will go. Betting on a semi-functional language with prior high-security and high-productivity work is a better start than most projects like this, though.

abstractbeliefs 11y ago

Fascinating project. I'm glad that alongside the philosophy of this being a FOSS endeavor, there's the practical aspect of this being adjustable on the fly by users in the old Lisp machine tradition. Though I'm sure it's too early for outsiders to shape the PoC code, is it available anywhere for us all to follow along?

thecombjelly 11y ago
> is it available anywhere for us all to follow along? A mailing list to follow development will be available soon unfortunately I haven't had time to get it up and running yet. edit: but check back to 3lproject.org soon or the kickstarter page. If you have kickstarted it I'll also make sure to send out an update.
mchahn 11y ago
> there's the practical aspect of this being adjustable on the fly by users in the old Lisp machine tradition. Is there the possibility that the OS code will be so mutable that it is impossible to predict if an application will run on any one installation? Will there be any posix-like standard to guarantee compatibility?

panic 11y ago

Does the language-based security model of 3L mean that all programs must be written in Lisp? Single-language systems tend to miss out on a lot of cross-platform software (which is typically written in or bootstrapped using C).

thecombjelly 11y ago
(creator of 3L) No. Almost any language we use today can run on 3L and transpilers are planned for the future of 3L. The security features will still be provided via the underlying lisp though. Technically the same security mechanism could work for most languages but it is much simpler, easier, and more natural to implement in lisp. Even languages like C can be supported safely via memory emulation (although right now that would mean it runs quite slowly but likely, although no guarantee, it will run well enough in the future as hardware becomes more powerful).

eklavya 11y ago

I have always been curious about Lisps but the lack of static typing turns me off. Sure you can have a type system (typed racket) but it doesn't feel organic, also then why not use any other statically typed language (haskell, scala, ..). With all the hype (very likely well founded) surrounding Lisps, I feel like I am missing out.

yarrel 11y ago
http://clhs.lisp.se/Body/d_type.htm http://clhs.lisp.se/Body/d_type.htm
efnx 11y ago
House, an experimental Haskell operating system is exactly that. If I had a lot of time on my hands I would love to revive that project. https://en.m.wikipedia.org/wiki/House_(operating_system) https://en.m.wikipedia.org/wiki/House_(operating_system)
- tathougies 11y ago
  (Shameless plug follows) Take a look at my own Haskell operating system project Hos (https://github.com/tathougies/hos https://github.com/tathougies/hos). Written in Haskell 98 using the JHC compiler (a bit outdated, but makes it easy to generate ANSI C from Haskell sources). Currently supports multi-tasking, interrupts, paging, garbage collection. Has the beginnings of a simple user-space ATA driver. Patches welcome. E-mail me at travis@athougies.net if you want help setting up a development environment!
  - iso-8859-1 11y ago
    Would be nice if you could get Hos listed on https://wiki.haskell.org/Applications_and_libraries/Operating_system https://wiki.haskell.org/Applications_and_libraries/Operatin... ...
cle 11y ago
http://3lproject.org/features http://3lproject.org/features Some of that is vastly simpler to do with a dynamic language. "You can boot into a REPL (Read Evaluate Print Loop) and can inspect or rewrite any part of the OS as you are using it." As far as I know, nobody has figured out a way to have your cake and eat it too with static typing. Some of the simple things that lisps have been doing for decades are very difficult if not impossible to do with statically compiled languages.

unsatchmo 11y ago

"Windows, Linux, Mac OS X, and the BSDs we have today are still built using the architecture developed when computing resources were limited even though it is no longer technically necessary." Nonsense! Why would you waste any resources on infrastructure when you could use them for applications. Every cycle that your garbage collector runs is a cycle that the cpu could have been sleeping and saving battery life. This concept that we don't need to care about resources anymore is absolute poppycock.

chipsy 11y ago
The advantage isn't really in that resources can be wasted by adding a feature like garbage collection, but that if you are going to have features like that in your programs, building it in at the OS level is a better way to go about it, as it aggregates the resulting issues in one place, where it can be better managed(at a whole-system level).
nlawalker 11y ago
>> Why would you waste any resources on infrastructure when you could use them for applications. For the same reason we waste them on application frameworks instead of more interesting computations - because applications are written by humans, who take a long time to do it and are notoriously bad at it. Every cycle your garbage collector runs supports the existence of software that might not exist yet, or ever, without it.
- unsatchmo 11y ago
  I am all for advanced memory management in application frameworks. The OS however isn't something you're going to rewrite very often so work harder and squeeze out efficiency so that your apps get them.
  - rickycook 11y ago
    why not? if it's all hot swappable and development at that level is as easy as in user space
  - 11y ago

peterburkimsher 11y ago

There is a market for a new OS, but security alone is not enough reason to make people switch. Supporting Unicode would affect users a lot more. I wish I saw an OS with good support for Chinese.

DigitalJack 11y ago
I would have thought the Chinese would have built one.
thecombjelly 11y ago
3L will support Unicode from the very bottom up.
astrange 11y ago
Are the Chinese not happy with the ones they have?

onre 11y ago

"Another change since the original OSes were initially being built is the need for security" I do not know what's referred to as "original OSes" but many old operating systems had actually pretty decent security architecture. MULTICS and VMS, for example, and many others too.

thecombjelly 11y ago
It's true that some did. I was more referring to the history of most of the major OSes we use today and specifically when it came to the first microcomputers. Generally when they were first being used (and the OS was being expanded) security was not much of a concern.

rodgerd 11y ago

So OSes designed in the 70s are bad[1] but we'll use a language designed in the 50s because it's good. Old stuff is bad except when we like it, I guess. [1] And quite dishonest to lump WinNT in with that...

nickpsecurity 11y ago
Architecture designed in the 70's to run common stuff on terrible computers was necessarily bad. There was solid architecture from 1960's-1980's that was great for management, reliability, security, and so on. HW was too expensive. The alternatives flourished on minicomputers and personal computers with tons of software built on them. Legacy effects took over from there with nobody able or willing to re-engineer everything from scratch or even existing, complex code. So, yes they suck. Yes superior architectures or languages of the time are worth copying or adopting over the knockoffs created due to hardware deficiencies. This is often even provable where one can look at problems that crop up in technology X caused by design point Y that's prevented by alternative design Z. And one of those alternatives was from 1961. ;)
thecombjelly 11y ago
OSes designed in the 70s aren't and weren't necessarily bad. It just depends on the circumstances. When they were written they fit the times quite well. But things have changed since then. Lisps have also changed since it was first created. Unlike many languages lisp has evolved tremendously over the years and has benefited from a tons of language research. The lisps we have today are much more than we had in the 60s.
- TheCondor 11y ago
  Do you have any concrete examples of things that no longer fit well?

nickpsecurity 11y ago

Very interesting project, Thomas. Here's one for inspiration: http://www.symbolics-dks.com/Genera-why-1.htm http://www.symbolics-dks.com/Genera-why-1.htm About time we have something modern that can do everything in that ad that's relevant today. I want my live-updating, live-fixing, per-function compiling, whole-system debugging, crash-resistant OS. If they could do it in the 80's then... you know... ;)

unsatchmo 11y ago
This is the best argument for a High Level Language OS that I've seen yet.
- nickpsecurity 11y ago
  Thank you!
thecombjelly 11y ago
Thanks. :) Genera (and other lisp OSes of the time) proved a lot of what 3L is building on. Unfortunately at the time the hardware required to run genera was expensive. Today the atmosphere is different.

thegenius2000 11y ago

I know (with software) to never say never, but to be quite frank, I'm content with the current Unix paradigm, and I haven't seen a compelling reason why any new OS would be fundamentally superior. But I would absolutely love to be proven wrong... http://www.catb.org/esr/writings/taoup/html/plan9.html http://www.catb.org/esr/writings/taoup/html/plan9.html

Yaggo 11y ago

Entirely new OS would be a waste of resources, but some big player could donate few millions to boost the development of Wayland. What the markets lacks is a system with openness / hardware support of Linux combined with solid graphical layer of OS X. Or maybe AOSP could transform into true open source (culturally) project?

jorangreef 11y ago

Too much network and filesystem data moves through an OS before the user application can work with it. As data bandwidth increases, this becomes a bottleneck. It would be great if an OS could push this out to userspace and focus on zero-copy stacks for network and disk.

thecombjelly 11y ago
3L does do this. You can share as much data between the OS and user applications as you want. The OS doesn't really mediate anything so there is no overhead from copying data or messages between processes or enforcing some security mechanism. The language runtime enforces the references that are available to some code that gets run but that's it and that's all that is needed. You really don't need a separate OS program facilitating everything to serve as an OS.
- jorangreef 11y ago
  Thanks, I looked through the 3L website and features section but didn't realize that from there.
  - thecombjelly 11y ago
    Yeah sorry my fault really. I'm working on improving the organization of the site and I haven't had near enough time to write everything down in an intelligible fashion. Most things are still in my notes or waiting to be extracted from various papers.

sedatk 11y ago

The article is full of gold: - OS's were designed for limited resources back in time. Now it's time to design something for unlimited resources. - Attempting to retrofit a new security model to a current OS would require everything would be rewritten so we decided to rewrite an OS, which will require everything to be rewritten for it, including the OS this time. - Sandboxing, local-scope FS access, user controlled permissions have never been done before, so we're doing those. - Runtime analysis will give out better performance - Compilers have always been hindered by OS design, so our new OS will break the barriers of compilers (finally! ) (and more!)

eloisant 11y ago
When they say we no longer have to design for limited resources, I really want to know what we can gain from dropping that constraint. Do I, as a user of the OS gain anything, or is it just easier for the kernel developers to not have to care about that?
- thecombjelly 11y ago
  > Do I, as a user of the OS gain anything, or is it just easier for the kernel developers to not have to care about that? Even if you don't gain anything directly you still gain significantly indirectly. Improvements in the speed and quality of kernel development means end users see new features quicker and bugs fixed sooner. It also makes it easier for CS researchers to try things out and make large advancements that can be easily applied to the OS.
  - capitalsigma 11y ago
    > Improvements in the speed...of kernel development means end users see new features quicker When is the last time you needed a feature that could only be provided by your kernel? > It also makes it easier for CS researchers to try things out How are they going to try things out if your OS doesn't support any of the techniques they're interested in (i.e. novel ways of getting at hardware, novel ways of doing virtual memory, etc)? Your premise is that life will be better because we won't have to deal with the ugliness of talking directly to hardware in C/C++/assembler, so what room does that leave for research? Also I just want to tag on a question based on one of your reddit posts: > One example is that 3L can save the entire system as you are using it. This means that if the power gets cut or the battery dies you can (instantly) reboot the system and it will be like you never lost power. You realize that this sort of thing is a huge area of current research, right? Because it's a really, really hard problem?

hasenj 11y ago

Resources aren't really that cheap. Just try running a video game on a Macbook Air. It doesn't work.

quartzli 11y ago
@thecombjelly Can programmer "touch" PreScheme in 3L?
- thecombjelly 11y ago
  Right now, not really. PreScheme currently compiles to C. Eventually the goal is to create an assembly backend and if that were combined with incremental compilation features you could manipulate it at runtime to some degree.

thecombjelly 11y ago

I'm the creator of 3L so feel free to ask me anything or point out how it will never work and is a terrible idea and is nothing new. :)

quartzli 11y ago
What is the reason you decide the "Fully Source Mapped" feature? What's the advantage and disadvantage? Thanks.
- thecombjelly 11y ago
  The only major disadvantage is it takes time to develop. The advantage is it makes development of the OS easier and application debugging easier. It is also makes future proofing the OS more robust. A program that can rewrite or inspect other programs is very powerful, in the same way lisp macros are. One concrete advantage of this is you could write a debugger as a user application, it wouldn't have to be integrated in to the language runtime.
  - hvidgaard 11y ago
    > A program that can rewrite or inspect other programs is very powerful And is a security nightmare. If I have an executeable, I have verified, I don't want the instructions to be altered in any way. In fact I prefer if it is impossible from a hardware level by marking them as executeable bits.
    - thecombjelly 11y ago
      This still falls within the 3L security model though. A program can only do that if you, the user, allows it. Also, 3L runs in two modes: development or secure. In secure mode you can completely disable manipulation of source code (or strip it out) and mark all code as read only and have the system verify the code hasn't been modified when it is run.
      - 11y ago

bronlund 11y ago

Yes! Please make it so you don't have to be time travelling cyborg in order to sort files the way you like. You have files right?

thecombjelly 11y ago
The "In Depth"[0] page addresses that a tiny bit. In summary, 3L drops explicit support for filesystems. Instead the system just provides a mapping of labels to storage blocks. From there those blocks can be utilized in any fashion you like. A filesystem library can be written to manage its set of blocks as it pleases. A database can skip the overhead of a filesystem and directly access the blocks to improve performance. But yes, assuming you have imported the filesystem library (which has probably already been done assuming you are at the REPL) then you can process files as regular lists of lisp data (or in whatever language you want really). Not sure that is really that special though since you can already do that. [0] http://3lproject.org/in-depth http://3lproject.org/in-depth

deleted 11y ago

[deleted]

bradley_long 11y ago

Keep going! To make a new OS successful, there are too many difficulties. Existing competitors, acceptance of consumers and cost of shifting from the old to the new. We are happy to see a new OS. Hope it can go well!