8 ms·

Systems Software Research is Irrelevant (2000) [pdf]

64 points by panic 8y ago

dang 8y ago
Quite a few previous threads: https://hn.algolia.com/?query=Systems%20Software%20Research%20is%20Irrelevant%20points%3E3&sort=byDate&dateRange=all&type=story&storyText=false&prefix=false&page=0 https://hn.algolia.com/?query=Systems%20Software%20Research%...
meuk 8y ago
I think the problem is that current OSes are too complicated. What does a common user really want? I'd say text editing, making presentations, multimedia, and browsing. While these sound like simple things, text editing usually happens in a Word-like format, which - from what I've heard - is ridiculously complex. The same thing holds for browsing (simple browsers don't exist anymore, and many use the same back-end for rendering). For multimedia, USB is very important, which is a very complicated format to use correctly. Then, there are tens of codecs and formats for multimedia, each with their own idiosyncrasies and ambiguities in the specs (or deviations from the spec that are so common that they are expected nowadays). The case is even worse when your OS is supposed to run on different hardware. I think this is the main reason that Apple, in general, has a much better UX: They only have to support a limited set of hardware. Casey Murati has a great rant about this: https://www.youtube.com/watch?v=kZRE7HIO3vk https://www.youtube.com/watch?v=kZRE7HIO3vk
- bcaa7f3a8bbc 8y ago
  I agree. I think the lack of new operating systems since late-90s can be (partially) attributed to the simple fact that modern computing world is so fully-developed, we have the computers for everything, and enjoy all the state-of-art features and functionalities. You already have, well, all the things you have on your personal computer. Although desktop apps are getting crappier and crappier as always (which is another story), but some parts of underlying infrastructure are created and only understood by the best developers in the world (high-performance TCP stack, 802.11n, multimedia, 3D graphics, TLSv1.3, PCI-e, USB 3.0, virtualization, exploit mitigation, web browser engines, C10K-free web server). By creating a new system, you lost everything immediately and render the computer less useful, making it less attractive for both hackers and academic/industrial researchers alike. Back in the early days, not too many things were being done on computers, and you had fewer things to lose. Want to play Space War on PDP-7, but don't even have an OS? Then Unix! Want to do some real work on 80s home computers? BASIC! Micro-Soft! When Linus purchased his computer, he found that the computer only runs DOS and was effectively useless for him. So he ordered a copy of Minix, meanwhile playing a DOS game to kill his time. After Minix arrived he eventually got a usable programming environment, was able to got online to Usenet with a modem, and also got the Minix filesystem which was suffering from performance issues at that time. This is everything he had for his computing. Then he decided to create his own operating system. He wrote the kernel, ported glibc, bash, gcc, and the system boots, later he added modem support. Now he could do everything he was able to do on Minix. That's it! Naturally, a few years later, it became a community project and gained huge momentum. Perhaps the early-90s was the last era, when hackers can "just" create a operating system in a tinker's way. On the other hand, new OS are still being constantly created on platforms which don't have the burdens of workstations/servers, we still see many hobbyist/research/practical systems on microcontrollers every year. As they became more and more powerful, then it also runs Linux and the story ends there... I think this is also one reason that retrocomputing is getting more and more popular, it brings back the "playground" nature of early computer systems, unlike the "factory" nature of modern ones.
  - pjmlp 8y ago
    You can still tinker away like the 90's on Arduino, ESP* like boards. The problem is the effort to actually build something half as usable as an existing free OS, so most just give up.
    - bcaa7f3a8bbc 8y ago
      Thus I mentioned "microcontrollers".
- wtallis 8y ago
  > I think this is the main reason that Apple, in general, has a much better UX: They only have to support a limited set of hardware. People assume this to be the case, without actually analyzing what hardware actually needs to be supported by operating systems these days. For internal components, most of the stuff in a PC is pretty standardized: AHCI and NVMe for storage, EHCI and XHCI for USB, HD Audio for sound (and it's almost always a Realtek codec). Apple doesn't have any significant advantage in any of those areas. They still have to support two out of three GPU vendors, the top two or three NIC vendors, and maintain a driver infrastructure that supports loading third-party drivers for any of the above when there are exceptions. (Sure, the expandable Mac Pro may be long-dead, but Thunderbolt docks and enclosures enable the same variety of components). Apple's real advantage seems to be that they don't have to try to work around anybody else's broken motherboard firmware. Microsoft could take the same stance and declare it to be Lenovo's problem if your Thinkpad's power management is broken. Microsoft probably should have changed strategy there when the industry switched to UEFI.
  - chrisseaton 8y ago
    Devices have bugs, corner cases, incomplete implementations of standards, and standards can be ambiguous. The advantage isn’t that Apple have to implement less interfaces, it’s that they have to test with less devices. I’m sure on paper Linux works with all my devices. In practice it does not. That’s the gap Apple close with their hardware.
yourbandsucks 8y ago
I wonder if Pike would still stand by that paper now, or if it was a product of his personal headspace in the immediate aftermath of plan 9 not quite catching on.
- pjmlp 8y ago
  Inferno was Pike's last OS, not Plan 9.
Animats 8y ago
And then everything changed. System software became about managing large numbers of machines. Good new languages were developed. Interprocess communication finally became mainstream, even if too much of it was over HTTP. The whole Javascript ecosystem appeared. Still too much Linux, though. Things running in containers on in VMs need far less of an OS, and a lot less OS state.
- yourbandsucks 8y ago
  I took a brief look at redox, which isn't a super serious project, and noticed that they were at that point only set up to run as a virtual machine. No actual hardware supported. I'm sure this was for expediency in their case... But then I thought, why not? It seems kind of antiquated for a server os to worry about actual hardware in this day and age. Funny how things turn out.
  - adrianN 8y ago
    That's pretty much how every OS is developed at least since QEMU was made. Why would you risk wrecking actual hardware with your toy OS and go through a code->compile->install->reboot cycle to test?
  - jononor 8y ago
    Yeah, its a lot of work to make an HAL with support for all desirable platforms. Just use Linux with KVM and the virtualized devices become the HAL, best-in-class hardware support already included.
- dunkelheit 8y ago
  I think most of the observations of that talk still stand. > System software became about managing large numbers of machines. Yes, but is academic research on these topics relevant? Most of the progress on infrastructure software has come from the attempts of internet companies to cope with their big data problems. The people who implemented these systems have all received their phds from academic CS institutions and they used bits and pieces of existing distributed systems research, but they did their innovative work in the context of commercial companies, not academic CS departments. > Good new languages were developed. And what are they? A quick look at https://tiobe.com/tiobe-index/ https://tiobe.com/tiobe-index/ (for the lack of a better resource) corroborates the thesis that the bulk of software is developed using the same old boring languages. Yes, they have evolved, but not much. Golang is, well, not exactly innovative. Rust still has a long way to go to be called a mainstream language. > Things running in containers on in VMs need far less of an OS, and a lot less OS state. Unikernels exist but is there a compelling non-niche use case for them? Docker acquired Unikernel systems and that's what we got so far: https://blog.docker.com/2016/05/docker-unikernels-open-source/ https://blog.docker.com/2016/05/docker-unikernels-open-sourc... These are all great examples confirming the thesis that producing good relevant systems research is hard. On the one hand we are piling up gross inefficiencies on top of decades old technology, so improving the current state of affairs should be easy, right? On the other, software systems are inherently very open systems with a lot of stakeholders, so doing any kind of successful "clean-slate redesign" is almost unthinkable.
kragen 8y ago
Consider this paper as the context for the creation of Golang, Kubernetes, MapReduce, Docker, Android, Dalvik, Chrome, V8, TraceMonkey, Splunk, paravirtualization — first as RTLinux and UserModeLinux and then as Xen —, AWS, Systemd, and iPython clustering, among other things. Systems software research became a lot more relevant.
- red_admiral 8y ago
  He's still mostly right about academia - over half the projects in your list are by Google. The best many universities can offer in the area of CS is to be a training school for the places where the real work is done. I hear lots of anecdotes from colleagues about academic research conferences in certain areas like "big data" where people are excitedly talking about "new" things that industry did years ago, but didn't publish in an academic journal. And then there's the people claiming they're doing "big data" because there's slightly over a million rows in their MySQL database.
  - jamesblonde 8y ago
    All state-of-the-art in Big Data from universities. Spark from UC Berkeley. Flink from TU Berlin. HopsFS from KTH Stockholm.
  - pjmlp 8y ago
    Most of the "industry" work on compilers and graphics wouldn't be possible with the actual research that was done by the universities. Yes the industry does pump a lot of money into research, but the people doing the real work are mostly doing some Msc or Phd at university XYZ. So it goes both ways. Just to cite the two areas that are my main hobby when doing computing related stuff outside boring enterprise work. As for OS, the universities are the only place left it seems, as all major vendors are very carefull when it doesn't bring short term profits, e.g. Midori.
  - kragen 8y ago
    In that paper, Pike wasn't talking about academia (primarily or exclusively) in the first place. I think when he wrote that paper he had spent his entire career at Bell Labs. He was lamenting the decline of the systems-research praxis that he, and his lab, had been such an important part of for so many years. There's an excellent reason he's working at Google now.
ameixaseca 8y ago
I heard similar arguments from a co-worker when talking about L4/QNX and OS research in general. "Nobody uses them / they don't work / where are they?" were the answers, all which have good replies if you google for 5 minutes. Hard to discuss with someone that considers what they know as everything that exists out there. It's a myoptic view of the present and the future. IMHO, this happens when one has more opinion than knowledge. Pike, unfortunately, seems to suffer from a similar problem.
everybodyknows 8y ago
Some inspiration here: >Be courageous. Try different things; experiment. Try to give a cool demo. >Measure success by ideas, not just papers and money. Make the industry want your work.

64 points by panic 8y ago

dang 8y ago

Quite a few previous threads: https://hn.algolia.com/?query=Systems%20Software%20Research%20is%20Irrelevant%20points%3E3&sort=byDate&dateRange=all&type=story&storyText=false&prefix=false&page=0 https://hn.algolia.com/?query=Systems%20Software%20Research%...

meuk 8y ago

I think the problem is that current OSes are too complicated. What does a common user really want? I'd say text editing, making presentations, multimedia, and browsing. While these sound like simple things, text editing usually happens in a Word-like format, which - from what I've heard - is ridiculously complex. The same thing holds for browsing (simple browsers don't exist anymore, and many use the same back-end for rendering). For multimedia, USB is very important, which is a very complicated format to use correctly. Then, there are tens of codecs and formats for multimedia, each with their own idiosyncrasies and ambiguities in the specs (or deviations from the spec that are so common that they are expected nowadays). The case is even worse when your OS is supposed to run on different hardware. I think this is the main reason that Apple, in general, has a much better UX: They only have to support a limited set of hardware. Casey Murati has a great rant about this: https://www.youtube.com/watch?v=kZRE7HIO3vk https://www.youtube.com/watch?v=kZRE7HIO3vk

bcaa7f3a8bbc 8y ago
I agree. I think the lack of new operating systems since late-90s can be (partially) attributed to the simple fact that modern computing world is so fully-developed, we have the computers for everything, and enjoy all the state-of-art features and functionalities. You already have, well, all the things you have on your personal computer. Although desktop apps are getting crappier and crappier as always (which is another story), but some parts of underlying infrastructure are created and only understood by the best developers in the world (high-performance TCP stack, 802.11n, multimedia, 3D graphics, TLSv1.3, PCI-e, USB 3.0, virtualization, exploit mitigation, web browser engines, C10K-free web server). By creating a new system, you lost everything immediately and render the computer less useful, making it less attractive for both hackers and academic/industrial researchers alike. Back in the early days, not too many things were being done on computers, and you had fewer things to lose. Want to play Space War on PDP-7, but don't even have an OS? Then Unix! Want to do some real work on 80s home computers? BASIC! Micro-Soft! When Linus purchased his computer, he found that the computer only runs DOS and was effectively useless for him. So he ordered a copy of Minix, meanwhile playing a DOS game to kill his time. After Minix arrived he eventually got a usable programming environment, was able to got online to Usenet with a modem, and also got the Minix filesystem which was suffering from performance issues at that time. This is everything he had for his computing. Then he decided to create his own operating system. He wrote the kernel, ported glibc, bash, gcc, and the system boots, later he added modem support. Now he could do everything he was able to do on Minix. That's it! Naturally, a few years later, it became a community project and gained huge momentum. Perhaps the early-90s was the last era, when hackers can "just" create a operating system in a tinker's way. On the other hand, new OS are still being constantly created on platforms which don't have the burdens of workstations/servers, we still see many hobbyist/research/practical systems on microcontrollers every year. As they became more and more powerful, then it also runs Linux and the story ends there... I think this is also one reason that retrocomputing is getting more and more popular, it brings back the "playground" nature of early computer systems, unlike the "factory" nature of modern ones.
- pjmlp 8y ago
  You can still tinker away like the 90's on Arduino, ESP* like boards. The problem is the effort to actually build something half as usable as an existing free OS, so most just give up.
  - bcaa7f3a8bbc 8y ago
    Thus I mentioned "microcontrollers".
wtallis 8y ago
> I think this is the main reason that Apple, in general, has a much better UX: They only have to support a limited set of hardware. People assume this to be the case, without actually analyzing what hardware actually needs to be supported by operating systems these days. For internal components, most of the stuff in a PC is pretty standardized: AHCI and NVMe for storage, EHCI and XHCI for USB, HD Audio for sound (and it's almost always a Realtek codec). Apple doesn't have any significant advantage in any of those areas. They still have to support two out of three GPU vendors, the top two or three NIC vendors, and maintain a driver infrastructure that supports loading third-party drivers for any of the above when there are exceptions. (Sure, the expandable Mac Pro may be long-dead, but Thunderbolt docks and enclosures enable the same variety of components). Apple's real advantage seems to be that they don't have to try to work around anybody else's broken motherboard firmware. Microsoft could take the same stance and declare it to be Lenovo's problem if your Thinkpad's power management is broken. Microsoft probably should have changed strategy there when the industry switched to UEFI.
- chrisseaton 8y ago
  Devices have bugs, corner cases, incomplete implementations of standards, and standards can be ambiguous. The advantage isn’t that Apple have to implement less interfaces, it’s that they have to test with less devices. I’m sure on paper Linux works with all my devices. In practice it does not. That’s the gap Apple close with their hardware.

yourbandsucks 8y ago

I wonder if Pike would still stand by that paper now, or if it was a product of his personal headspace in the immediate aftermath of plan 9 not quite catching on.

pjmlp 8y ago
Inferno was Pike's last OS, not Plan 9.

Animats 8y ago

And then everything changed. System software became about managing large numbers of machines. Good new languages were developed. Interprocess communication finally became mainstream, even if too much of it was over HTTP. The whole Javascript ecosystem appeared. Still too much Linux, though. Things running in containers on in VMs need far less of an OS, and a lot less OS state.

yourbandsucks 8y ago
I took a brief look at redox, which isn't a super serious project, and noticed that they were at that point only set up to run as a virtual machine. No actual hardware supported. I'm sure this was for expediency in their case... But then I thought, why not? It seems kind of antiquated for a server os to worry about actual hardware in this day and age. Funny how things turn out.
- adrianN 8y ago
  That's pretty much how every OS is developed at least since QEMU was made. Why would you risk wrecking actual hardware with your toy OS and go through a code->compile->install->reboot cycle to test?
- jononor 8y ago
  Yeah, its a lot of work to make an HAL with support for all desirable platforms. Just use Linux with KVM and the virtualized devices become the HAL, best-in-class hardware support already included.
dunkelheit 8y ago
I think most of the observations of that talk still stand. > System software became about managing large numbers of machines. Yes, but is academic research on these topics relevant? Most of the progress on infrastructure software has come from the attempts of internet companies to cope with their big data problems. The people who implemented these systems have all received their phds from academic CS institutions and they used bits and pieces of existing distributed systems research, but they did their innovative work in the context of commercial companies, not academic CS departments. > Good new languages were developed. And what are they? A quick look at https://tiobe.com/tiobe-index/ https://tiobe.com/tiobe-index/ (for the lack of a better resource) corroborates the thesis that the bulk of software is developed using the same old boring languages. Yes, they have evolved, but not much. Golang is, well, not exactly innovative. Rust still has a long way to go to be called a mainstream language. > Things running in containers on in VMs need far less of an OS, and a lot less OS state. Unikernels exist but is there a compelling non-niche use case for them? Docker acquired Unikernel systems and that's what we got so far: https://blog.docker.com/2016/05/docker-unikernels-open-source/ https://blog.docker.com/2016/05/docker-unikernels-open-sourc... These are all great examples confirming the thesis that producing good relevant systems research is hard. On the one hand we are piling up gross inefficiencies on top of decades old technology, so improving the current state of affairs should be easy, right? On the other, software systems are inherently very open systems with a lot of stakeholders, so doing any kind of successful "clean-slate redesign" is almost unthinkable.

kragen 8y ago

Consider this paper as the context for the creation of Golang, Kubernetes, MapReduce, Docker, Android, Dalvik, Chrome, V8, TraceMonkey, Splunk, paravirtualization — first as RTLinux and UserModeLinux and then as Xen —, AWS, Systemd, and iPython clustering, among other things. Systems software research became a lot more relevant.

red_admiral 8y ago
He's still mostly right about academia - over half the projects in your list are by Google. The best many universities can offer in the area of CS is to be a training school for the places where the real work is done. I hear lots of anecdotes from colleagues about academic research conferences in certain areas like "big data" where people are excitedly talking about "new" things that industry did years ago, but didn't publish in an academic journal. And then there's the people claiming they're doing "big data" because there's slightly over a million rows in their MySQL database.
- jamesblonde 8y ago
  All state-of-the-art in Big Data from universities. Spark from UC Berkeley. Flink from TU Berlin. HopsFS from KTH Stockholm.
- pjmlp 8y ago
  Most of the "industry" work on compilers and graphics wouldn't be possible with the actual research that was done by the universities. Yes the industry does pump a lot of money into research, but the people doing the real work are mostly doing some Msc or Phd at university XYZ. So it goes both ways. Just to cite the two areas that are my main hobby when doing computing related stuff outside boring enterprise work. As for OS, the universities are the only place left it seems, as all major vendors are very carefull when it doesn't bring short term profits, e.g. Midori.
- kragen 8y ago
  In that paper, Pike wasn't talking about academia (primarily or exclusively) in the first place. I think when he wrote that paper he had spent his entire career at Bell Labs. He was lamenting the decline of the systems-research praxis that he, and his lab, had been such an important part of for so many years. There's an excellent reason he's working at Google now.

ameixaseca 8y ago

I heard similar arguments from a co-worker when talking about L4/QNX and OS research in general. "Nobody uses them / they don't work / where are they?" were the answers, all which have good replies if you google for 5 minutes. Hard to discuss with someone that considers what they know as everything that exists out there. It's a myoptic view of the present and the future. IMHO, this happens when one has more opinion than knowledge. Pike, unfortunately, seems to suffer from a similar problem.

everybodyknows 8y ago

Some inspiration here: >Be courageous. Try different things; experiment. Try to give a cool demo. >Measure success by ideas, not just papers and money. Make the industry want your work.