4 ms·

Systems Software Research is Irrelevant by Rob Pike (2000)

39 points by uriel 17y ago

timf 17y ago
Written August 5th, 2000 "Irrelevant: Does not influence industry" Two years later came Xen, a systems research project that has definitely influenced industry.
- uriel 17y ago
  If Xen is the best you can come up as a result of research impacting industry in the last ten years... IBM had been doing basically the same for decades, so I'm not sure one could consider Xen particularly revolutionary. Edit: Also note that Xen in no way fulfills the definition of Systems Research, it is simply a new minor twist on a very old idea.
- ams6110 17y ago
  OK, to play devil's advocate, Xen is nothing new in the same way that Pike claims Linux is nothing new. Hypervisors and virutalization have been around since the late '60s.
  - Locke1689 17y ago
    Look up "paravirtualization."
    - deleted 17y ago
      [deleted]
  - timf 17y ago
    Sure, I agree with you, there was no "huge paradigm shift" (that is so extremely rare in CS). But there was a virtualization efficiency advance that got an impressive number of companies to actively jump into the project (with time and money) which (along with it being free) made it explode, mature, and eventually make things like commodity computing rental via EC2 possible to implement just years later. I agree with him that there is so much established infrastructure/standardization out there that doing something completely game changing is hard if not impossible. Which is why I think weighing things against such a behemoth industry, Xen's story is impressive. It did what he wanted: "Make the industry want your work." But it's because it is useful, not just because it is cool. As for grant money's role in all this, it's the same issue. Like arts and humanities vs. energy/science, the lion's share of research funding is always going to go towards what will most likely advance the country's economy/technology.
    - uriel 17y ago
      > But there was a virtualization efficiency advance You are just reinforcing Rob's point. Xen doesn't do anything new, it is simply an optimization, that doesn't mean it is not useful (as he also points out), but confirms his point that all 'research' this days is just optimizations and 'phenomenology', and nothing that is and feels truly different and new. In the end Xen is a textbook illustration of the problems he is pointing at.
      - timf 17y ago
        People across the globe are doing something that is and feels truly different and new: they're thinking and arranging their entire stacks in terms of remote virtual machines. We could argue about the definition of novelty all day and in the end I think it's simply a judgement call what one deems as exciting or worth talking about. If you're looking for something so profoundly different along the lines of Einstein and Heisenberg in physics, I agree with him through and through, nothing in systems will probably happen like that given the current funding paradigms (I can't actually think of anything like that in the history of computer science -- you can for example be as reductionist as you want if you want to take Pike's side of the argument: "oh, it's all just a collection of logic gates"...).
        uriel 17y ago
        > People across the globe are doing something that is and feels truly different and new: they're thinking and arranging their entire stacks in terms of remote virtual machines. Except that there had been people doing precisely that for ages. Seems that computer science is condemned to eternally reinvent wheels... sigh
        timf 17y ago
        Well I guess your definition of "precisely" must be much different than mine.
        slackenerny 17y ago
- jackchristopher 17y ago
  http://news.ycombinator.com/item?id=245227 http://news.ycombinator.com/item?id=245227 EDIT: Here's Pike's update: http://interviews.slashdot.org/article.pl?sid=04/10/18/1153211&tid=189&tid=156&tid=130&tid=11 http://interviews.slashdot.org/article.pl?sid=04/10/18/11532...
  - Kaizyn 17y ago
    Thank you for posting this.
  - gruseom 17y ago
    The following passage from that second piece is fascinating (I had never heard of the "Unix room"... it sounds like an illustration of Conway's Law, and a lot of fun too). One odd detail that I think was vital to how the group functioned was a result of the first Unix being run on a clunky minicomputer with terminals in the machine room. People working on the system congregated in the room - to use the computer, you pretty much had to be there. (This idea didn't seem odd back then; it was a natural evolution of the old hour-at-a-time way of booking machines like the IBM 7090.) The folks liked working that way, so when the machine was moved to a different room from the terminals, even when it was possible to connect from your private office, there was still a `Unix room' with a bunch of terminals where people would congregate, code, design, and just hang out. (The coffee machine was there too.) The Unix room still exists, and it may be the greatest cultural reason for the success of Unix as a technology. More groups could profit from its lesson, but it's really hard to add a Unix-room-like space to an existing organization. You need the culture to encourage people not to hide in their offices, you need a way of using systems that makes a public machine a viable place to work - typically by storing the data somewhere other than the 'desktop' - and you need people like Ken and Dennis (and Brian Kernighan and Doug McIlroy and Mike Lesk and Stu Feldman and Greg Chesson and ...) hanging out in the room, but if you can make it work, it's magical.
- Locke1689 17y ago
  Agreed. My research: http://v3vee.org/ http://v3vee.org/. I don't even know what to say about this article - it's just so poorly thought out and researched that I can't believe it.
  - uriel 17y ago
    > I don't even know what to say about this article - it's just so poorly thought out and researched that I can't believe it. You are clearly missing the point by a few miles...
    - deleted 17y ago
      [deleted]
    - Locke1689 17y ago
      Yeah. I see the date now. Ignore that.
      - uriel 17y ago
        The date has nothing to do with it, the kind and nature of the research is the issue.
      - Locke1689 17y ago
        In that case, you're just wrong. Look at what has come out of systems research: Xen and its ilk created a huge jump in virtualization research. Microsoft and VMWare now have large paravirtualization as parts of their virtual machines, which we are seeing more and more uses for every day. There are a number of other things that come to mind, mostly in high performance computing. Parallel FORTRAN is one of the most common HPC languages and I believe the primary development was based on work at Rice university. The problem is that people too often get caught up on implementation: most academic departments do not develop "products," they work to develop ideas. Many of these ideas have been built on recently to produce successful products.
- deleted 17y ago
  [deleted]
- kmavm 17y ago
  And two years earlier had come VMware. The explosion of interest in virtualization was started by VMware, a commercial entity, which rather makes Pike's point. Xen was started in response to the lack of an academia-friendly platform for virtualization research; "paravirtualization" was not initially a design goal, but rather an expedient choice to avoid the difficulties of correctly virtualizing the x86. And while I'm biased, VMware's solution to this problem (dynamic binary translation from supervisor x86 to user-level x86) is massively more inventive, interesting and useful than Xen's solution (hack up the kernel). I emphasize "useful", because a lot of VMware's customers were most interested in running Windows, which was not paravirtulization-ready until 2008, a full ten years after VMware's products were available.
  - timf 17y ago
    VMware's awesome... but its roots are also in academia, the company was started by a Stanford professor and his PhD students. It seems like a circular argument if the criteria of an academic project "successfully influencing industry" is widespread adoption but that adoption cannot be facilitated by a commercial entity. [As a side note -- the impetus for Xen, and also somewhat for VMware, was in a vision of global appliance-based computing, not in "virtualization research." I think that is really interesting given what just happened in this area in the last 3 or 4 years.]
  - deleted 17y ago
    [deleted]
GeneralMaximus 17y ago
The UNIX point really resonates with me. I'm writing this post from Haiku (http://haiku-os.org http://haiku-os.org), and it sometimes saddens me to see that a lot of energy is spent on POSIX compatibility (Haiku is not UNIX, but there's a compatibility layer that allows trivial POSIX software to compile with very few changes, and makes porting larger pieces of software possible). This, of course, has to be done. Otherwise there will be a whole bunch of software future Haiku users will miss out on, and a whole bunch of trivial software the developers would have to rewrite from scratch. Oh, and then it would take way too much effort to port Perl, Python, Ruby, Vim ... These days, "portable software" pretty much means "targets POSIX".
- uriel 17y ago
  There is a reason Plan 9 dumped PoSix, there is APE ( http://doc.cat-v.org/plan_9/4th_edition/papers/ape http://doc.cat-v.org/plan_9/4th_edition/papers/ape ) but to properly maintain it would take way more effort than the rest of the OS. And it is becoming even worse, 'all the world is a vax' has become 'all cpus are x86(-64), all the kernels are linux, all the libcs are glibc, and all the compilers are gcc', portability of most software is awful, and auto*hell only makes things much worse.
  - krakensden 17y ago
    On the other hand, in that world at least you only have the broken insanity of one operating system/library stack to worry about instead of three or four. ``Not Linux'' is not the same as 'perfect', no matter what the *BSD people say.
heresyforme 17y ago
New OS stagnation could be showing the trend away from the OS. A return to the terminal? Cloud computing could be the first new step of many. In many ways we may no longer need the local OS.
dinkumthinkum 17y ago
Maybe someone should inform SOSP. By the way guys virtualization is hardly the only thing going on in systems research. As far as the Web replacing the OS... Honestly if HTTP/HTML is the sole future of innovation in computing then that is a bit sad. As for Cloud Computing, well, let's say the reports of the demise of systems research have been greatly, greatly exagerrated. Actually, looking at my dept this just seems to be a pitifully naive view.there are plenty of projects and not the time nor enough grad students to do them all.
cpr 17y ago
I'm wondering if it's time for a revival of the capability-based OS's of the 70's, such as Hydra/C.mmp, CMU's flagship OS of the time. Seems like the raw performance is there, and the CPU parallelism is there. Certainly the time is ripe for a more secure basis for operating systems, and capabilities still seem like one of the best theoretical foundations around.

39 points by uriel 17y ago

timf 17y ago

Written August 5th, 2000 "Irrelevant: Does not influence industry" Two years later came Xen, a systems research project that has definitely influenced industry.

uriel 17y ago
If Xen is the best you can come up as a result of research impacting industry in the last ten years... IBM had been doing basically the same for decades, so I'm not sure one could consider Xen particularly revolutionary. Edit: Also note that Xen in no way fulfills the definition of Systems Research, it is simply a new minor twist on a very old idea.
ams6110 17y ago
OK, to play devil's advocate, Xen is nothing new in the same way that Pike claims Linux is nothing new. Hypervisors and virutalization have been around since the late '60s.
- Locke1689 17y ago
  Look up "paravirtualization."
  - deleted 17y ago
    [deleted]
- timf 17y ago
  Sure, I agree with you, there was no "huge paradigm shift" (that is so extremely rare in CS). But there was a virtualization efficiency advance that got an impressive number of companies to actively jump into the project (with time and money) which (along with it being free) made it explode, mature, and eventually make things like commodity computing rental via EC2 possible to implement just years later. I agree with him that there is so much established infrastructure/standardization out there that doing something completely game changing is hard if not impossible. Which is why I think weighing things against such a behemoth industry, Xen's story is impressive. It did what he wanted: "Make the industry want your work." But it's because it is useful, not just because it is cool. As for grant money's role in all this, it's the same issue. Like arts and humanities vs. energy/science, the lion's share of research funding is always going to go towards what will most likely advance the country's economy/technology.
  - uriel 17y ago
    > But there was a virtualization efficiency advance You are just reinforcing Rob's point. Xen doesn't do anything new, it is simply an optimization, that doesn't mean it is not useful (as he also points out), but confirms his point that all 'research' this days is just optimizations and 'phenomenology', and nothing that is and feels truly different and new. In the end Xen is a textbook illustration of the problems he is pointing at.
    - timf 17y ago
      People across the globe are doing something that is and feels truly different and new: they're thinking and arranging their entire stacks in terms of remote virtual machines. We could argue about the definition of novelty all day and in the end I think it's simply a judgement call what one deems as exciting or worth talking about. If you're looking for something so profoundly different along the lines of Einstein and Heisenberg in physics, I agree with him through and through, nothing in systems will probably happen like that given the current funding paradigms (I can't actually think of anything like that in the history of computer science -- you can for example be as reductionist as you want if you want to take Pike's side of the argument: "oh, it's all just a collection of logic gates"...).
      - uriel 17y ago
        > People across the globe are doing something that is and feels truly different and new: they're thinking and arranging their entire stacks in terms of remote virtual machines. Except that there had been people doing precisely that for ages. Seems that computer science is condemned to eternally reinvent wheels... sigh
        timf 17y ago
        Well I guess your definition of "precisely" must be much different than mine.
        slackenerny 17y ago
jackchristopher 17y ago
http://news.ycombinator.com/item?id=245227 http://news.ycombinator.com/item?id=245227 EDIT: Here's Pike's update: http://interviews.slashdot.org/article.pl?sid=04/10/18/1153211&tid=189&tid=156&tid=130&tid=11 http://interviews.slashdot.org/article.pl?sid=04/10/18/11532...
- Kaizyn 17y ago
  Thank you for posting this.
- gruseom 17y ago
  The following passage from that second piece is fascinating (I had never heard of the "Unix room"... it sounds like an illustration of Conway's Law, and a lot of fun too). One odd detail that I think was vital to how the group functioned was a result of the first Unix being run on a clunky minicomputer with terminals in the machine room. People working on the system congregated in the room - to use the computer, you pretty much had to be there. (This idea didn't seem odd back then; it was a natural evolution of the old hour-at-a-time way of booking machines like the IBM 7090.) The folks liked working that way, so when the machine was moved to a different room from the terminals, even when it was possible to connect from your private office, there was still a `Unix room' with a bunch of terminals where people would congregate, code, design, and just hang out. (The coffee machine was there too.) The Unix room still exists, and it may be the greatest cultural reason for the success of Unix as a technology. More groups could profit from its lesson, but it's really hard to add a Unix-room-like space to an existing organization. You need the culture to encourage people not to hide in their offices, you need a way of using systems that makes a public machine a viable place to work - typically by storing the data somewhere other than the 'desktop' - and you need people like Ken and Dennis (and Brian Kernighan and Doug McIlroy and Mike Lesk and Stu Feldman and Greg Chesson and ...) hanging out in the room, but if you can make it work, it's magical.
Locke1689 17y ago
Agreed. My research: http://v3vee.org/ http://v3vee.org/. I don't even know what to say about this article - it's just so poorly thought out and researched that I can't believe it.
- uriel 17y ago
  > I don't even know what to say about this article - it's just so poorly thought out and researched that I can't believe it. You are clearly missing the point by a few miles...
  - deleted 17y ago
    [deleted]
  - Locke1689 17y ago
    Yeah. I see the date now. Ignore that.
    - uriel 17y ago
      The date has nothing to do with it, the kind and nature of the research is the issue.
    - Locke1689 17y ago
      In that case, you're just wrong. Look at what has come out of systems research: Xen and its ilk created a huge jump in virtualization research. Microsoft and VMWare now have large paravirtualization as parts of their virtual machines, which we are seeing more and more uses for every day. There are a number of other things that come to mind, mostly in high performance computing. Parallel FORTRAN is one of the most common HPC languages and I believe the primary development was based on work at Rice university. The problem is that people too often get caught up on implementation: most academic departments do not develop "products," they work to develop ideas. Many of these ideas have been built on recently to produce successful products.
deleted 17y ago
[deleted]
kmavm 17y ago
And two years earlier had come VMware. The explosion of interest in virtualization was started by VMware, a commercial entity, which rather makes Pike's point. Xen was started in response to the lack of an academia-friendly platform for virtualization research; "paravirtualization" was not initially a design goal, but rather an expedient choice to avoid the difficulties of correctly virtualizing the x86. And while I'm biased, VMware's solution to this problem (dynamic binary translation from supervisor x86 to user-level x86) is massively more inventive, interesting and useful than Xen's solution (hack up the kernel). I emphasize "useful", because a lot of VMware's customers were most interested in running Windows, which was not paravirtulization-ready until 2008, a full ten years after VMware's products were available.
- timf 17y ago
  VMware's awesome... but its roots are also in academia, the company was started by a Stanford professor and his PhD students. It seems like a circular argument if the criteria of an academic project "successfully influencing industry" is widespread adoption but that adoption cannot be facilitated by a commercial entity. [As a side note -- the impetus for Xen, and also somewhat for VMware, was in a vision of global appliance-based computing, not in "virtualization research." I think that is really interesting given what just happened in this area in the last 3 or 4 years.]
- deleted 17y ago
  [deleted]

GeneralMaximus 17y ago

The UNIX point really resonates with me. I'm writing this post from Haiku (http://haiku-os.org http://haiku-os.org), and it sometimes saddens me to see that a lot of energy is spent on POSIX compatibility (Haiku is not UNIX, but there's a compatibility layer that allows trivial POSIX software to compile with very few changes, and makes porting larger pieces of software possible). This, of course, has to be done. Otherwise there will be a whole bunch of software future Haiku users will miss out on, and a whole bunch of trivial software the developers would have to rewrite from scratch. Oh, and then it would take way too much effort to port Perl, Python, Ruby, Vim ... These days, "portable software" pretty much means "targets POSIX".

uriel 17y ago
There is a reason Plan 9 dumped PoSix, there is APE ( http://doc.cat-v.org/plan_9/4th_edition/papers/ape http://doc.cat-v.org/plan_9/4th_edition/papers/ape ) but to properly maintain it would take way more effort than the rest of the OS. And it is becoming even worse, 'all the world is a vax' has become 'all cpus are x86(-64), all the kernels are linux, all the libcs are glibc, and all the compilers are gcc', portability of most software is awful, and auto*hell only makes things much worse.
- krakensden 17y ago
  On the other hand, in that world at least you only have the broken insanity of one operating system/library stack to worry about instead of three or four. ``Not Linux'' is not the same as 'perfect', no matter what the *BSD people say.

heresyforme 17y ago

New OS stagnation could be showing the trend away from the OS. A return to the terminal? Cloud computing could be the first new step of many. In many ways we may no longer need the local OS.

dinkumthinkum 17y ago

Maybe someone should inform SOSP. By the way guys virtualization is hardly the only thing going on in systems research. As far as the Web replacing the OS... Honestly if HTTP/HTML is the sole future of innovation in computing then that is a bit sad. As for Cloud Computing, well, let's say the reports of the demise of systems research have been greatly, greatly exagerrated. Actually, looking at my dept this just seems to be a pitifully naive view.there are plenty of projects and not the time nor enough grad students to do them all.

cpr 17y ago

I'm wondering if it's time for a revival of the capability-based OS's of the 70's, such as Hydra/C.mmp, CMU's flagship OS of the time. Seems like the raw performance is there, and the CPU parallelism is there. Certainly the time is ripe for a more secure basis for operating systems, and capabilities still seem like one of the best theoretical foundations around.