4 ms·

> My real job is a professor and researcher in the area of operating systems. > As a result of my occupation, I think I know a bit about where operating are g

> My real job is a professor and researcher in the area of operating systems. > As a result of my occupation, I think I know a bit about where operating are going in the next decade or so. The gap between industry and academia must have been less well recognized at this stage. I think of PL researchers today, most of whom would not confidently assert they know the way programming languages will go—they'd instead confine themselves to asserting that they know where PLs ought to go, while acknowledging that the industry doesn't tend to care at all what PL researchers think a PL should look like. One thing I'm curious about is why the industry-academia gap is so large? Is this true in other disciplines? I'd expect some baseline level of ivory-tower effect in any industry, but I'd also expect there to be a significant number of people who actually do cross the gap and make an effort to study the way things actually work rather than just the way they theoretically ought to work. Where are the OS researchers who research why Linux won? Where are the PL researchers who study what makes a great industry language?

marcosdumay 2y ago
On the OS, there are valid technical reasons for explaining some of the gap. The networked OS idea from the 90s simply isn't great, and as computers became more powerful it became a non-brainier to make each one do everything and get rid of the unreliable network. But well, that's just some of the gap. The truth is that most of what the industry does is chosen by really stupid reasons (or some times raw corruption), where the people making the choice has no power to fix any problem, and the problems are kept there by people not interested on improving anything. If you want to research why the industry does what it does, you should study political science, not anything related to IT.
- dartos 2y ago
  Exactly. The real issue is that academics know about the subject matter, but the decision makers (bizdev people, generally) don’t and couldn’t care less.
vilhelm_s 2y ago
I think the gap was just beginning to emerge around 1990. Until then, people really were re-developing the full computing stack based on new research every few years. Rob Pike identified 1990 as the year systems research became irrelevant[0] because from then on people kept on using and iterating on the same software. [0] https://tianyin.github.io/misc/irrelevant.pdf https://tianyin.github.io/misc/irrelevant.pdf
nfriedly 2y ago
I think alternating between teaching and working in the industry is a good way to stay balanced and keep the gap small. The tech lead on my team was a college professor for a while before joining us, and he occasionally got in spats with one of the other more senior folks on our team, which could be oversimplified to "correct vs pragmatic". However, they also respected each other and always resolved it amicably in the end. A couple of times I thanked them both for working through it and pointed out that I think we end up with significantly better software as a result, even if getting there was difficult. I learned a lot from both of them.
deleted 2y ago
[deleted]
hecanjog 2y ago
Industry is primarily concerned with profit? I don't mean to be dismissive, I think that's significant, and seems like everything else just flows out of it.
LeFantome 2y ago
It is not a problem unique to academia. Anytime you prioritize the perspective of your own expertise, you have this problem. As you said, he may have been right about where things “ought” to go. In that way, it is the same as an engineer telling you that they know where their field will go in 10 years. They are often wrong when they forget that technology is not the only force at work. Why Linux won is good history to know but few of the reasons will advance OS research. He was probably not wrong that microkernels as the future in that most new OS projects will use that design at some point. Just like most dev will be in memory safe languages at some point. Look at Redox, using both a microkernel and Rust. The trick is that recognizing that “at some point” may not be now even if you are right. As Keynes said about investing (also future prediction), “the market can stay irrational longer than you can stay solvent”. Also note that the design of Linux itself changed after this debate as well. The system of pluggable modules is not a microkernel but it is not exactly a true monolithic kernel either. We do mot “compile in” all our device drivers. Also, the FUSE user mode filesystem looks pretty microkernel-like to me. Microkernels have a lot of advantages. Enough that Andrew T thought they were the obvious choice. So did HURD. They are also harder to implement. In this era for easy virtual machines, containers, and the cloud, I think it is a lot easier. It was a lot harder when Linux was being written in-place of 386 PC hardware. More importantly, microkernels are a bit less efficient. Again, that does not matter quite as much today in many cases. The kernel itself is only using a tiny fraction of your memory and CPU capacity. In the era of 386 hardware with while digit megabytes of RAM, it still mattered a lot. Remember that MINIX, the microkernel being defended, got installed in very Intel CPU for years. There may have been more instances of Minix than there was of Linux. Also, I do not think that Linux “won” because of technology. BSD already existed when Linux was born and for years BSD was clearly technically superior. Linux got the momentum and the mindshare. Why? I think the AT&T lawsuit was the primary factor. A lot of other proper credit the GPL. Few would argue it was because Linux was better back then. Why Linux won is not going to advance OS research. Companies kirk Microsoft and IBM have big “research” arms that produce “working” tech all the time that showcase ideas that are “the future”. It is not like these companies frequently throw-out their successful products every time this happens. But ideas do trickle in. And as I said above, even Linux has borrowed from the “losing” ideas showcased in “Linux is obsolete”.
- 2y ago
LegionMammal978 2y ago
As another commenter notes (https://news.ycombinator.com/item?id=42980980 https://news.ycombinator.com/item?id=42980980), researchers usually focus on things with novel ideas, so something like an OS or PL that only uses old ideas but executes them well will fly under the academic radar. I don't think it's all just network effects, there really are important practicalities that many researchers tend to gloss over. Perhaps it's possible to close this gap, and make an OS or PL that combines new actually-good ideas with great execution, but there may just not be enough of a push by any party to do so. Or perhaps there's just too much disdain in both directions. (Those dumb enterprise programmers, toiling with Java and Windows because they can't be bothered to learn anything better! / Those dumb researchers, getting grant money to come up with abstract nonsense that no one asked for and no one else will seriously use!) Also, especially in PL research, a lot of the language useful for expressing novel ideas is very different from the kinds of documentation used in practical applications. Research-oriented people will swear that it's great because of how precise it is (e.g., https://news.ycombinator.com/item?id=42918840 https://news.ycombinator.com/item?id=42918840, on the WASM Core Specification), but it's hardly like precision must be at odds with accessibility to a wider audience.

marcosdumay 2y ago

On the OS, there are valid technical reasons for explaining some of the gap. The networked OS idea from the 90s simply isn't great, and as computers became more powerful it became a non-brainier to make each one do everything and get rid of the unreliable network. But well, that's just some of the gap. The truth is that most of what the industry does is chosen by really stupid reasons (or some times raw corruption), where the people making the choice has no power to fix any problem, and the problems are kept there by people not interested on improving anything. If you want to research why the industry does what it does, you should study political science, not anything related to IT.

dartos 2y ago
Exactly. The real issue is that academics know about the subject matter, but the decision makers (bizdev people, generally) don’t and couldn’t care less.

vilhelm_s 2y ago

I think the gap was just beginning to emerge around 1990. Until then, people really were re-developing the full computing stack based on new research every few years. Rob Pike identified 1990 as the year systems research became irrelevant[0] because from then on people kept on using and iterating on the same software. [0] https://tianyin.github.io/misc/irrelevant.pdf https://tianyin.github.io/misc/irrelevant.pdf

nfriedly 2y ago

I think alternating between teaching and working in the industry is a good way to stay balanced and keep the gap small. The tech lead on my team was a college professor for a while before joining us, and he occasionally got in spats with one of the other more senior folks on our team, which could be oversimplified to "correct vs pragmatic". However, they also respected each other and always resolved it amicably in the end. A couple of times I thanked them both for working through it and pointed out that I think we end up with significantly better software as a result, even if getting there was difficult. I learned a lot from both of them.

deleted 2y ago

[deleted]

hecanjog 2y ago

Industry is primarily concerned with profit? I don't mean to be dismissive, I think that's significant, and seems like everything else just flows out of it.

LeFantome 2y ago

It is not a problem unique to academia. Anytime you prioritize the perspective of your own expertise, you have this problem. As you said, he may have been right about where things “ought” to go. In that way, it is the same as an engineer telling you that they know where their field will go in 10 years. They are often wrong when they forget that technology is not the only force at work. Why Linux won is good history to know but few of the reasons will advance OS research. He was probably not wrong that microkernels as the future in that most new OS projects will use that design at some point. Just like most dev will be in memory safe languages at some point. Look at Redox, using both a microkernel and Rust. The trick is that recognizing that “at some point” may not be now even if you are right. As Keynes said about investing (also future prediction), “the market can stay irrational longer than you can stay solvent”. Also note that the design of Linux itself changed after this debate as well. The system of pluggable modules is not a microkernel but it is not exactly a true monolithic kernel either. We do mot “compile in” all our device drivers. Also, the FUSE user mode filesystem looks pretty microkernel-like to me. Microkernels have a lot of advantages. Enough that Andrew T thought they were the obvious choice. So did HURD. They are also harder to implement. In this era for easy virtual machines, containers, and the cloud, I think it is a lot easier. It was a lot harder when Linux was being written in-place of 386 PC hardware. More importantly, microkernels are a bit less efficient. Again, that does not matter quite as much today in many cases. The kernel itself is only using a tiny fraction of your memory and CPU capacity. In the era of 386 hardware with while digit megabytes of RAM, it still mattered a lot. Remember that MINIX, the microkernel being defended, got installed in very Intel CPU for years. There may have been more instances of Minix than there was of Linux. Also, I do not think that Linux “won” because of technology. BSD already existed when Linux was born and for years BSD was clearly technically superior. Linux got the momentum and the mindshare. Why? I think the AT&T lawsuit was the primary factor. A lot of other proper credit the GPL. Few would argue it was because Linux was better back then. Why Linux won is not going to advance OS research. Companies kirk Microsoft and IBM have big “research” arms that produce “working” tech all the time that showcase ideas that are “the future”. It is not like these companies frequently throw-out their successful products every time this happens. But ideas do trickle in. And as I said above, even Linux has borrowed from the “losing” ideas showcased in “Linux is obsolete”.

2y ago

LegionMammal978 2y ago

As another commenter notes (https://news.ycombinator.com/item?id=42980980 https://news.ycombinator.com/item?id=42980980), researchers usually focus on things with novel ideas, so something like an OS or PL that only uses old ideas but executes them well will fly under the academic radar. I don't think it's all just network effects, there really are important practicalities that many researchers tend to gloss over. Perhaps it's possible to close this gap, and make an OS or PL that combines new actually-good ideas with great execution, but there may just not be enough of a push by any party to do so. Or perhaps there's just too much disdain in both directions. (Those dumb enterprise programmers, toiling with Java and Windows because they can't be bothered to learn anything better! / Those dumb researchers, getting grant money to come up with abstract nonsense that no one asked for and no one else will seriously use!) Also, especially in PL research, a lot of the language useful for expressing novel ideas is very different from the kinds of documentation used in practical applications. Research-oriented people will swear that it's great because of how precise it is (e.g., https://news.ycombinator.com/item?id=42918840 https://news.ycombinator.com/item?id=42918840, on the WASM Core Specification), but it's hardly like precision must be at odds with accessibility to a wider audience.