5 ms·

For big metal rigs like supercomputers, there is almost no choice but Linux. Combined with Linux' dominance of the server market in general, there's a risk emer

by sspiff 9y ago

For big metal rigs like supercomputers, there is almost no choice but Linux. Combined with Linux' dominance of the server market in general, there's a risk emerging to get stuck in a monoculture on the OS layer. Most companies, including very big ones with a long history of OS development are abandoning their efforts in favor of just using Linux. The perception is often that there is no innovation that can be done on the OS layer that will earn back its investment. This leads to a lack of experimentation, tunnel vision, and in the long run stagnation.

gtirloni 9y ago
Companies experiment on Linux everyday. Unless they want to use a different programming language, they can implement all sorts of innovation in the Linux Kernel without incurring in massive overhead costs of rolling their own. I don't think the situation is nearly as bad as portrait here. They can always use *BSD as a second choice too. Additionally, where's the stagnation we are so worried about here? Any real world examples?
- ThrowawayR2 9y ago
  > Additionally, where's the stagnation we are so worried about here? Any real world examples? BeOS? Singularity? Plan 9? Hell, just read the last 20 years of OS research and realize that large swathes of it will never get used because anything that gets implemented has to be compatible with the way the Linux kernel does things. (By way of analogy, consider the x86 architecture, also a '70s era technology, and the massive contortions Intel had to go through to maintain compatibility over the years while adding new features and how all the newer alternative architectures died in the marketplace because x86 was too entrenched.) Fuschia and, ironically, Windows are the last hopes for advancing OS architecture and that's really only possible because the semi-monopolists that created them have enough cash and engineering talent to push them until they take off.
  - gtirloni 9y ago
    Sorry, maybe I wasn't clear. We are talking about HPC computing and how Linux being the only viable choice leads to stagnation. You are proposing those OSes advance the status quo (and I agree) but how is it that HPC computing is stagnated because Linux is the only viable option? If anything, HPC would be an awesome testbed for those experimental OSes and they still choose Linux. Where's HPC stagnated because awesome ideas from other OSes can't be used? Finally, are these other ideas so extremely awesome that they require a new OS from scratch (with all the uphill battle that it's to create it)? Can they be implemented on top of the Linux kernel? In which case, is Linux's success really such a bad thing?
  - waddlesplash 9y ago
    Hey, Haiku's still around, and while we don't have time to experiment with such crazy ideas ourselves (keeping an operating system project afloat is no joke), we'll more than likely accept patches for such things ... provided they actually work and aren't just needless bloat, and especially if they fix or improve some area we've neglected :-p
- sspiff 9y ago
  I understand that there is experimentation and innovation inside and on top of the Linux kernel, but that still leads to some kind of local optimum, the best approach that fits within the design and implementation of the Linux kernel. I'm not saying this is necessarily a problem, but it risks becoming a problem. It's clear we're seeing reduced investment in Linux competitors in exchange for Linux flavours from large players the past decade or so.
ageofwant 9y ago
That's all baseless speculation and hand-wringing over straw-boogeyman. A 9 year old can install Linux and experiment and "innovate" all she wants. Linux's open source nature and the community is what powers its ongoing evolution, and that is encoded in its essence. I fail to see how the vein of Linux innovation on big iron is any close to being spent.
mseebach 9y ago
Stagnation is a good thing -- when it happens because all the hard problems in a domain has been solved.
- digi_owl 9y ago
  More like all the easy optimizations have been exploited, and all that remains are the hard (and much less cost effective) ones.
Tuna-Fish 9y ago
The competition has moved inside Linux. The large companies all maintain their own patch sets. There are still ways they believe that they can make OSes better, they just don't believe they can compete with Linux in all the other ways, so they just write their own modifications for it instead. Which is absolutely wonderful, because Linux is licensed under GPL, so the best improvements end up in the main tree and everyone benefits.
- mrkgnao 9y ago
  > Linux is licensed under GPL, so the best improvements end up in the main tree Without taking sides in the debate, grsecurity/PaX patches are a counterexample.
  - bryanlarsen 9y ago
    That's a great example of https://news.ycombinator.com/item?id=15772233 https://news.ycombinator.com/item?id=15772233 (currently also on the front page), where something appears superior on the surface, but there are good reasons why it didn't "win".
  - srcmap 9y ago
    "grsecurity/PaX" didn't show up in the main tree but is alive in a different branch or as patch. Various distributions and users are free to enable and use it upon demand. It kind of prove Linux kernel is not mono-culture and natural selection and evolution are happening. I believe the AOSP's kernel patches are also not mainlined - also proof Linux kernel are evolving very healthily.
- digi_owl 9y ago
  So in essence Linux have become what UNIX was back in the day, a baseline for everyone from IBM to HP to build upon.
icelancer 9y ago
A "winner" is all but guaranteed when it comes to slicing the smallest performance gains in any field. And this is a niche field of performance; as I said in another comment, Win10 is still superior to Linux in other compute-bound systems (HBM2 is incapable of being exploited to its fullest potential outside of Win10 due to AMD's poor compute/blockchain drivers for Linux, for example - if you need blazing fast memops for the price, nothing beats Vega architecture right now).

by sspiff 9y ago

gtirloni 9y ago

Companies experiment on Linux everyday. Unless they want to use a different programming language, they can implement all sorts of innovation in the Linux Kernel without incurring in massive overhead costs of rolling their own. I don't think the situation is nearly as bad as portrait here. They can always use *BSD as a second choice too. Additionally, where's the stagnation we are so worried about here? Any real world examples?

ThrowawayR2 9y ago
> Additionally, where's the stagnation we are so worried about here? Any real world examples? BeOS? Singularity? Plan 9? Hell, just read the last 20 years of OS research and realize that large swathes of it will never get used because anything that gets implemented has to be compatible with the way the Linux kernel does things. (By way of analogy, consider the x86 architecture, also a '70s era technology, and the massive contortions Intel had to go through to maintain compatibility over the years while adding new features and how all the newer alternative architectures died in the marketplace because x86 was too entrenched.) Fuschia and, ironically, Windows are the last hopes for advancing OS architecture and that's really only possible because the semi-monopolists that created them have enough cash and engineering talent to push them until they take off.
- gtirloni 9y ago
  Sorry, maybe I wasn't clear. We are talking about HPC computing and how Linux being the only viable choice leads to stagnation. You are proposing those OSes advance the status quo (and I agree) but how is it that HPC computing is stagnated because Linux is the only viable option? If anything, HPC would be an awesome testbed for those experimental OSes and they still choose Linux. Where's HPC stagnated because awesome ideas from other OSes can't be used? Finally, are these other ideas so extremely awesome that they require a new OS from scratch (with all the uphill battle that it's to create it)? Can they be implemented on top of the Linux kernel? In which case, is Linux's success really such a bad thing?
- waddlesplash 9y ago
  Hey, Haiku's still around, and while we don't have time to experiment with such crazy ideas ourselves (keeping an operating system project afloat is no joke), we'll more than likely accept patches for such things ... provided they actually work and aren't just needless bloat, and especially if they fix or improve some area we've neglected :-p
sspiff 9y ago
I understand that there is experimentation and innovation inside and on top of the Linux kernel, but that still leads to some kind of local optimum, the best approach that fits within the design and implementation of the Linux kernel. I'm not saying this is necessarily a problem, but it risks becoming a problem. It's clear we're seeing reduced investment in Linux competitors in exchange for Linux flavours from large players the past decade or so.

ageofwant 9y ago

That's all baseless speculation and hand-wringing over straw-boogeyman. A 9 year old can install Linux and experiment and "innovate" all she wants. Linux's open source nature and the community is what powers its ongoing evolution, and that is encoded in its essence. I fail to see how the vein of Linux innovation on big iron is any close to being spent.

mseebach 9y ago

Stagnation is a good thing -- when it happens because all the hard problems in a domain has been solved.

digi_owl 9y ago
More like all the easy optimizations have been exploited, and all that remains are the hard (and much less cost effective) ones.

Tuna-Fish 9y ago

The competition has moved inside Linux. The large companies all maintain their own patch sets. There are still ways they believe that they can make OSes better, they just don't believe they can compete with Linux in all the other ways, so they just write their own modifications for it instead. Which is absolutely wonderful, because Linux is licensed under GPL, so the best improvements end up in the main tree and everyone benefits.

mrkgnao 9y ago
> Linux is licensed under GPL, so the best improvements end up in the main tree Without taking sides in the debate, grsecurity/PaX patches are a counterexample.
- bryanlarsen 9y ago
  That's a great example of https://news.ycombinator.com/item?id=15772233 https://news.ycombinator.com/item?id=15772233 (currently also on the front page), where something appears superior on the surface, but there are good reasons why it didn't "win".
- srcmap 9y ago
  "grsecurity/PaX" didn't show up in the main tree but is alive in a different branch or as patch. Various distributions and users are free to enable and use it upon demand. It kind of prove Linux kernel is not mono-culture and natural selection and evolution are happening. I believe the AOSP's kernel patches are also not mainlined - also proof Linux kernel are evolving very healthily.
digi_owl 9y ago
So in essence Linux have become what UNIX was back in the day, a baseline for everyone from IBM to HP to build upon.

icelancer 9y ago

A "winner" is all but guaranteed when it comes to slicing the smallest performance gains in any field. And this is a niche field of performance; as I said in another comment, Win10 is still superior to Linux in other compute-bound systems (HBM2 is incapable of being exploited to its fullest potential outside of Win10 due to AMD's poor compute/blockchain drivers for Linux, for example - if you need blazing fast memops for the price, nothing beats Vega architecture right now).