5 ms·
Yeah, the compatibility element is really important. I've been doing tons of research around illumos, SmartOS, etc., lately and I'm hoping to deploy it in some
by cookiecaper 9y ago
Yeah, the compatibility element is really important. I've been doing tons of research around illumos, SmartOS, etc., lately and I'm hoping to deploy it in some projects soon. DTrace knowledge becoming properly portable across systems is a great thing.
I've been very impressed with illumos from a technical standpoint, as a Linux admin for over 10 years whose never really had cause to seriously wander into the now-esoteric "proper Unices". Combination of the massive relief that is ZOL and the disastrous aftermath of the [ongoing] Linuxland "container revolution" got me looking for more.
But before I commit to illumos in a substantial way, I'm trying to understand the social dynamics around it more, and whether the community is essentially just crossing fingers and hoping that Samsung doesn't shutter Joyent, as they've done to other open-source acquisitions, especially as alternate distributors like OmniTI have bowed out. So I've been doing lots of sleuthing and trying to piece together some of the drama behind this whole ecosystem.
A lot of the momentum on SmartOS seems like it gave out during the first half of 2014 when Brendan and many other important Sun alumni left Joyent. Bryan Cantrill addressed this in an AMA by saying that he thinks some of them just tired of fighting an uphill battle (which, IMO, is totally understandable and actually doesn't seem like it's too far off the mark), but the close timing with which basically everyone except Bryan Cantrill made this decision makes me wonder if there was some internal event at Joyent that signaled a need to head for the hills.
This theory is inflamed by Brendan's apparently complete abandonment of illumos and the tone he takes when he addresses illumos/Solaris, frequently telling people to avoid it and that Linux is much better nowadays. He didn't leave Joyent and continue to extoll the virtues of illumos or SmartOS, even though his farewell post says that his high opinion on the value of illumos tech hadn't changed. He left Joyent and seemed like he was relieved that he could finally let it all out and talk about the futility of illumos/Solaris.
These days, he frequently leaves comments saying things like (paraphrasing here) "You could use [illumos distro], but I don't know why you would." and "We'd be in trouble if we picked the wrong OS at big companies like Netflix, and none of us are picking Solaris."
It just feels like there are pieces of the puzzle missing here. I get that there is a long and storied history to illumos (as Cantrill points out in one talk, the codebase is older than he is!) and that everyone brings a lot of personal perspective and emotion to it, so I don't expect there to be a simple answer per se, but I just want to feel like I've got my arms more around it.
SmartOS really does seem like a saner system for people who want things that work without constantly fussing, but I don't want to become dependent on a system whose future appears to hinge completely on Cantrill's continued employment with Joyent, especially post-exit, as he is likely bound by golden handcuffs.
Node.js is another Joyent puzzler with its own long and tortured history (though obviously, much shorter than Solaris/illumos). It's strange, imo, that they were ever really involved in that in the first place. I can only guess that they were quick to snatch up Dahl and ride the wave before Google et al could get their wits about them and establish control over that segment, but maybe it has more to do with Joyent's relationship with web publishing via TextDrive, I don't really know. But if SmartOS weren't otherwise so charming, its integration of Node.js for system control scripts would seriously put me off.
Disclaimer: I have no idea what I'm talking about, I wasn't there, I don't know any of the participants involved, and I'm not trying to assign any particular feelings or values to anyone. This is just my read of the ecosystem and dynamic from reading stuff on the internet, and it is very likely wrong. Corrections are welcome.
- bcantrill 9y agoWow -- there's a lot here! Just to unpack this a bit... First, there are a bunch of people that you might be unaware of doing important work in illumos and SmartOS (both at Joyent and in the broader community); I recommend taking a look at the actual repo[1], and some of the recent work we've done like LX-branded zones[2] and (more currently) bhyve.[3] Second, in terms of the engineering team at Joyent: yes, people have come and gone over the years -- but we've grown a bunch since the acquisition by Samsung, adding many engineers from a wide variety of backgrounds. Yes, this has included some ex-Sun folks (we have more ex-Sun now than at any time in our history, if that kind of thing matters to you), but (importantly to me, anyway) it's also included a bunch of people new to the system. These new engineers bring new perspectives and fresh thinking -- whether they've had two decades of experience or are fresh out of school. Finally, in terms of Joyent and node.js, it might be helpful to see my talk on platform as a reflection of values[4][5]; it offers my perspective on how and why we diverged from the node.js community -- but also why we're okay with that. Hope all of this helps! Let me know if you have any questions (my Twitter DMs are always open) -- or introduce yourself to the community in #illumos or #smartos on Freenode! [1] https://github.com/joyent/illumos-joyent https://github.com/joyent/illumos-joyent [2] https://www.youtube.com/watch?v=lnesNFulpPE https://www.youtube.com/watch?v=lnesNFulpPE [3] https://github.com/joyent/illumos-joyent/tree/bhyve https://github.com/joyent/illumos-joyent/tree/bhyve [4] https://www.slideshare.net/bcantrill/platform-as-reflection-of-values-joyent-nodejs-and-beyond https://www.slideshare.net/bcantrill/platform-as-reflection-... [5] https://vimeo.com/230142234 https://vimeo.com/230142234
- brendangregg 9y agoI'm sorry, but Bryan did not "address" the real reasons I left Joyent. What he said was inaccurate. I no longer believe in illumos. It made a lot of sense in 2010 when competing with Linux, a lot less by 2014, and very little today. When we created illumos in August 2010, the latest Linux version was 2.6.35. The cutting-edge latest. Most people were still running older versions, 2.6.32, 2.6.27, etc. Linux had many performance issues at the time: it still had the Big Kernel Lock (BKL), performance issues with mutexes, IPC, ext4, XFS, dcache, VFS, slub, and other subsystems, no transparent huge pages, no NUMA balancing, no inbound or outbound network fanout, and before many TCP improvements: increasing the initial window size, early retransmit, fast open, tail loss probe, autocorking, anti-bufferbloat, and so on. Linux was slow. I worked on Solaris performance, and beating Linux was something I'd expect to do nine times out of ten. illumos, based on OpenSolaris, seemed like a great idea. You get a faster, more scalable kernel, and you get three great features: ZFS, DTrace, and Zones. At one point we had "ZFS, DTrace, Zones" printed in big letters on T-shirts. I evangelized illumos. I believed in it. However, all of those Linux performance issues were fixed in the years that followed. With the changed kernel version: BKL (2.6.37), mutexes (2.6.36, 3.10), IPC (2.6.33, 2.6.35), ext4 (2.6.37+), XFS (2.6.37+), dcache (2.6.38), VFS (3.1), slub (3.1), transparent huge pages (2.6.38), NUMA balancing (3.8), inbound fanout (RPS, 2.6.35), outbound fanout (XFS, 2.6.38), TCP window size (2.6.39), early retransmit (3.5), fast open (3.6, 3.7), tail loss probe (3.10), autocorking (3.14), anti-bufferbloat (3.14), etc. It came down to ZFS, DTrace, and Zones as the differentiating features. If you look at Linux today, and what we run at Netflix, we already have ZFS, eBPF, and containers. Anyway, I've discussed this topic before, here on hackernews: https://news.ycombinator.com/item?id=13081465 https://news.ycombinator.com/item?id=13081465 You might still have some corner case reason to still run illumos. But bear in mind that Linux today, in 2018, is completely, completely, different to 2010 when we created illumos. 8 years is a long time in Linux. edit: you have got me thinking, however. As someone who promoted illumos years ago (or any technology), and then stopped, I may have a responsibility to properly explain why I stopped (beyond answering questions when directly asked). I was hoping my Solaris to Linux post[1] was the last time I pour effort down the Solaris drain, but maybe I need one more. [1] http://www.brendangregg.com/blog/2017-09-05/solaris-to-linux-2017.html http://www.brendangregg.com/blog/2017-09-05/solaris-to-linux...