3 ms·
An alternate explanation of his frustation/bitterness is that the way the market forces have worked out, there hasn't been a good business case to create a new
by tytso 12y ago
An alternate explanation of his frustation/bitterness is that the way the market forces have worked out, there hasn't been a good business case to create a new "systems company" or do ground breaking new systems work. About a decade ago, before ZFS was announced, I participated in an company-wide investigation about whether there was a viable business case to invest in new file system features. There was very large number of very experienced technical people involved: distinguished engineers that had far more experience at the company than I, business executives, etc. We explored a large number of file system enhancements, how much effort doing this development would cost, estimates of how many customers would cause customers to switch from their existing Legacy Unix systems, etc. The result of this exploration was the decision that there was no valid business case to add new features at the file system level, as opposed to other layers of the storage stack (i.e., storage arrays, enterprise databases, etc.)
After this company-wide technical study was completed, I stopped doing any ext4 work as part of my day job; it was no longer my primary job responsibility (although to be fair my employer did let me do some ext4 work on the side during business hours, so long as it didn't interfere with my primary work), and it remained that way until I started working at Google to deploy ext4 across all of Google's production servers.
Anyway, going back to my pre-Google days, there was some concern about whether the conclusions of this study was valid after Sun announced ZFS and started using it in their marketing and started pushing it in social media. This resulted in a behind-the-scenes effort to launch btrfs, but the various companies involved never did assign enough headcount so it would be successful. And despite these concerns, Solaris didn't really seem to gain significant market share versus Linux and other proprietary Linux systems. In fact, the opposite was true.
Perhaps this means that Sun was brave where other companies feared to tread --- but given that (a) various ex-Sun employees have stated that there was a lot of Solaris engineering work was done before consulting the marketing or sales "tribes", and (b) ultimately, Sun wasn't able to succeed in the marketplace --- leads me to propose that maybe pouring all of that engineering effort into ZFS and dtrace (while both are amazing technologies, I will be the first to acknowledge) --- perhaps wasn't a good product/market fit, and that a VC who looked at things from a very cold-hearted profit and loss perspective would have never allowed a lot of the Solaris engineering work to go on as long as it had unless there was a demonstrated way it could be somehow monetized.
These days, when I try to propose new ext4 work that will actually be supported by company resources (as opposed to hobby work that gets done for fun in off-hours), I have to take into account how much headcount (if any) I can get assigned, and there are often some very hard deadlines which mean that I have to make some compromises in the technical design. Of course, I try to make sure the "minimum viable product" is extensible enough that I can later on go back and try to make it better, but this is a very different way of doing systems engineering compared to some of the histories that I've seen in presentations about ZFS and Dtrace --- where some of the former Solaris engineers were proud that they didn't ask permission or get authorization from management before committing very large amounts of engineering resources, and proud of the fact that they were striving first for technical perfection.
Some former Sun engineers have in the past said some very disdainful comments about Linux: "By amateurs, for amateurs". This ignores the fact that most of the Linux engineers are in fact paid to work on Linux; but at the same time, it's also true that many of us who are paid to work on Linux have to respect real world economic considerations (which sometimes conflict with technical excellence for the sake of technical excellence) --- and very often the work to make things better than the minimum necessary to make a feature freeze deadline is in fact done because we are amateurs in the original greek sense of the word --- as a lover of the art.
But if that means that "amateurs" have been able to make an OS like Linux become widely used and at this point, has at least an order of magnitude more paid engineers working on it compared to Solaris/Open Solaris, which is considered by ex-Sun engineers a "professional" OS, it is perhaps not that surprising that someone such as the OP feel the bitterness and frustration which was in that blog entry.
- quesera 12y ago> ... leads me to propose that maybe pouring all of that engineering effort into ZFS and dtrace (while both are amazing technologies, I will be the first to acknowledge) --- perhaps wasn't a good product/market fit ... My understanding is that neither ZFS nor DTrace were the products of large engineering teams. And the market is enthusiastically receptive to these products. If they didn't drive adequate additional hardware sales for Sun, it might be important to remember that Sun didn't charge for them... Regardless, I think you'd be seriously misunderstanding things to believe that engineering over-commitment contributed in any way to Sun's death.
- bcantrill 12y agoJust about everything said here about "Solaris engineering" is false. For example: Perhaps this means that Sun was brave where other companies feared to tread --- but given that (a) various ex-Sun employees have stated that there was a lot of Solaris engineering work was done before consulting the marketing or sales "tribes", and (b) ultimately, Sun wasn't able to succeed in the marketplace --- leads me to propose that maybe pouring all of that engineering effort into ZFS and dtrace (while both are amazing technologies, I will be the first to acknowledge) --- perhaps wasn't a good product/market fit, and that a VC who looked at things from a very cold-hearted profit and loss perspective would have never allowed a lot of the Solaris engineering work to go on as long as it had unless there was a demonstrated way it could be somehow monetized. Ignoring the terrible reasoning (correlation does not imply causation) and the child-like view of venture capital, this is wrong on the facts: the ZFS and (especially) DTrace teams were tiny; the work that we did was with management's blessing; the work that we did was a direct result of our experiences with customers; and (most importantly, from my perspective), the work paid for itself many, many times over. In fact, this work had paid for itself many times over even before 2006 when the core members of the DTrace and ZFS teams started a new effort inside of Sun to build a new line of products based on the two technologies.[1] The resulting product -- which we had initially projected to be a $1 billion business within five years -- paid for its entire effort in the first fifteen days of shipping product in late 2008, and went on to exceed those initial projections (acquisition by Oracle notwithstanding). It should also be mentioned that Keith was a key engineer on this effort -- and its outsized commercial success was actually a triumph of systems thinking, not an example of its defeat. So yes, Sun failed -- but it was because it ultimately heeded systems thinking too little, not too much.[2] [1] http://dtrace.org/blogs/bmc/2008/11/10/fishworks-now-it-can-be-told/ http://dtrace.org/blogs/bmc/2008/11/10/fishworks-now-it-can-... [2] https://news.ycombinator.com/item?id=2287033 https://news.ycombinator.com/item?id=2287033