15 ms·
The grand irony is that Larry was one of the earliest advocates of open sourcing the operating system at Sun[1] -- and believed that by the time Sun finally co
by bcantrill 10y ago
The grand irony is that Larry was one of the earliest advocates of open sourcing the operating system at Sun[1] -- and believed that by the time Sun finally collectively figured it out and made it happen (in 2005), it was a decade or more too late.[2] So on the one hand, you can view the story of BitKeeper with respect to open source as almost Greek in its tragic scope: every reason that Larry outlined for "sourceware"[3] for Sun applied just as much to BK as it did to SunOS -- with even the same technologist (Torvalds) leading the open source alternative! And you can say to BK and Larry now that it's "too late", just as Larry told Sun in 2005, but I also think this represents a forced dichotomy of "winners" and "losers." To the contrary, I would like to believe that the ongoing innovation in the illumos communities (SmartOS, OmniOS, etc.) proves that it's never too late to open source software -- that open source communities (like cities) can be small yet vibrant, serving a critical role to their constituencies. In an alternate universe, might we be running BK on SunOS instead of git on Linux? Sure -- but being able to run an open source BK on an open source illumos is also pretty great; the future of two innovative systems has been assured, even if it took a little longer than everyone might like.
So congratulations to Larry and crew -- and damn, were you ever right in 1993! ;)
[1] Seriously, read this: http://www.landley.net/history/mirror/unix/srcos.html http://www.landley.net/history/mirror/unix/srcos.html
[2] The citation here is, in that greatest of all academic euphemisms, "Personal communication."
[3] "Sourceware" because [1] predates the term "open source"
- luckydude 10y agoYeah this irony is not lost on me. But in both cases, the companies acted in self interest. Neither had the guts to walk away from their existing revenue stream. It's hard to say what would have happened. It's been an interesting ride and if nothing else, BK was the inspiration for Git and Hg, that's a contribution to the field. And maybe, just maybe, people will look at the SCCS weave and realize that Tichy pulled the wool over our eyes. SCCS is profoundly better.
- qwertyuiop924 10y agoI though SCCS had the same problems as RCS. What did it do differently?
- luckydude 10y agoRCS is patch based, the most recent version is kept in clear text and the previous version is stored as a reverse patch and so on back to the first version. So getting the most recent version could be fast (it isn't) but the farther back you go in history the more time it takes. And branches are even worse, you have to patch backwards to the branch point and then forwards to the tip of the branch. SCCS is a "weave". The time to get the tip is the same as the time to get the first version or any version. The file format looks like ^AI 1 this is the first line in the first version. ^AE 1 That's "insert in version 1" data "end of insert for version one". Now lets say you added another line in version 2: ^AI 1 this is the first line in the first version. ^AE 1 ^AI 2 this is the line that was added in the second version ^AE 2 So how do you get a particular version? You build up the set of versions that are in that version. In version 1, that's just "1", in version 2, that's "1, 2". So if you wanted to get version 1 you sweep through the file and print anything that's in your set. So you print the first line, get to the ^AI 2 and look to see if that's in your set, it isn't, so you skip until you get to the ^AE 2. So any version is the same time. And that time is fast, the largest file in our source base is slib.c, 18K lines, checks out in 20 milliseconds.
- qwertyuiop924 10y agoThat actually is pretty neat
- clacke2 10y agoAha, so that's where bzr got it from. :-)
- luckydude 10y agobzr got more than that from BK, it got one of my favorite things, per-file checkin comments. I liken those to regression tests, when you start out you don't really value them but over time the value builds up. The fact that Git doesn't have them bugs me to no end. BZR was smart enough to copy that feature and that's why MySQL choose bzr when they left BK. The thing bzr didn't care about, sadly, is performance. An engineer at Intel once said to me, firmly, "Performance is a feature".
- rbanffy 10y ago> the companies acted in self interest Sun, could, at least, make a profit building workstations and servers and licensing chips. It's actually very sad they don't build those SPARC desktops anymore.
- nickpsecurity 10y agoI really doubt it. There's not enough demand to make them competitive in price-performance. The server chips have stayed badass but still niche market. There's even open-source, SPARC HW with reference boards for sell from Gaisler. That didn't take off. I liked the desktops but there's no money in them. Market always rejects it. So does FOSS despite it being the only open ISA with mainstream, high-performance implementations.
- ryao 10y agoThe problems were Sun's failure to recognize that cheap IBM PC clones would disrupt them like they disrupted mainframes and Sun not trying hard enough to overcome Wintel's network effect. Sun needed to die shrink old designs to get something that they could fabricate at low cost and compete on price. Such a thing would have canabalized Sun workstation sales, which might be why they never did it.
- nickpsecurity 10y agoCould be. It's gone now, though.
- Annatar 10y agoYou again! (:-) You know that the VHDL code for UltraSPARC T1 and T2 has been open sourced? If I had enough knowledge about synthesizing code inside of an FPGA, I would be building my own SPARC-based servers like there is no tomorrow! As long as the code for those processors remains free, and a license to implement a SPARC ISA compliant processor only costs $50, the SPARC will never really, truly be gone, especially not for those people capable of synthesizing their own FPGA's, or even building their own hardware. Some people did exactly that, a while back. Too bad they didn't turn their designs into ready-to-buy servers.
- adharmad 10y agoThank you for contributing to the development and evangalizing of DVCS, directly (BK) and indirectly (the ideas and inspiration for git, hg).
- peonicles 10y agoIt's probably fair to say the DVCS accelerated the growth of the entire software industry. Was BitKeeper the first version control system to "think distributed" ?
- lobster_johnson 10y agoSun's TeamWare [1] was probably the first real distributed version control system. It worked on top of SCCS. Larry McBoy, BitKeeper's creator, was involved in its development. I believe BitKeeper also uses parts of SCCS internally. [1] https://en.wikipedia.org/wiki/Sun_WorkShop_TeamWare https://en.wikipedia.org/wiki/Sun_WorkShop_TeamWare
- luckydude 10y agoWe did a clean room reimplementation of SCCS and added a pile of extensions.
- qwertyuiop924 10y agoNSE begat NSE-lite begat TeamWare begat BK begat git. Or so says Cantrill.
- ryao 10y ago> in both cases, the companies acted in self interest. Neither had the guts to walk away from their existing revenue stream. Why does bitmover have the guts now?
- riffraff 10y agohe explained the move in another comment https://news.ycombinator.com/item?id=11668492 https://news.ycombinator.com/item?id=11668492
- bch 10y ago> realize that Tichy pulled the wool over our eyes. ref: https://en.wikipedia.org/wiki/Revision_Control_System https://en.wikipedia.org/wiki/Revision_Control_System, https://en.wikipedia.org/wiki/Walter_F._Tichy https://en.wikipedia.org/wiki/Walter_F._Tichy
- luckydude 10y agoYeah, that. I think he got a PhD for RCS and what he should have gotten is shown the door. RCS sucks, SCCS is far, far better. Just as an example, RCS could have been faster than SCCS if they had stored at the top of the file the offset to where the tip revision starts. You read the first block, then seek past all the stuff you don't need, start reading where the tip is. But RCS doesn't do that, it reads all the data until it gets to the tip. Which means it is reading as much data as SCCS but only has sorta good performance for the tip. SCCS is more compat and is as fast or faster for any rev. And BK blows them both away, we lz4 compress everything which means we do less I/O. RCS sucked but had good marketing. We're here to say that SCCS was a better design.
- qwertyuiop924 10y agoI actually posted before I saw your comment. I guess I was right. So are you happy to finally have an open source bring-over-modify-merge VCS whose command set makes sense?
- jordigh 10y agoThis is a bit funny because Mercurial is partly named after Larry. https://groups.google.com/d/msg/mercurial_general/c3_SM3p7S1k/3tvec0emdpcJ https://groups.google.com/d/msg/mercurial_general/c3_SM3p7S1...
- luckydude 10y agoJust for the record this thread is where I learned that. And yup, it sorta fits. For better or worse.
- SwellJoe 10y agoI've read that "sourceware" article before, in the distant past when it was still a roughly accurate picture of the market (maybe 1995 or 1996). It's weird to read it again now, in a world that is so remarkably changed. Linux, the scrappy little upstart with a million or so users at the time of the paper, is now the most popular OS (or at least kernel) on the planet, powering billions of phones and servers. NT was viewed as the unfortunate but inevitable future of server operating systems. I remember looking at IT jobs back then, and seeing a business world covered in Windows NT machines; I even got my MCSE (alongside some UNIX certifications that I was more excited about), because of it. Looking at jobs now, the difference is remarkable, to say the least. Nearly every core technology a system administrator needs to know is Open Source and almost certainly running on Linux. And, the funny thing is that the general prescription (make a great Open Source UNIX) is exactly what it took to save UNIX. It just didn't involve any of the big UNIX vendors in a significant way (the ones spending a gazillion dollars on UNIX development at the time). Linux got better faster than Sun got smarter, and ate everybody's lunch, including Microsoft. Innovator's Dilemma strikes again. Apple is an interesting blip on the UNIX history radar, too...though, they're likely to lose to the same market forces in the end, as phones become commodities. I'm a bit concerned that it's going to be Android, however, that wins the mobile world since Android is nowhere near the ideal OS from an Open Source and ethical perspective; but, I guess they got the bits right that Larry was suggesting needed to be right. Anyway, it was a weird flashback to read that article. Things change, and on a scale that seems slow, until you look back on it, and see it's "only" been a couple of decades. In the grand scheme of things, and compared to the motion of technology prior to the 1900, that's a blink of an eye.
- qwertyuiop924 10y agoEh. Linux does have a LOT of problems. Well, so does everything, but it's not like it's "great." More like "good." But yeah, it was weird that everybody thought NT was going to be the future. And now, MS has opensourced a good deal of infrastructure, is working with Node, has announced an integrated POSIX environment for Windows. And since it's in corporate, it might even be able to fix the fork(2) performance problems.
- 10y ago
- tikhonj 10y agoThe point about small communities is great. I've never seen it spelled out like that, but it captures what I was thinking perfectly. People get caught up on popularity as the only meaningful metric for success, but it really isn't.
- tw04 10y agoSun is also no more... and it's not at all clear they would've survived had Solaris been open sourced a decade earlier. Yes, you're absolutely right that there are a ton of startups built on opensolaris (who have proprietary code they haven't and don't intend to ever give back to the community), and there is smartos/omnios/illumos as well. But none of those projects would have in any way contributed to the health of Sun Microsystems, nor provided the funding to get Solaris to where it is today. ZFS may have never seen the light of day if Solaris were open sourced in 1995.
- ryao 10y ago> ZFS may have never seen the light of day if Solaris were open sourced in 1995. It depends on how that would have affected Jeff Bonwick. If it kept him from deciding that Sun ought to develop a new filesystem, promising Matthew Ahrens a job writing one out of college and working together with Matt on it, ZFS would never have existed.
- luckydude 10y agoSome history. I was at Sun and Bob Hagmann was teaching at Stanford and got me to be a TA there. He retired and Stanford asked me if I would teach CS240B so I did. Jeff Bonwick was student and I recognized his ability and recruited him to Sun. He said "I have no experience programming in C" and I said "You are smart. I can teach you C, I can't teach you smart". I also told him that he'd go way farther at Sun than I did and I was right, I think he made DE, I didn't. He played the game better. Smart guy. Him, Bryan, Bill Moore, those guys were the new Sun in my mind.
- ryao 10y agoIn that case, you are one of the guys on whose shoulders much of what I have done stands. Thank you.
- ryao 10y agoBit keeper is a great example of what happens when you do not open source your code. I have cited it that way many times.
- luckydude 10y agoExcept that we've been around for 18 years and made payroll without fail that entire time. Supported a team of 10-15 people every year. That's something, many many companies in the valley, including many that open sourced everything, have not done as well. You may have done more by open sourcing whatever it is that you have done; if so congrats.
- ryao 10y agoMy remark was intended to cite how much farther bit keeper could have gone had it been open source from the start rather than belittle what bit keeper accomplished. At work, many of my newer colleagues have backgrounds in closed source software development. We are developing software that has no exact analog to existing software, and we hope that it will have a big impact. If it becomes as important as we think it could be, then bit keeper vs git is a fantastic example of why our work should be open source from the start.
- debaserab2 10y agoExcept you're not Linus and you probably don't have a cult like following for any work you produce. Linus brought DVCS to the masses, but to pretend there wasn't more at play than simply open sourcing a project and hoping it all works out is complete rubbish. People have families to feed. Closed source is not inherently evil. It takes a unique situation to produce something like git that's product is beyond the sum of the project itself.
- ryao 10y agoI wrote enough patches to ZFSOnLinux that I have the distinction of number 2 by commit count. It was a hobby for me at first and quite frankly, I never expected it to make a difference for more than a few hundred people. Now ZoL is on millions of systems through Ubuntu in part because of my work and there are far more places using it than I can count. Open sourcing those patches rather than keeping them to myself made a difference that was greater than anything I imagined. Similarly, the impact of making ZFS open source far surpassed the expectations of the original team at Sun. I think that making any worthwhile piece of software open source will lead to adoption beyond the scope of what its authors envisioned. All it takes is people looking for something better than what previously existed. As for closed source being inherently evil (your words, not mine), how do you fix bugs in closed source software that a vendor is not willing to fix? How do you catch things like a hard coded password that gives root privileges? How do you know that the software is really as good as they say? It is far easier with open source software than with closed source software. Closed source software is a bad idea.
- Annatar 10y ago> And you can say to BK and Larry now that it's "too late", just as Larry told Sun in 2005, In case of software, it is never too late: as you once put it so eloquently, software does not suddenly stop working and does not have an expiration date. If this software works and works well, then Paul Graham's revolutionary idea of when you choose technology, you have to ignore what other people are doing, and consider only what will work the best applies. (Common sense really, but apparently not to the rest of our industry.) If this software will work the best, and do exactly what I want and need it to do, I have enough experience to know not to care that everyone else runs something like git just because that is trendy right now. (A lesson appreciated by those who run SmartOS because it is the best available technology for virtualization, cloud, and performance, instead of running Linux and Docker.)
- JdeBP 10y ago> software [...] does not have an expiration date. xscreensaver does. (-: * https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=819703 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=819703 * https://bugs.launchpad.net/ubuntu/+source/xscreensaver/+bug/1406825 https://bugs.launchpad.net/ubuntu/+source/xscreensaver/+bug/... * https://news.ycombinator.com/item?id=11412081 https://news.ycombinator.com/item?id=11412081