9 ms·

BitMover/BitKeeper. Nice idea but too late, the market has gone to git. Which chaps my hide, git is an awful design, it's a tarball server pretending to be a

BitMover/BitKeeper. Nice idea but too late, the market has gone to git. Which chaps my hide, git is an awful design, it's a tarball server pretending to be a source management system. Source management is basically an accounting program for programmers, it's suppose to faithfully log what you do. How does git do that when it has no concept of a file? Answer: it guesses about renames. And blame is slow because there are no file create or file delete events. It was trivial for us to create a bk fast-export command and make it work incrementally. It's next to impossible to do it the other way incrementally because git doesn't have a versioned file object. It's like a file system that has no inodes. To me, it makes no sense. But it won. Yeah. Not.

chubot 9y ago
Thanks for the background. There are indeed awful things about git's design, but I think it would have been very hard to out-market git. Git is both free AND viral. It's viral because if you want to use someone else's code, you have to use their source control system. Linux using git is a big deal in that respect (and yes I realize the kernel used BitKeeper :) ). Also, Github as company and git the open source project are pretty complementary. It's hard to house both entities under the same roof, and you need that for ecosystem to succeed. Github is basically a UI/app company, and git is systems plumbing. It's analogous to some PaaS companies wanting to make their own web framework. No actually that's a huge amount of work and you underestimated it. For all its faults, rewriting Rails is not something anyone can do. Heroku ended up being one of the early success stories because they just took your rails app and ran it. Likewise everyone thinks they can rewrite Github, but that's pretty hard. It's at least 10x more work than you think, and you can get it wrong. If a tool isn't open source, there's much less incentive for a company like Github to come around and "popularize" it. It seems like it's very hard to make money in developer tools in general, because so much is open source. And honestly they should be -- I would be a hypocrite if I said I didn't choose tools because they are open source. I guess Github and Atlassian are the outliers. Anyway, my overall point is that I wouldn't necessarily chalk it up to marketing :) I do think open source got much bigger in the last decade, and it's hard for me to imagine that it wouldn't be built around an open source VCS.
- EpicEng 9y ago
  >Git is both free AND viral. It's viral because if you want to use someone else's code, you have to use their source control system. Linux using git is a big deal in that respect (and yes I realize the kernel used BitKeeper :) ). And BK is (was, until last year [too late]) closed software that requires a paid commercial license to use. Pretty much a show stopper for open source contributors donating their free time.
  - luckydude 9y ago
    Yeah, too late. Shoulda, coulda, woulda. There's some nice code in there though, I wish people would steal our libc stuff. Some really nice ideas in there. And a pleasant license so you can use it where ever you want.
    - Artlav 9y ago
      It would make it much easier to steal if you mention (that is, market) which ideas in there are worth stealing ;)
      - luckydude 9y ago
        I've long thought I should do a blog on it and submit it here, just haven't gotten around to it. Until then... The "lines" data structure. This is a list library that scales down to a very small thing and up to pretty decent sized things. Think perl arrays and the ops on them. lines.h snippet below. /* * liblines - interfaces for autoexpanding data structures * * s= allocLines(n) * pre allocate space for slightly less than N entries. * s = addLine(s, line) * add line to s, allocating as needed. * line must be a pointer to preallocated space. * freeLines(s, freep) * free the lines array; if freep is set, call that on each entry. * if freep is 0, do not free each entry. * buf = popLine(s) * return the most recently added line (not an alloced copy of it) * reverseLines(s) * reverse the order of the lines in the array * sortLines(space, compar) * sort the lines using the compar function if set, else string_sort() * removeLine(s, which, freep) * look for all lines which match "which" and remove them from the array * returns number of matches found * removeLineN(s, i, freep) * remove the 'i'th line. * lines = splitLine(buf, delim, lines) * split buf on any/all chars in delim and put the tokens in lines. * buf = joinLines(":", s) * return one string which is all the strings glued together with ":" * does not free s, caller must free s. * buf = findLine(lines, needle); * Return the index the line in lines that matches needle */ The stdio stuff. We have a way that you can "push" code on top of a FILE. Some layers we implemented were compression and CRC and XOR. The compression is lz4 (we may still have gzip); the CRC stuff splits the file up into "blocks" (size is uniform in a file but it varies by overall file size), crcs each one. The XOR stuff adds an XOR block at the end. The result to a user is if they fpush(&f, fopen_crc(...)); then the file will be crc "behind the scenes". That fpush is the only new line of code to get the crc stuff. Ditto for compression. It gives you compression and hardened files under the stdio API and it's super fast. Like GB/sec fast. There's other stuff as well, like the spawn() code (I haven't called fork() in a decade or more) that brings a form of Windows spawn to Unix (since you can't make fork() work well on Windows). All of the code is Windows/Unix portable.
iamacynic 9y ago
you probably could have captured a significant commercial marketshare but i highly doubt you could have beaten git in the open source world, since it was created by your highest profile customer who also happens to be the highest profile open source author (linus torvalds) in direct response to your licensing situation.
- luckydude 9y ago
  Yeah, well. We did what we could. I've been beat up about the licensing and eventually saw the light, the whole thing is open source under the Apache v2 license. As to the "direct response" stuff, so do you have that straight from Linus? Because that's not what he said to me and we were very close at the time, emailed or got on the phone daily. I'm a little tired of the licensing stuff. It wasn't as bad as people made it out to be but I get it, it's a more fun story when someone is the bad guy. Can I ask you to let it go or is that too much?
  - iamacynic 9y ago
    no, of course not. i'm a nobody. but that's the impression that's out there. i don't really care, tbh, it seems like it was just bad timing. anyway, glad to hear you are retired by the sea in norcal, that's not exactly a sob story, which is why i feel comfortable commenting about it (and you're on a public forum).
    - luckydude 9y ago
      I was a nobody and am again. Don't sell yourself short, if you are hanging here and you know the licensing stuff you are paying attention. With that much energy my feeling is you can do stuff. Go do it. If I can help let me know.

chubot 9y ago

Thanks for the background. There are indeed awful things about git's design, but I think it would have been very hard to out-market git. Git is both free AND viral. It's viral because if you want to use someone else's code, you have to use their source control system. Linux using git is a big deal in that respect (and yes I realize the kernel used BitKeeper :) ). Also, Github as company and git the open source project are pretty complementary. It's hard to house both entities under the same roof, and you need that for ecosystem to succeed. Github is basically a UI/app company, and git is systems plumbing. It's analogous to some PaaS companies wanting to make their own web framework. No actually that's a huge amount of work and you underestimated it. For all its faults, rewriting Rails is not something anyone can do. Heroku ended up being one of the early success stories because they just took your rails app and ran it. Likewise everyone thinks they can rewrite Github, but that's pretty hard. It's at least 10x more work than you think, and you can get it wrong. If a tool isn't open source, there's much less incentive for a company like Github to come around and "popularize" it. It seems like it's very hard to make money in developer tools in general, because so much is open source. And honestly they should be -- I would be a hypocrite if I said I didn't choose tools because they are open source. I guess Github and Atlassian are the outliers. Anyway, my overall point is that I wouldn't necessarily chalk it up to marketing :) I do think open source got much bigger in the last decade, and it's hard for me to imagine that it wouldn't be built around an open source VCS.

EpicEng 9y ago
>Git is both free AND viral. It's viral because if you want to use someone else's code, you have to use their source control system. Linux using git is a big deal in that respect (and yes I realize the kernel used BitKeeper :) ). And BK is (was, until last year [too late]) closed software that requires a paid commercial license to use. Pretty much a show stopper for open source contributors donating their free time.
- luckydude 9y ago
  Yeah, too late. Shoulda, coulda, woulda. There's some nice code in there though, I wish people would steal our libc stuff. Some really nice ideas in there. And a pleasant license so you can use it where ever you want.
  - Artlav 9y ago
    It would make it much easier to steal if you mention (that is, market) which ideas in there are worth stealing ;)
    - luckydude 9y ago
      I've long thought I should do a blog on it and submit it here, just haven't gotten around to it. Until then... The "lines" data structure. This is a list library that scales down to a very small thing and up to pretty decent sized things. Think perl arrays and the ops on them. lines.h snippet below. /* * liblines - interfaces for autoexpanding data structures * * s= allocLines(n) * pre allocate space for slightly less than N entries. * s = addLine(s, line) * add line to s, allocating as needed. * line must be a pointer to preallocated space. * freeLines(s, freep) * free the lines array; if freep is set, call that on each entry. * if freep is 0, do not free each entry. * buf = popLine(s) * return the most recently added line (not an alloced copy of it) * reverseLines(s) * reverse the order of the lines in the array * sortLines(space, compar) * sort the lines using the compar function if set, else string_sort() * removeLine(s, which, freep) * look for all lines which match "which" and remove them from the array * returns number of matches found * removeLineN(s, i, freep) * remove the 'i'th line. * lines = splitLine(buf, delim, lines) * split buf on any/all chars in delim and put the tokens in lines. * buf = joinLines(":", s) * return one string which is all the strings glued together with ":" * does not free s, caller must free s. * buf = findLine(lines, needle); * Return the index the line in lines that matches needle */ The stdio stuff. We have a way that you can "push" code on top of a FILE. Some layers we implemented were compression and CRC and XOR. The compression is lz4 (we may still have gzip); the CRC stuff splits the file up into "blocks" (size is uniform in a file but it varies by overall file size), crcs each one. The XOR stuff adds an XOR block at the end. The result to a user is if they fpush(&f, fopen_crc(...)); then the file will be crc "behind the scenes". That fpush is the only new line of code to get the crc stuff. Ditto for compression. It gives you compression and hardened files under the stdio API and it's super fast. Like GB/sec fast. There's other stuff as well, like the spawn() code (I haven't called fork() in a decade or more) that brings a form of Windows spawn to Unix (since you can't make fork() work well on Windows). All of the code is Windows/Unix portable.

iamacynic 9y ago

you probably could have captured a significant commercial marketshare but i highly doubt you could have beaten git in the open source world, since it was created by your highest profile customer who also happens to be the highest profile open source author (linus torvalds) in direct response to your licensing situation.

luckydude 9y ago
Yeah, well. We did what we could. I've been beat up about the licensing and eventually saw the light, the whole thing is open source under the Apache v2 license. As to the "direct response" stuff, so do you have that straight from Linus? Because that's not what he said to me and we were very close at the time, emailed or got on the phone daily. I'm a little tired of the licensing stuff. It wasn't as bad as people made it out to be but I get it, it's a more fun story when someone is the bad guy. Can I ask you to let it go or is that too much?
- iamacynic 9y ago
  no, of course not. i'm a nobody. but that's the impression that's out there. i don't really care, tbh, it seems like it was just bad timing. anyway, glad to hear you are retired by the sea in norcal, that's not exactly a sob story, which is why i feel comfortable commenting about it (and you're on a public forum).
  - luckydude 9y ago
    I was a nobody and am again. Don't sell yourself short, if you are hanging here and you know the licensing stuff you are paying attention. With that much energy my feeling is you can do stuff. Go do it. If I can help let me know.