26 ms·

Apache considered harmful

300 points by timf 15y ago

davidw 15y ago
Some good points, but github doesn't take the place of a community. When it's working well, it helps, but when there is a breakdown of collaboration and communication, you get one of those codebases that has been forked 1298 times where none of the people doing the forking is sharing anything. That's a community fail, not a version control system issue. I do think that it'd be nice if the ASF offered git alongside svn, and concentrated on the community aspect of things, which it does tend to do fairly well.
- latchkey 15y ago
  Just because you can't 'see' the fork with subversion doesn't mean it doesn't exist. I'm sure plenty of ASF projects are 'forked' within companies and the code is never shared. What I've been advocating for a while now on the members@apache list is that the ASF look at using Github (either they host it or the ASF does) as the basis to build a new type of community that the ASF has never experienced before. Something that isn't tied to the old school.
  - davidw 15y ago
    Sure, github may bring more things into the open, but it is ultimately just an aid to a community of people, who must communicate about the project. You can't just dump the code on github without communicating with other people working on it. I agree that outsourcing some of the infrastructure to github might be an interesting idea.
compay 15y ago
Hate to be the grammar cop here, but the consistent misspelling of "its" in the article is distracting. If the author is here, could you please fix that? It's taking away from a very well-written and insightful piece of writing.
- rukkyg 15y ago
  I agree. There are also several other typos like missing letters or extra words.
- greatquux 15y ago
  I stopped reading at the misuse of weary for wary.
apu 15y ago
Note that this article is about the Apache Software Foundation, not the webserver.
- andrewflnr 15y ago
  Indeed. It would be nice if the mods (we have mods here, right?) would change the title to make that clear.
- ccashell 15y ago
  Yeah, I found the title to be misleading and quite annoying. It should definitely be fixed to clarify. To the vast majority of the IT world, Apache == Web Server, not Apache Software Foundation.
ryanmarsh 15y ago
I'm curious as to the average age of committers by project under the ASF vs. popular projects on github. My hypothesis is that they would be older by a significant margin.
- chaseideas 15y ago
  I would agree with that hypothesis. What I've observed from running http://www.Apache.com http://www.Apache.com for several years, is indeed an older crowd (40+) by a nice margin compared to a lot of the younger projects floating around that are generating a ton of buzz. It's been much more rare in my experience to see a 20 something hipster programmer seriously diving in with the ASF. I'm generalizing though of course... The type of questions and people I interact with through that project are older engineering types, and those with a long history in the programming and computer scene. Usually with an old-school *nix approach to things. Just wanted to chime in with that, since like stated elsewhere in the HN comments, I think this blog post is more about the organization structure and members of the ASF than the actual Apache Web Server project... which we all love so dearly. ;) - Chase
- nirvdrum 15y ago
  I think you're right, but they overlap too, so hard to give a completely clear picture. I.e., every Apache person I've met also uses GitHub. Obviously the converse is not true.
nknight 15y ago
Much of this is pretty unsurprising, especially for people like me who watched the attempted transition of OOo to Apache. The way the article puts it is that the ASF is trying to solve problems that don't exist anymore, which is true to an extent, but the deeper problem is that the ASF has a particular view of how open source development and project management work, and attempts to impose that view on far too diverse a community, even as it tries to absorb more and more communities. The ASF is simultaneously trying to be "big tent" and unified, and the balance is all out of whack. It's easy to draw parallels to recent political problems in the US and EU. In all three cases, there's going to have to be some transformations, probably in both society/community and structure, to come back to a place where the institution contributes to the greater good, instead of being a source of unending tension and meta-arguments.
- jzawodn 15y ago
  Very well said. I was going to write something like that but don't have to. Yay for up votes!
latchkey 15y ago
Well written. I wrote a similar article recently as well... http://lookfirst.com/2011/11/contributing-to-open-source.html http://lookfirst.com/2011/11/contributing-to-open-source.htm...
jowiar 15y ago
I second this thoroughly. I was was almost driven to start blogging last week by ASF's poor job of maintaining its projects. There was a small bug in Solr. I was not the first to find this bug, and someone had not only reported the bug, but filed a patch on the bug tracker a year and a half ago. The patch was never merged in, nobody provided any feedback as to why the patch wasn't merged in. One huge plus with Github is that if the official steward of a project would like to hand it off to someone else, or is failing to maintain it, it is trivial for someone else to take over the project.
- jaaron 15y ago
  Then fork it. It's open source for a reason. Or raise a fuss on the mailing list. That's how open source works.
  - jerf 15y ago
    While forking is a solution, it hardly precludes discussion of other less drastic potential solutions to the problem at hand. It does nicely bound the maximum negative impact that Apache social problems can cause, though.
    - jaaron 15y ago
      If you're using git, as the OP suggets, then you're going to fork it just to work on it. Forking isn't dramatic. Code is open source for a reason. If you have a critical bug in code you need running in your infrastructure, take ownership of it and fork it. Then do the dirty work to get the patch pushed back up stream. If the maintainers really aren't doing what they volunteered to do, then volunteer yourself and get it done.
      - jerf 15y ago
        Sorry, explain to me clearly why that is the One Answer and we must stop discussing alternatives? You're answering a question I'm not even remotely asking.
ww520 15y ago
I'm curious how does it harm anything? Did it kill any puppies? Maybe it's inefficient but harmful? Also remember GitHub is a for profit company. Its allowance for Open Source hosting is a marketing tactic. Anytime they feel the marketing value is not there, they will shut it down. Not that I'm against GitHub. It's a great company for itself. But comparing the Apache to GitHub is like comparing apple and orange.
- Legion 15y ago
  Don't get caught up on GitHub specifically. GitHub is just the most popular example of how projects don't need Apache to host them anymore, yet Apache still expects to do so, and worse, expects to assert a lot of restriction over it. The point is, that is at odds with what the community wants and needs. As the author pointed out, ten years ago, rolling your own SCM hosting was a big pain. Now, it's not, partly because of GitHub and Bitbucket and others, but also because rolling your own isn't as hard either. Anyone with minimal server admin experience and knowledge of Git can run their own Git server on a VPS with something like Gitolite. I know because I succeeded in doing so myself, and I'm neither a pro server admin, nor did I have any Git experience at the time I did the initial setup of Gitolite. Prior to that, I had set up a Mercurial server with no prior Mercurial experience either. It's pretty easy now. So, yeah, GitHub is there, but GitHub could disappear tomorrow and the community still wouldn't need to turn to Apache for project hosting. In that respect, they're still solving a problem nobody has anymore, and that was the point the author was making.
  - hrabago 15y ago
    GitHub isn't a new version of Apache - GitHub is a new version of SourceForge. I don't think that even 10 years ago anyone with a line (or 100 lines) of code can set up their own Apache projects. 10 years ago if you needed free SCM, you'd use SourceForge, not Apache. I don't think it would've been that big of a pain even then. GitHub projects don't necessarily come with its own community with diverse contributors, whereas Apache projects require it.
  - nl 15y ago
    In that respect, they're still solving a problem nobody has anymore, and that was the point the author was making. That wasn't the author's point at all.
jaaron 15y ago
Ok, there's a lot to cover here. First off, the Apache Software Foundation isn't trying to absorb anyone or anything. Projects and people come to the ASF. It's a specific policy of the Foundation to NOT solicit projects. If someone says they're representing Apache and soliciting projects, they're wrong. Secondly, Apache is very opinionated about how projects should be run. This comes from years of experience as not only a successful project, but as a successful non-profit organization overseeing dozens of projects. If a community doesn't like the ASF's style or rules (such as no dictators, benevolent or otherwise), they don't need to be there. No one wants to keep projects hostage. Part of the point of the Incubator is to get this figured out earlier than later. Thirdly, about git and subversion. First off, there's increasing support for git at Apache (see http://git.apache.org/ http://git.apache.org/) but there are some serious drawbacks for use of git. Consider this: subversion was practically made for Apache in the way Linus made git for Linux. With that in mind, subversion isn't going anywhere at the ASF. Some of the rational is just plain stubbornness, but some of it goes straight to the core values of the Foundation. Apache has become, for better or worse, the place where lots of projects go when they grow up. Growing up is hard to do. It's not fun. You have to do things like get a job, pay taxes, etc. When a project grows up, people start caring about who contributed what, under which license and making sure every line of code is legit. A lot of engineers don't care about this, but businesses and their lawyers do. A lot of the Apache Foundation "bureaucracy" is to handle this oversight and paperwork. Git is an impressive tool and github is awesome for what it is, but it's not a non-profit foundation and it won't replace one. Confusing the Apache Software Foundation for your coding sandbox only suggests you don't understand the true purpose of either.
- visural 15y ago
  While correct, it would benefit Apache in the long run to foster better communities and tools for their projects to grow. While they provide value in terms of the protection/process/management it's all for nothing if the developers can't collaborate effectively. GitHub has changed many developer's UX of opensource development to the point that they don't want to do it "the old way". Apache should be looking to grow in this direction to keep their developers and projects engaged.
VladRussian 15y ago
push vs. pull. Basically the tools and overall environment grew up to have pull as a feasible model.
apike 15y ago
The ASF is the first home that comes to mind when an successful open-source project needs independent stewardship. Often when a company wants to "spin off" an open source project, they turn to Apache. What alternative organizations fill this need in a more lightweight fashion? Most other umbrella open source organizations I know of focus on copyleft and other issues that can be hostile to commercial interests.
- sanxiyn 15y ago
  Software Freedom Conservancy. http://sfconservancy.org/ http://sfconservancy.org/ Conservancy doesn't care about licenses as long as they are free. For example, jQuery (MIT license) is part of Conservancy. Conservancy members include prominent projects such as Boost, BusyBox, Darcs, Git, Inkscape, jQuery, Mercurial, PyPy, Samba, Selenium, Squeak, uClibc, Wine.
  - ternaryoperator 15y ago
    "Conservancy doesn't care about licenses as long as they are free." They do care about licenses and license terms are part of the requirements for application. The project license must be either free (per FSF) or open (per OSI). Docs must be made available under Creative Commons licenses. And the project must be completely non-profit. (All these requirements must be met.) That being said, you're right, it's a good home without the politics discussed here.
  - apike 15y ago
    Very cool. It seems like they are the type of organization Mikeal is encouraging ASF to become: legal and administrative support for open source projects, and other services if the project's leaders wish. Quite hands-off. Comparing the lists of projects, I'm surprised to find I use more SFC software than ASF software.
inopinatus 15y ago
This essay put me in mind of The Cathedral and the Bazaar. It is a neat demonstration of how tools and processes are inextricably linked. Who knew, though, that the ASF would be cast as the new priests in the cathedral? I suppose it took a whole new level of social development, enabled by tools, to cast them in that role.
- rgardler 15y ago
  Actually I think it is either: a) a complete misunderstanding of how and why the ASF operates the way it does or b) a desire for sensationalist blog pieces with almost no factual content The ASF is working with Git, it has been for years. It doesn't yet provide a canonical repository from which to make releases. This is due to a number of non-trivial technical issues introduced by the processes adopted by Apache projects. The Apache infrastructure team believe that they have now solved those issues and are testing them in CouchDB. Assuming the CouchDB experiment is a success the ASF will be rolling out Git as the canonical repository to all projects that want it. Once the ASF has mapped the tools to the processes we can all move on and stop wasting our time with this spurious argument. Disclaimer: Unlike the author of this blog I do have access to all the discussions about Git in the ASF and I am one of the mentors of PhoneGap, a project mentioned in the article.
polychrome 15y ago
So you're saying a system that has had repeated successes is harmful. I really think you make a good point here about the need to remain open to change. So talk about that. Obviously github has some very positive impact. How can Apache adapt to that? You're not really talking about the tools here, you're talking about community. I see a potential solution here being that Apache has different rules for projects in different stages. Do you think that would solve the issues? Remember, you're view of anarchy on GitHub will only last so long. Rules and order come out of anarchy for a reason and like all things GitHub will become the exact same stale community you're complaining of now in 10 years.
linuxhansl 15y ago
I am not entirely sure what the article is trying to get at. Politics and law in open source are real and needed, especially in the face of software patents. Many contributors develop open source code as part of their paid work, as such it is quite important to establish the legal framework to allow contributing the companies IP to an open source project (which includes necessary patent grants). Comitters need to submit an Individual Contributor License Agreement stating that they have the legal right to contribute the code they're contributing. If worked on contributions as employee the company also typically needs to submit a corporate contributor license agreement. Like it or not (and I personally do very much not like it), you cannot just upload some code somewhere these days. As such the even FSF is an extremely important organization. Much frowned upon and usually not understood. Open source licenses would not work without Copyright Law, most developers don't know or understand that. And the whole thing of svn vs git. Personally I don't get it. I use svn when it makes sense and I use git when that makes sense, just like I use Java/Ruby/C++/Python/Javascript/Closure/Scala/whatever or GoogleDocs/PDF/OpenOffice/MS Word(yes) when needed. The main feature I need from VCS are atomic commits. So CVS is out for me for that reason. Sure git's nice and all, but I spend 99.9% of my time writing or thinking about code and software architecture, not tinkering with my VCS, so as long as it works, I don't care. Easy forking and branching is nice too. In the end, though, just as with Linux there is/are some de-facto master branch(es) somewhere from which "releases" are cut. Currently with apache it's more convenient to use svn, so I do that. I don't get the religious opposition against one version control system vs another. Disclaimer: Apache committer here.
- bad_user 15y ago
  Ever since I started using Git, I would never go back to SVN, because branching and merging is now an essential part of my workflow, even on small project on which I work by myself with no other contributers. just as with Linux there is/are some de-facto master branch(es) Well yeah, but I don't get how that's an argument for SVN. The thing I like about Git is that branching is now really, really cheap. You can now keep track of dozens of local branches with experiments that you don't have to push to master. You can now share your experiments with a colleague and push to master whenever something is actually ready. You can now also ban commits to the main repository that haven't been code-reviewed (something which is a PITA with SVN). And so on and so forth. I don't get the religious opposition against one version control system vs another. Even though I prefer Git, neither do I, especially since you can just use the Git-SVN bridge :) I've used it for more than a year, it does has some quirks, but it works fine. Also, the Apache Foundation does its job and does it well. There's room for both anarchy and bureaucracy and both are needed.
ck2 15y ago
I wish at least one open source replacement adopted .htaccess (and httpd.conf) compatibility. Litespeed is the only product in existence which has made switching over from a complex Apache install a one hour affair, but it's free version is limited to 5 hosts and the commercial version can only be justified in a profitable environment. The performance difference is breathtaking however.
aristedes 15y ago
Where to start with this blog post? It appears that the author has seen a couple of private emails and thinks he knows all about the internal workings of the Apache Foundation. He is wrong on so many counts. His entire dislike of the Apache Foundation appears to be predicated on the fact that the organisation did not force every project to move to this blogger's favourite version control tool. Making a change as large as this requires many different things, but in particular: 1. Community change. How committers interact with each other when there are lots of forks is quite different to the current situation. That suits some projects and not others. Not every project at Apache will benefit. Some will. All who change will need to think long and hard about release processes, merging strategies and much more. Git encourages the idea that every commit or fork is completely equal to every other fork or commit. The Apache Foundation is built on the concept of meritocracy: commit rights are given in response to demonstrated skill. This is not an intractible problem with git, but new challenges need to be solved. 2. Legal change. Right now there is a simple process for signing off intellectual property for contributions which were merged from external contributors (who have not signed a release). That changes with git and becomes more complex. There are solutions, but they require careful planning. 3. Infrastructure. Hosting a large git repository with the level of downtime acceptable to Apache isn't something you do quickly. That needs planning and maintenance. 4. Toolsets. Lots of things in Apache are tied into subversion. From mailing list commit hooks to build servers and much more. Changing those things takes work. 5. Splitting the community. Right now the entire organisation's intellectual property is held in a single repository. Everyone knows where everything is to be found. Changing this simplicity requires a very good reason. So what do we have now? A blogger who (it appears) doesn't actually contribute code to any Apache project. Telling other people how to run their organisation (which is wildly successful). And that they should change to this blogger's favourite new tool (they should have done it in 2008!) or face irrelevance. If Apache moved every project to github tomorrow would that satisfy this blogger? More importantly, would that have caused this guy to commit high quality code toward one of the Apache projects? Or is he just blowing a lot of hot air about something he knows little? And what brought on this great complaint? That the Apache Foundation is currently underway with trials for one project to see how git would succeed for their workflow. And to then evaluate its suitability for other projects across Apache. Apache is not Github. That is, Apache is much more than a website, a couple of tools and a repository of code of random quality. Disclaimer: I am an Apache member, but not speaking on behalf of Apache
sktrdie 15y ago
1) Apache Software Foundation and GitHub are two totally different things. Who cares about their internal preferences and bureaucracies. They're both producing outstanding open-source projects which are used by hundreds of thousands of companies and people. Open-source (and the world) has only gained positive things out of these communities. 2) If you're suggesting that ASF needs to change its bureaucracy, I disagree. Frankly, I feel the bureaucracy has worked, given the success the foundation projects have had. 3) I'm not sure what other points your post brings, but if you're simply just saying that ASF needs to keep itself up-to-date with new tech (dunno git?) then this is also a totally absurd argument since the tech being used in Apache is totally amazing and new. I feel like your outcry is referred to general institutions... you should probably refer to governments and other political entities instead of bashing on a foundation that has given the world amazing products.
abrahamsen 15y ago
Free project infrastructure wasn't hard to setup five years ago. It hasn't really been a problem since 1999 when SourceForge opened. Before that, the SunSites did a nice job, and before that you basically had to know a friendly university sysadm (which wasn't _that_ hard to find). I'm not sure what people think they gain by going under the Apache umbrella, but it must be something since they bother. There are no lack of alternatives.
luriel 15y ago
This might be slightly offtopic, but might be a symptom of the "institutional"/"organizational" issues addressed in the article: I always thought Apache 2 and Subversion were two of the best examples of second-system effect. I mentioned this once to one of the core Apache (and Svn) developers years ago, and not only was he blissfully unaware of the effect, he indicated that he had helped build incredibly successful pieces of software (ie., Apache 1.x) and didn't need any advice from from Fred Brooks or anyone on how to do it. Both Apache 2 and svn have been extremely successful projects, but both were late, didn't really match expectations or even the success of their predecessors, and are slowly being outcompeted by much smaller and usually more efficient projects (eg., nginx, lighttpd, git, hg) that are developed much more quickly by much smaller teams.
delinka 15y ago
Or perhaps "Apache [Foundation] considered harmful" since in common usage "Apache" tends to mean "The Apache web server."
- rmc 15y ago
  Agreed. The title should be edited.
suprgeek 15y ago
Some people at the ASF think that Git may have some issues so are not wholeheartedly endorsing it for any and all projects. Somehow this gets translated into "Apache considered harmful". Why is this even on the front page?
- funkah 15y ago
  I would be seriously pissed if I donated my time and effort towards something like ASF, trying to help the open source world, only to be "considered harmful". WTF.
wyck 15y ago
..why isn't project x on git /rant tl;dr Github has a great social interface, it is a tool, there are others and there will be more. Yes it is an evolution towards DVC + great social interfaces, older industry mammoths will be slower to adopt then newer more agile projects. ps. I just switched from git to mercurial and it's a breath of fresh air.
nirvdrum 15y ago
Just to get two things out of the way: I'm an ASF member (albeit not very active lately) and a huge fan of git with or without GitHub. I'm one of the many people advocating for git internally at the ASF. I have been met with opposition in the past, but a lot of it has been around who's going to maintain the infrastructure, given it's a volunteer system. Let's just take it as axiomatic that the ASF is going to self-host its code. So it's at least a fairly pragmatic argument. And I think we finally have a solution. My real issue is with the bouncing back-and-forth the author does in his post around the notion of IP. It's a shitty topic that most devs don't want to be bothered with, but alas, it's quite important in the real world. And GitHub is mostly a landmine field when it comes to this. I don't think it's a failing of GitHub itself, but most projects just don't have licenses attached to them. Unlike with SourceForge, there's no requirement to have an OSS license on public projects. Then many that do fail to meet the copyright header requirements for the license. Or you could have a public project with a restrictive license [1]. Being public doesn't mean you get to do whatever you want with the code. This is dangerous and bad for OSS. Apache gives you that protection. There's never any question about it. That's the primary reason projects go through the incubator -- to make sure the IP is all in order. It's an annoying, bureaucratic, but necessary process in a litigious society. But because of the care and protections Apache provides in this regard, I think they've done more to get OSS adopted in traditionally closed companies than just about anyone else. [1] I came across Tom Preston-Werner's repo for his site. He's one of the GitHub founders. It's a public repo with a license that restricts usage of certain portions of the project (generally his content): https://github.com/mojombo/mojombo.github.com https://github.com/mojombo/mojombo.github.com
- MattRogish 15y ago
  This is a good point. Anyone who starts an open-source project should, from day one, have a vetted Contributor License Agreement and ONLY accept pull requests that include signed CLAs (or from a person who has one on file).
  - nirvdrum 15y ago
    It'd be kinda nifty if GitHub had this built-in. I personally don't require CLAs for every project because it can be onerous. But at the least I try to pick a license that wouldn't allow for submarine patent claims.
caniszczyk 15y ago
Well written... I wrote a response last night that dives into some numbers and my experiences with helping the Eclipse Foundation move to Git and Gerrit... http://aniszczyk.org/2011/11/23/apache-and-politics-over-code/ http://aniszczyk.org/2011/11/23/apache-and-politics-over-cod... Apache is pretty much the last major open source community to not move to some form of distributed version control. It's either politics (they host the Subversion project there) or negligence in my opinion.
jmathes 15y ago
"Considered Harmful" considered pretentious
pyrhho 15y ago
As an aside, the Clay Shirky quote ("Institutions will try to preserve the problem to which they are the solution.") was new to me, but puts the RIAA/MPAA pretty much perfectly into perspective. Not really related to the article, but it clicked as I was reading.

300 points by timf 15y ago

Some good points, but github doesn't take the place of a community. When it's working well, it helps, but when there is a breakdown of collaboration and communication, you get one of those codebases that has been forked 1298 times where none of the people doing the forking is sharing anything. That's a community fail, not a version control system issue. I do think that it'd be nice if the ASF offered git alongside svn, and concentrated on the community aspect of things, which it does tend to do fairly well.

latchkey 15y ago
Just because you can't 'see' the fork with subversion doesn't mean it doesn't exist. I'm sure plenty of ASF projects are 'forked' within companies and the code is never shared. What I've been advocating for a while now on the members@apache list is that the ASF look at using Github (either they host it or the ASF does) as the basis to build a new type of community that the ASF has never experienced before. Something that isn't tied to the old school.
- davidw 15y ago
  Sure, github may bring more things into the open, but it is ultimately just an aid to a community of people, who must communicate about the project. You can't just dump the code on github without communicating with other people working on it. I agree that outsourcing some of the infrastructure to github might be an interesting idea.

compay 15y ago

Hate to be the grammar cop here, but the consistent misspelling of "its" in the article is distracting. If the author is here, could you please fix that? It's taking away from a very well-written and insightful piece of writing.

rukkyg 15y ago
I agree. There are also several other typos like missing letters or extra words.
greatquux 15y ago
I stopped reading at the misuse of weary for wary.

apu 15y ago

Note that this article is about the Apache Software Foundation, not the webserver.

andrewflnr 15y ago
Indeed. It would be nice if the mods (we have mods here, right?) would change the title to make that clear.
ccashell 15y ago
Yeah, I found the title to be misleading and quite annoying. It should definitely be fixed to clarify. To the vast majority of the IT world, Apache == Web Server, not Apache Software Foundation.

ryanmarsh 15y ago

I'm curious as to the average age of committers by project under the ASF vs. popular projects on github. My hypothesis is that they would be older by a significant margin.

chaseideas 15y ago
I would agree with that hypothesis. What I've observed from running http://www.Apache.com http://www.Apache.com for several years, is indeed an older crowd (40+) by a nice margin compared to a lot of the younger projects floating around that are generating a ton of buzz. It's been much more rare in my experience to see a 20 something hipster programmer seriously diving in with the ASF. I'm generalizing though of course... The type of questions and people I interact with through that project are older engineering types, and those with a long history in the programming and computer scene. Usually with an old-school *nix approach to things. Just wanted to chime in with that, since like stated elsewhere in the HN comments, I think this blog post is more about the organization structure and members of the ASF than the actual Apache Web Server project... which we all love so dearly. ;) - Chase
nirvdrum 15y ago
I think you're right, but they overlap too, so hard to give a completely clear picture. I.e., every Apache person I've met also uses GitHub. Obviously the converse is not true.

nknight 15y ago

Much of this is pretty unsurprising, especially for people like me who watched the attempted transition of OOo to Apache. The way the article puts it is that the ASF is trying to solve problems that don't exist anymore, which is true to an extent, but the deeper problem is that the ASF has a particular view of how open source development and project management work, and attempts to impose that view on far too diverse a community, even as it tries to absorb more and more communities. The ASF is simultaneously trying to be "big tent" and unified, and the balance is all out of whack. It's easy to draw parallels to recent political problems in the US and EU. In all three cases, there's going to have to be some transformations, probably in both society/community and structure, to come back to a place where the institution contributes to the greater good, instead of being a source of unending tension and meta-arguments.

jzawodn 15y ago
Very well said. I was going to write something like that but don't have to. Yay for up votes!

latchkey 15y ago

Well written. I wrote a similar article recently as well... http://lookfirst.com/2011/11/contributing-to-open-source.html http://lookfirst.com/2011/11/contributing-to-open-source.htm...

jowiar 15y ago

I second this thoroughly. I was was almost driven to start blogging last week by ASF's poor job of maintaining its projects. There was a small bug in Solr. I was not the first to find this bug, and someone had not only reported the bug, but filed a patch on the bug tracker a year and a half ago. The patch was never merged in, nobody provided any feedback as to why the patch wasn't merged in. One huge plus with Github is that if the official steward of a project would like to hand it off to someone else, or is failing to maintain it, it is trivial for someone else to take over the project.

jaaron 15y ago
Then fork it. It's open source for a reason. Or raise a fuss on the mailing list. That's how open source works.
- jerf 15y ago
  While forking is a solution, it hardly precludes discussion of other less drastic potential solutions to the problem at hand. It does nicely bound the maximum negative impact that Apache social problems can cause, though.
  - jaaron 15y ago
    If you're using git, as the OP suggets, then you're going to fork it just to work on it. Forking isn't dramatic. Code is open source for a reason. If you have a critical bug in code you need running in your infrastructure, take ownership of it and fork it. Then do the dirty work to get the patch pushed back up stream. If the maintainers really aren't doing what they volunteered to do, then volunteer yourself and get it done.
    - jerf 15y ago
      Sorry, explain to me clearly why that is the One Answer and we must stop discussing alternatives? You're answering a question I'm not even remotely asking.

ww520 15y ago

I'm curious how does it harm anything? Did it kill any puppies? Maybe it's inefficient but harmful? Also remember GitHub is a for profit company. Its allowance for Open Source hosting is a marketing tactic. Anytime they feel the marketing value is not there, they will shut it down. Not that I'm against GitHub. It's a great company for itself. But comparing the Apache to GitHub is like comparing apple and orange.

Legion 15y ago
Don't get caught up on GitHub specifically. GitHub is just the most popular example of how projects don't need Apache to host them anymore, yet Apache still expects to do so, and worse, expects to assert a lot of restriction over it. The point is, that is at odds with what the community wants and needs. As the author pointed out, ten years ago, rolling your own SCM hosting was a big pain. Now, it's not, partly because of GitHub and Bitbucket and others, but also because rolling your own isn't as hard either. Anyone with minimal server admin experience and knowledge of Git can run their own Git server on a VPS with something like Gitolite. I know because I succeeded in doing so myself, and I'm neither a pro server admin, nor did I have any Git experience at the time I did the initial setup of Gitolite. Prior to that, I had set up a Mercurial server with no prior Mercurial experience either. It's pretty easy now. So, yeah, GitHub is there, but GitHub could disappear tomorrow and the community still wouldn't need to turn to Apache for project hosting. In that respect, they're still solving a problem nobody has anymore, and that was the point the author was making.
- hrabago 15y ago
  GitHub isn't a new version of Apache - GitHub is a new version of SourceForge. I don't think that even 10 years ago anyone with a line (or 100 lines) of code can set up their own Apache projects. 10 years ago if you needed free SCM, you'd use SourceForge, not Apache. I don't think it would've been that big of a pain even then. GitHub projects don't necessarily come with its own community with diverse contributors, whereas Apache projects require it.
- nl 15y ago
  In that respect, they're still solving a problem nobody has anymore, and that was the point the author was making. That wasn't the author's point at all.

jaaron 15y ago

Ok, there's a lot to cover here. First off, the Apache Software Foundation isn't trying to absorb anyone or anything. Projects and people come to the ASF. It's a specific policy of the Foundation to NOT solicit projects. If someone says they're representing Apache and soliciting projects, they're wrong. Secondly, Apache is very opinionated about how projects should be run. This comes from years of experience as not only a successful project, but as a successful non-profit organization overseeing dozens of projects. If a community doesn't like the ASF's style or rules (such as no dictators, benevolent or otherwise), they don't need to be there. No one wants to keep projects hostage. Part of the point of the Incubator is to get this figured out earlier than later. Thirdly, about git and subversion. First off, there's increasing support for git at Apache (see http://git.apache.org/ http://git.apache.org/) but there are some serious drawbacks for use of git. Consider this: subversion was practically made for Apache in the way Linus made git for Linux. With that in mind, subversion isn't going anywhere at the ASF. Some of the rational is just plain stubbornness, but some of it goes straight to the core values of the Foundation. Apache has become, for better or worse, the place where lots of projects go when they grow up. Growing up is hard to do. It's not fun. You have to do things like get a job, pay taxes, etc. When a project grows up, people start caring about who contributed what, under which license and making sure every line of code is legit. A lot of engineers don't care about this, but businesses and their lawyers do. A lot of the Apache Foundation "bureaucracy" is to handle this oversight and paperwork. Git is an impressive tool and github is awesome for what it is, but it's not a non-profit foundation and it won't replace one. Confusing the Apache Software Foundation for your coding sandbox only suggests you don't understand the true purpose of either.

visural 15y ago
While correct, it would benefit Apache in the long run to foster better communities and tools for their projects to grow. While they provide value in terms of the protection/process/management it's all for nothing if the developers can't collaborate effectively. GitHub has changed many developer's UX of opensource development to the point that they don't want to do it "the old way". Apache should be looking to grow in this direction to keep their developers and projects engaged.

VladRussian 15y ago

push vs. pull. Basically the tools and overall environment grew up to have pull as a feasible model.

apike 15y ago

The ASF is the first home that comes to mind when an successful open-source project needs independent stewardship. Often when a company wants to "spin off" an open source project, they turn to Apache. What alternative organizations fill this need in a more lightweight fashion? Most other umbrella open source organizations I know of focus on copyleft and other issues that can be hostile to commercial interests.

sanxiyn 15y ago
Software Freedom Conservancy. http://sfconservancy.org/ http://sfconservancy.org/ Conservancy doesn't care about licenses as long as they are free. For example, jQuery (MIT license) is part of Conservancy. Conservancy members include prominent projects such as Boost, BusyBox, Darcs, Git, Inkscape, jQuery, Mercurial, PyPy, Samba, Selenium, Squeak, uClibc, Wine.
- ternaryoperator 15y ago
  "Conservancy doesn't care about licenses as long as they are free." They do care about licenses and license terms are part of the requirements for application. The project license must be either free (per FSF) or open (per OSI). Docs must be made available under Creative Commons licenses. And the project must be completely non-profit. (All these requirements must be met.) That being said, you're right, it's a good home without the politics discussed here.
- apike 15y ago
  Very cool. It seems like they are the type of organization Mikeal is encouraging ASF to become: legal and administrative support for open source projects, and other services if the project's leaders wish. Quite hands-off. Comparing the lists of projects, I'm surprised to find I use more SFC software than ASF software.

inopinatus 15y ago

This essay put me in mind of The Cathedral and the Bazaar. It is a neat demonstration of how tools and processes are inextricably linked. Who knew, though, that the ASF would be cast as the new priests in the cathedral? I suppose it took a whole new level of social development, enabled by tools, to cast them in that role.

rgardler 15y ago
Actually I think it is either: a) a complete misunderstanding of how and why the ASF operates the way it does or b) a desire for sensationalist blog pieces with almost no factual content The ASF is working with Git, it has been for years. It doesn't yet provide a canonical repository from which to make releases. This is due to a number of non-trivial technical issues introduced by the processes adopted by Apache projects. The Apache infrastructure team believe that they have now solved those issues and are testing them in CouchDB. Assuming the CouchDB experiment is a success the ASF will be rolling out Git as the canonical repository to all projects that want it. Once the ASF has mapped the tools to the processes we can all move on and stop wasting our time with this spurious argument. Disclaimer: Unlike the author of this blog I do have access to all the discussions about Git in the ASF and I am one of the mentors of PhoneGap, a project mentioned in the article.

polychrome 15y ago

So you're saying a system that has had repeated successes is harmful. I really think you make a good point here about the need to remain open to change. So talk about that. Obviously github has some very positive impact. How can Apache adapt to that? You're not really talking about the tools here, you're talking about community. I see a potential solution here being that Apache has different rules for projects in different stages. Do you think that would solve the issues? Remember, you're view of anarchy on GitHub will only last so long. Rules and order come out of anarchy for a reason and like all things GitHub will become the exact same stale community you're complaining of now in 10 years.

linuxhansl 15y ago

I am not entirely sure what the article is trying to get at. Politics and law in open source are real and needed, especially in the face of software patents. Many contributors develop open source code as part of their paid work, as such it is quite important to establish the legal framework to allow contributing the companies IP to an open source project (which includes necessary patent grants). Comitters need to submit an Individual Contributor License Agreement stating that they have the legal right to contribute the code they're contributing. If worked on contributions as employee the company also typically needs to submit a corporate contributor license agreement. Like it or not (and I personally do very much not like it), you cannot just upload some code somewhere these days. As such the even FSF is an extremely important organization. Much frowned upon and usually not understood. Open source licenses would not work without Copyright Law, most developers don't know or understand that. And the whole thing of svn vs git. Personally I don't get it. I use svn when it makes sense and I use git when that makes sense, just like I use Java/Ruby/C++/Python/Javascript/Closure/Scala/whatever or GoogleDocs/PDF/OpenOffice/MS Word(yes) when needed. The main feature I need from VCS are atomic commits. So CVS is out for me for that reason. Sure git's nice and all, but I spend 99.9% of my time writing or thinking about code and software architecture, not tinkering with my VCS, so as long as it works, I don't care. Easy forking and branching is nice too. In the end, though, just as with Linux there is/are some de-facto master branch(es) somewhere from which "releases" are cut. Currently with apache it's more convenient to use svn, so I do that. I don't get the religious opposition against one version control system vs another. Disclaimer: Apache committer here.

bad_user 15y ago
Ever since I started using Git, I would never go back to SVN, because branching and merging is now an essential part of my workflow, even on small project on which I work by myself with no other contributers. just as with Linux there is/are some de-facto master branch(es) Well yeah, but I don't get how that's an argument for SVN. The thing I like about Git is that branching is now really, really cheap. You can now keep track of dozens of local branches with experiments that you don't have to push to master. You can now share your experiments with a colleague and push to master whenever something is actually ready. You can now also ban commits to the main repository that haven't been code-reviewed (something which is a PITA with SVN). And so on and so forth. I don't get the religious opposition against one version control system vs another. Even though I prefer Git, neither do I, especially since you can just use the Git-SVN bridge :) I've used it for more than a year, it does has some quirks, but it works fine. Also, the Apache Foundation does its job and does it well. There's room for both anarchy and bureaucracy and both are needed.

ck2 15y ago

I wish at least one open source replacement adopted .htaccess (and httpd.conf) compatibility. Litespeed is the only product in existence which has made switching over from a complex Apache install a one hour affair, but it's free version is limited to 5 hosts and the commercial version can only be justified in a profitable environment. The performance difference is breathtaking however.

aristedes 15y ago

Where to start with this blog post? It appears that the author has seen a couple of private emails and thinks he knows all about the internal workings of the Apache Foundation. He is wrong on so many counts. His entire dislike of the Apache Foundation appears to be predicated on the fact that the organisation did not force every project to move to this blogger's favourite version control tool. Making a change as large as this requires many different things, but in particular: 1. Community change. How committers interact with each other when there are lots of forks is quite different to the current situation. That suits some projects and not others. Not every project at Apache will benefit. Some will. All who change will need to think long and hard about release processes, merging strategies and much more. Git encourages the idea that every commit or fork is completely equal to every other fork or commit. The Apache Foundation is built on the concept of meritocracy: commit rights are given in response to demonstrated skill. This is not an intractible problem with git, but new challenges need to be solved. 2. Legal change. Right now there is a simple process for signing off intellectual property for contributions which were merged from external contributors (who have not signed a release). That changes with git and becomes more complex. There are solutions, but they require careful planning. 3. Infrastructure. Hosting a large git repository with the level of downtime acceptable to Apache isn't something you do quickly. That needs planning and maintenance. 4. Toolsets. Lots of things in Apache are tied into subversion. From mailing list commit hooks to build servers and much more. Changing those things takes work. 5. Splitting the community. Right now the entire organisation's intellectual property is held in a single repository. Everyone knows where everything is to be found. Changing this simplicity requires a very good reason. So what do we have now? A blogger who (it appears) doesn't actually contribute code to any Apache project. Telling other people how to run their organisation (which is wildly successful). And that they should change to this blogger's favourite new tool (they should have done it in 2008!) or face irrelevance. If Apache moved every project to github tomorrow would that satisfy this blogger? More importantly, would that have caused this guy to commit high quality code toward one of the Apache projects? Or is he just blowing a lot of hot air about something he knows little? And what brought on this great complaint? That the Apache Foundation is currently underway with trials for one project to see how git would succeed for their workflow. And to then evaluate its suitability for other projects across Apache. Apache is not Github. That is, Apache is much more than a website, a couple of tools and a repository of code of random quality. Disclaimer: I am an Apache member, but not speaking on behalf of Apache

sktrdie 15y ago

1) Apache Software Foundation and GitHub are two totally different things. Who cares about their internal preferences and bureaucracies. They're both producing outstanding open-source projects which are used by hundreds of thousands of companies and people. Open-source (and the world) has only gained positive things out of these communities. 2) If you're suggesting that ASF needs to change its bureaucracy, I disagree. Frankly, I feel the bureaucracy has worked, given the success the foundation projects have had. 3) I'm not sure what other points your post brings, but if you're simply just saying that ASF needs to keep itself up-to-date with new tech (dunno git?) then this is also a totally absurd argument since the tech being used in Apache is totally amazing and new. I feel like your outcry is referred to general institutions... you should probably refer to governments and other political entities instead of bashing on a foundation that has given the world amazing products.

abrahamsen 15y ago

Free project infrastructure wasn't hard to setup five years ago. It hasn't really been a problem since 1999 when SourceForge opened. Before that, the SunSites did a nice job, and before that you basically had to know a friendly university sysadm (which wasn't _that_ hard to find). I'm not sure what people think they gain by going under the Apache umbrella, but it must be something since they bother. There are no lack of alternatives.

luriel 15y ago

This might be slightly offtopic, but might be a symptom of the "institutional"/"organizational" issues addressed in the article: I always thought Apache 2 and Subversion were two of the best examples of second-system effect. I mentioned this once to one of the core Apache (and Svn) developers years ago, and not only was he blissfully unaware of the effect, he indicated that he had helped build incredibly successful pieces of software (ie., Apache 1.x) and didn't need any advice from from Fred Brooks or anyone on how to do it. Both Apache 2 and svn have been extremely successful projects, but both were late, didn't really match expectations or even the success of their predecessors, and are slowly being outcompeted by much smaller and usually more efficient projects (eg., nginx, lighttpd, git, hg) that are developed much more quickly by much smaller teams.

delinka 15y ago

Or perhaps "Apache [Foundation] considered harmful" since in common usage "Apache" tends to mean "The Apache web server."

rmc 15y ago
Agreed. The title should be edited.

suprgeek 15y ago

Some people at the ASF think that Git may have some issues so are not wholeheartedly endorsing it for any and all projects. Somehow this gets translated into "Apache considered harmful". Why is this even on the front page?

funkah 15y ago
I would be seriously pissed if I donated my time and effort towards something like ASF, trying to help the open source world, only to be "considered harmful". WTF.

wyck 15y ago

..why isn't project x on git /rant tl;dr Github has a great social interface, it is a tool, there are others and there will be more. Yes it is an evolution towards DVC + great social interfaces, older industry mammoths will be slower to adopt then newer more agile projects. ps. I just switched from git to mercurial and it's a breath of fresh air.

nirvdrum 15y ago

Just to get two things out of the way: I'm an ASF member (albeit not very active lately) and a huge fan of git with or without GitHub. I'm one of the many people advocating for git internally at the ASF. I have been met with opposition in the past, but a lot of it has been around who's going to maintain the infrastructure, given it's a volunteer system. Let's just take it as axiomatic that the ASF is going to self-host its code. So it's at least a fairly pragmatic argument. And I think we finally have a solution. My real issue is with the bouncing back-and-forth the author does in his post around the notion of IP. It's a shitty topic that most devs don't want to be bothered with, but alas, it's quite important in the real world. And GitHub is mostly a landmine field when it comes to this. I don't think it's a failing of GitHub itself, but most projects just don't have licenses attached to them. Unlike with SourceForge, there's no requirement to have an OSS license on public projects. Then many that do fail to meet the copyright header requirements for the license. Or you could have a public project with a restrictive license [1]. Being public doesn't mean you get to do whatever you want with the code. This is dangerous and bad for OSS. Apache gives you that protection. There's never any question about it. That's the primary reason projects go through the incubator -- to make sure the IP is all in order. It's an annoying, bureaucratic, but necessary process in a litigious society. But because of the care and protections Apache provides in this regard, I think they've done more to get OSS adopted in traditionally closed companies than just about anyone else. [1] I came across Tom Preston-Werner's repo for his site. He's one of the GitHub founders. It's a public repo with a license that restricts usage of certain portions of the project (generally his content): https://github.com/mojombo/mojombo.github.com https://github.com/mojombo/mojombo.github.com

MattRogish 15y ago
This is a good point. Anyone who starts an open-source project should, from day one, have a vetted Contributor License Agreement and ONLY accept pull requests that include signed CLAs (or from a person who has one on file).
- nirvdrum 15y ago
  It'd be kinda nifty if GitHub had this built-in. I personally don't require CLAs for every project because it can be onerous. But at the least I try to pick a license that wouldn't allow for submarine patent claims.

caniszczyk 15y ago

Well written... I wrote a response last night that dives into some numbers and my experiences with helping the Eclipse Foundation move to Git and Gerrit... http://aniszczyk.org/2011/11/23/apache-and-politics-over-code/ http://aniszczyk.org/2011/11/23/apache-and-politics-over-cod... Apache is pretty much the last major open source community to not move to some form of distributed version control. It's either politics (they host the Subversion project there) or negligence in my opinion.

jmathes 15y ago

"Considered Harmful" considered pretentious

pyrhho 15y ago

As an aside, the Clay Shirky quote ("Institutions will try to preserve the problem to which they are the solution.") was new to me, but puts the RIAA/MPAA pretty much perfectly into perspective. Not really related to the article, but it clicked as I was reading.