12 ms·

I am increasingly bothered by the "not invented here" syndrome where instead of taking existing projects and enhancing them, in true open source fashion, people

by jimjag 10y ago

I am increasingly bothered by the "not invented here" syndrome where instead of taking existing projects and enhancing them, in true open source fashion, people instead re-create from scratch. It is then justified that their creation is needed because "no one else has these kinds of problems" but then they open source them as if lots of other people could benefit from it. Why open source something if it has an expected user base of 1? Again, I am not surprised by this. They whole push of Github is not to create a community which works together on a single project in a collaborative, consensus based method, but rather lots of people doing their own thing and only occasionally sharing code. It is no wonder that they follow this meme internally.

bogomipz 10y ago
I'm not sure I agree, they mention HAProxy and Foo over UDP so they are leveraging existing open source technologies. Custom additions to suit ones particular use case isn't necessarily the same as NIH syndrome.
- jimjag 10y ago
  They mention HAProxy, but it doesn't look like it's based on it.
  - logicalstack 10y ago
    Joe from GitHub here, we'll talk about it later posts but GLB is based on a number of open source projects including, haproxy, iptables, FoU and pf_ring. Many existing open source solutions are optimized for short lived HTTP requests and don't address the long running connection issue (like a large git clone). We wanted something better for our use case.
    - jimjag 10y ago
      That is good to know. Thx.
    - halifaxbeard 10y ago
      I'm currently working with GitHub Support on dealing with zip downloads of a 5GB repo failing after 2-3 minutes, with curl error "transfer closed with outstanding read data remaining". Sure about the long running connection issue being solved? :-)
alexandros 10y ago
here's an alternative interpretation: Due to their unique requirements, they were forced to investigate alternative approaches to the problem, and the approach they went with, they believe, is useful to a lot more people than just themselves. This is both plausible and matches the announcement, so why take a bad faith approach to someone releasing some code to the open source? Would you prefer if they kept it closed? And since we're talking about github, haven't they already launched a highly successful pair of projects in atom/electron, in areas where both had competition? why start with negativity before we see what they come out with?
- saurik 10y ago
  Huh? Atom/Electron is another great example of GitHub duplicating a ton of existing projects (whether dependencies such as CEF and node-webkit or high-level solutions such as ACE) without seemingly having any interest at all in joining those existing projects. Just because someone is successful at doing this does not make what they are doing any more reasonable: if anything it should just put them in a similar place in your mind to the Microsoft of the 90s which many people here would denigrate. GitHub's model of "open source"--the one which it is, devastatingly, teaching to an entire generation of developers--is only about code being available as opposed to being about community and collaborative design. Asking if one would prefer an alternative where the code is simply kept closed source ignores the premise of the compliant: that the code being an advertised separate project undermines the premise that working with an existing project to solve a problem tons of those users almost certainly also have. :/
  - alexandros 10y ago
    Open source does not entail any responsibility to work or not work together with any pre-existing project. What you describe is more cathedral than bazaar. There are thousands of possible reasons, from architectural choices to personalities, that someone may have not chosen to work with an existing project. I will not fault them for giving me a superior result, for free (in both senses). To compare this with Microsoft in the nineties is sheer madness.
  - cstejerean 10y ago
brazzledazzle 10y ago
It may not based on haproxy but they definitely used it prior to switching. If we're being generous in our interpretation that would indicate that they found that it didn't work for them. I'm not saying your overall point isn't valid but I wouldn't be so quick to judge.
merb 10y ago
basically a load balancer must often do special stuff and most companies building custom software around haproxy, nginx whatever to support their needs.
gjvc 10y ago
by that rationale, Apache httpd would have stayed Apache httpd, and we wouldn't have gotten nginx.
- paulddraper 10y ago
  Shudder.
  - gjvc 10y ago
    imagine a world full of sendmail, not postfix
  - deleted 10y ago
    [deleted]
dismantlethesun 10y ago
For me, NIH is about the idea of already having deep knowledge of the problem domain so a solution is relatively straight forwards. Sure it'll take X time, and there might be some hiccups, but its all about effort and not having to learn anything new. Joining a pre-existing project is more reasonable when you just can't replicate the basis of the project without learning a lot of new things. That's the reason why there's a plethora of compile to JS languages, but only a few actual javascript virtual machines.
ckdarby 10y ago
Have you ever contributed to HaProxy? Have you ever tried committing massive alterations to major open source projects? It isn't as simple as here's my massive rewrite, click the accept button and everything works out for the open source community. Let me be the first to say that the level of politics, circle jerking and knowing people is ridiculous.
- senko 10y ago
  "Contributing to open source is hard, let's write our own and open source it instead."
- polpo 10y ago
  Given the good reaction to an out-of-the-blue patch from me on the HAProxy mailing list, I'd imagine that contributing even major changes to HAProxy probably would go rather well. It's one of the best open source development communities I've experienced. Welcoming, but still highly focused on quality contributions. The quality and performance of HAProxy reflects this approach.
  - paulddraper 10y ago
    HAProxy is exceptionally good in this regard.
- JungleGymSam 10y ago
  Sounds like you're suggesting you can't use your own modified fork until your PR is accepted.
- zepolen 10y ago
  As someone who's contributed multiple patches to multiple open source projects, this is 100% truth. I have >3 month old pull requests to add tests which have never been looked at - whereas someone who knows the project maintainer will get a PR looked and merged the next day.
jcastro 10y ago
> in true open source fashion, people instead re-create from scratch. True open source fashion is also the freedom to work on whatever you want. "Just think about it, if they would have contributed to foo instead of working on bar, then foo would be twice as good!" keeps being thrown around every time someone announces something new around here.
serge2k 10y ago
> I am increasingly bothered by the "not invented here" syndrome I'm bothered by the increasing prevalence of "never invent here". FOSS is great, but if it's not meeting your needs then writing your own is perfectly valid.
- Retra 10y ago
  And often one of your needs is "have people on hand who know exactly how it works", and the easiest way to achieve that is to have those people build it from scratch. That's the only way to actually ensure it makes your needs priority #1.
eikenberry 10y ago
NIH syndrome is overrated. A small application doing specifically what you want is usually more robust, easier to maintain and easier extend as you need than a pre-existing application that includes most of what you need as a subset of its functionality.
jedberg 10y ago
> It is then justified that their creation is needed because "no one else has these kinds of problems" but then they open source them as if lots of other people could benefit from it. Why open source something if it has an expected user base of 1? Two reasons. 1) Recruiting. Check out our awesome code! Don't you want to work on this too? 2) Our unique problem today will be the problems of everyone in three years. This has borne out with the Netflix opes source. At the time it was a problem unique to Netflix -- now a bunch of people are using that software or derivatives.
im4w1l 10y ago
Why would you start a new company when you can join and enhance an existing one?
mobiuscog 10y ago
This is just ridiculous. Open Source is about just that - allowing anyone and everyone to have access to the source and (if free) making their own version. There never needs to be justification to write something from scratch, even if it's been done a million times before. You can appreciate open-source, you can wish that proprietary code was open source, but you never have the right to tell people what they should do, nor are you ever likely to be correct as to what they should do - you are not them.

by jimjag 10y ago

bogomipz 10y ago

I'm not sure I agree, they mention HAProxy and Foo over UDP so they are leveraging existing open source technologies. Custom additions to suit ones particular use case isn't necessarily the same as NIH syndrome.

jimjag 10y ago
They mention HAProxy, but it doesn't look like it's based on it.
- logicalstack 10y ago
  Joe from GitHub here, we'll talk about it later posts but GLB is based on a number of open source projects including, haproxy, iptables, FoU and pf_ring. Many existing open source solutions are optimized for short lived HTTP requests and don't address the long running connection issue (like a large git clone). We wanted something better for our use case.
  - jimjag 10y ago
    That is good to know. Thx.
  - halifaxbeard 10y ago
    I'm currently working with GitHub Support on dealing with zip downloads of a 5GB repo failing after 2-3 minutes, with curl error "transfer closed with outstanding read data remaining". Sure about the long running connection issue being solved? :-)

alexandros 10y ago

here's an alternative interpretation: Due to their unique requirements, they were forced to investigate alternative approaches to the problem, and the approach they went with, they believe, is useful to a lot more people than just themselves. This is both plausible and matches the announcement, so why take a bad faith approach to someone releasing some code to the open source? Would you prefer if they kept it closed? And since we're talking about github, haven't they already launched a highly successful pair of projects in atom/electron, in areas where both had competition? why start with negativity before we see what they come out with?

saurik 10y ago
Huh? Atom/Electron is another great example of GitHub duplicating a ton of existing projects (whether dependencies such as CEF and node-webkit or high-level solutions such as ACE) without seemingly having any interest at all in joining those existing projects. Just because someone is successful at doing this does not make what they are doing any more reasonable: if anything it should just put them in a similar place in your mind to the Microsoft of the 90s which many people here would denigrate. GitHub's model of "open source"--the one which it is, devastatingly, teaching to an entire generation of developers--is only about code being available as opposed to being about community and collaborative design. Asking if one would prefer an alternative where the code is simply kept closed source ignores the premise of the compliant: that the code being an advertised separate project undermines the premise that working with an existing project to solve a problem tons of those users almost certainly also have. :/
- alexandros 10y ago
  Open source does not entail any responsibility to work or not work together with any pre-existing project. What you describe is more cathedral than bazaar. There are thousands of possible reasons, from architectural choices to personalities, that someone may have not chosen to work with an existing project. I will not fault them for giving me a superior result, for free (in both senses). To compare this with Microsoft in the nineties is sheer madness.
- cstejerean 10y ago

brazzledazzle 10y ago

It may not based on haproxy but they definitely used it prior to switching. If we're being generous in our interpretation that would indicate that they found that it didn't work for them. I'm not saying your overall point isn't valid but I wouldn't be so quick to judge.

merb 10y ago

basically a load balancer must often do special stuff and most companies building custom software around haproxy, nginx whatever to support their needs.

gjvc 10y ago

by that rationale, Apache httpd would have stayed Apache httpd, and we wouldn't have gotten nginx.

paulddraper 10y ago
Shudder.
- gjvc 10y ago
  imagine a world full of sendmail, not postfix
- deleted 10y ago
  [deleted]

dismantlethesun 10y ago

For me, NIH is about the idea of already having deep knowledge of the problem domain so a solution is relatively straight forwards. Sure it'll take X time, and there might be some hiccups, but its all about effort and not having to learn anything new. Joining a pre-existing project is more reasonable when you just can't replicate the basis of the project without learning a lot of new things. That's the reason why there's a plethora of compile to JS languages, but only a few actual javascript virtual machines.

ckdarby 10y ago

Have you ever contributed to HaProxy? Have you ever tried committing massive alterations to major open source projects? It isn't as simple as here's my massive rewrite, click the accept button and everything works out for the open source community. Let me be the first to say that the level of politics, circle jerking and knowing people is ridiculous.

senko 10y ago
"Contributing to open source is hard, let's write our own and open source it instead."
polpo 10y ago
Given the good reaction to an out-of-the-blue patch from me on the HAProxy mailing list, I'd imagine that contributing even major changes to HAProxy probably would go rather well. It's one of the best open source development communities I've experienced. Welcoming, but still highly focused on quality contributions. The quality and performance of HAProxy reflects this approach.
- paulddraper 10y ago
  HAProxy is exceptionally good in this regard.
JungleGymSam 10y ago
Sounds like you're suggesting you can't use your own modified fork until your PR is accepted.
zepolen 10y ago
As someone who's contributed multiple patches to multiple open source projects, this is 100% truth. I have >3 month old pull requests to add tests which have never been looked at - whereas someone who knows the project maintainer will get a PR looked and merged the next day.

jcastro 10y ago

> in true open source fashion, people instead re-create from scratch. True open source fashion is also the freedom to work on whatever you want. "Just think about it, if they would have contributed to foo instead of working on bar, then foo would be twice as good!" keeps being thrown around every time someone announces something new around here.

serge2k 10y ago

> I am increasingly bothered by the "not invented here" syndrome I'm bothered by the increasing prevalence of "never invent here". FOSS is great, but if it's not meeting your needs then writing your own is perfectly valid.

Retra 10y ago
And often one of your needs is "have people on hand who know exactly how it works", and the easiest way to achieve that is to have those people build it from scratch. That's the only way to actually ensure it makes your needs priority #1.

eikenberry 10y ago

NIH syndrome is overrated. A small application doing specifically what you want is usually more robust, easier to maintain and easier extend as you need than a pre-existing application that includes most of what you need as a subset of its functionality.

jedberg 10y ago

> It is then justified that their creation is needed because "no one else has these kinds of problems" but then they open source them as if lots of other people could benefit from it. Why open source something if it has an expected user base of 1? Two reasons. 1) Recruiting. Check out our awesome code! Don't you want to work on this too? 2) Our unique problem today will be the problems of everyone in three years. This has borne out with the Netflix opes source. At the time it was a problem unique to Netflix -- now a bunch of people are using that software or derivatives.

im4w1l 10y ago

Why would you start a new company when you can join and enhance an existing one?

mobiuscog 10y ago

This is just ridiculous. Open Source is about just that - allowing anyone and everyone to have access to the source and (if free) making their own version. There never needs to be justification to write something from scratch, even if it's been done a million times before. You can appreciate open-source, you can wish that proprietary code was open source, but you never have the right to tell people what they should do, nor are you ever likely to be correct as to what they should do - you are not them.