6 ms·

I think they could have worded this better: "GitHub offers the largest free storage quota among the big SCM hosters, and we came to the conclusion that we didn’

by seiji 17y ago

I think they could have worded this better: "GitHub offers the largest free storage quota among the big SCM hosters, and we came to the conclusion that we didn’t want to subsidize that quota for non-Ruby developers." It sounds odd Engine Yard hosts an entire for-profit company for free just in exchange for some complimentary accounts. My imagination tells me GitHub is tired of being slow and Engine Yard couldn't do anything more to help them. Instead of the news being "GitHub Leaves Engine Yard Because It's Too Slow" the news is "Engine Yard Kicks GitHub To The Curb Because GitHub has Non-Ruby Repos." I don't see either headline being positive towards Engine Yard.

jeremymcanally 17y ago
> It sounds odd Engine Yard hosts an entire for-profit company for free just in exchange for some complimentary accounts. Well, sort of. It's really a value trade-off. EY offers its customers something that would cost them $50 a month, so it's costing Github (customers) * $50 to offer that. They exchange the lost value for added value in hosting (i.e., they're "losing" a few thousand a month in free accounts for our customers, so we'll offer them free hosting to continue to offer that). It's simple barter.
- jhancock 17y ago
  Very excited to see GitHub continuing to grow and EngineYard and GitHub handling their barter deals so cordially. Both companies are class acts!!!
jcapote 17y ago
That assumes that Github is slow because of their hosting (Engineyard) and not because of how it's written. I dunno, it could go either way for me: "Engineyard drops Github for being too costly" vs "Github needs better hosting, so we're dropping Engineyard"
- jhancock 17y ago
  Its probably more about costs for both sides. I doubt that EY is _slow_ without regard to cost. Just that instead of rewritting GitHub's web app to deal with inherent performance issues, it makes more sense to throw more hardware at the problem. There does come a point where owning more of your server stack makes more financial sense.
  - fnid 17y ago
    It is going to be interesting to see how the performance of Github and their freemium offering changes when they start paying for infrastructure somewhere else. Unless they can smooth talk another host into giving them free infrastructure.
    - jhancock 17y ago
      Based on replies from my support queries about performance, they are aware that paying customers expect better/more-consistent performance than what we've had of late.
      - fnid 17y ago
        I see, so even if they did pay, it still wouldn't be fast enough. So free even has its costs, in terms of dissatisfied customers.
conoryoung 17y ago
It's like when your boss let's you "resign" so you don't have to tell a future employer you were fired. EY was fired.
- fnid 17y ago
  How can github fire EY when EY was giving them free stuff. Why doesn't github just pay EY for the services they are using? This is another failure of Freemium. GH is only able to provide such large repos because they get free infrastructure. I can't really blame EY for bailing. The partnership is no longer equitable. GH is getting something for nothing. I shouldn't say nothing, it's something, but it's not enough.
  - scott_s 17y ago
    If your service doesn't meet my needs, it doesn't matter how cheap it is - including $0. According to my understanding, EngineYard doesn't meet GitHub's needs anymore.
    - mtalantikite 17y ago
      I don't think that's totally fair to say, we've put a considerable amount of effort into GitHub over here at EY, across all of our departments. GitHub offers a great product and it's really amazing to see them grow so fast, but that success also introduces a whole lot of new problems to the situation, many of which can be quite costly to solve. It's just a conclusion that makes the most sense for both sides of the arrangement.
      - marclove 17y ago
        EY is worth every penny they charge because they provide an incredible quality and commitment of service. They are without a doubt the best hosting company I've ever had the pleasure of working with. Not only are they quick with assistance 24/7, but they all know what they're doing. This really shows poorly on the guys at Github. It sounds like EY went out of their way to accommodate the growth rate (I have yet to hear the Github guys dispute that) and just because EY was asking to be paid for all the hard work, Github was unwilling to, for once, be paid...Github ditched them.
fizx 17y ago
Well, EY could make Github fast again by providing more resources. But then, someone would have to pay for the additional resources. EY doesn't think it's worth it, and Github can get the additional resources elsewhere for cheaper (EY is expensive, because you pay for premium support). It seems like it's more about the fit between the companies, than any particular performance issues.
noodle 17y ago
> It sounds odd Engine Yard hosts an entire for-profit company for free just in exchange for some complimentary accounts. why? its fairly high quality, targeted advertising with a built-in demonstration.
anonysock 17y ago
Ok, as far as I can tell, the real story is that Engine Yard relies on a GFS/SAN setup that doesn't scale in the unique way that Github needs. If you think about it, Github is one of the few sites that actually directly uses the filesystem heavily. Everyone else hits scaling issues on the DB first. The sad thing of all of this is it's not really a matter of scaling, and it never has been. Our bottleneck has always been the file system. GFS just... sucks. I'm sorry, but I have to say it. Case in point, your graph. The first rebuild I ran timed out because of GFS. The second one ran fine, took maybe a minute to process, if that. GFS impacts everything... gem build failures due to cloning... GFS. Network graphs taking long time to build... GFS. Caching jobs not completing... GFS. I think you see where I'm going here. There's no plans to deploy the new code to the live servers, and I think the reason is that we're afraid it'll make GFS performance worse, not better. But on the new servers where we don't have to fight GFS, it's amazing.
- ezmobius 17y ago
  Funny thing is that we told github that gfs would not scale for them over a year ago, we also outlined how to move to a shared nothing chunk server architecture. They didn't take our advice so it's mostly their own architecture decisions that were holding them back with regards to gfs. Anyway there seems to be plenty of airchair quarterback on this one. The real story is that we can't afford to host them for free anymore.
  - trevorturk 17y ago
    FWIW - thanks for hosting them for so long. GH is a wonderful service, and I'm sure EY contributed greatly to it's success.
  - blasdel 17y ago
    Is it so hard to imagine that they weren't jumping at the chance to write their own custom non-filesystem storage backend?
durin42 17y ago
It's also inaccurate: GitHub doesn't offer the largest free storage quota. That's still Google Code, which offers 1024 mb/project, rather than 300 mb/user.
- antonovka 17y ago
  Google is also happy to bump up your quotas if you have a genuine need.
  - tekkub 17y ago
    As are we, disk space should never be an issue with us unless you're doing something very wrong (hosting videos, etc).
    - jrockway 17y ago
      The GNU Emacs repository uses almost all of my (paid) storage space. I am not doing anything wrong, either, Emacs just has a lot of history and a lot of code.
      - scorpion032 17y ago
        Emacs officially still uses CVS, right?
        jrockway 17y ago
        bzr. But throwing everything on github is a lot easier than using bzr, so I do that. There is no bzrhub, and I don't have commit-access to GNU Emacs.

by seiji 17y ago

jeremymcanally 17y ago

> It sounds odd Engine Yard hosts an entire for-profit company for free just in exchange for some complimentary accounts. Well, sort of. It's really a value trade-off. EY offers its customers something that would cost them $50 a month, so it's costing Github (customers) * $50 to offer that. They exchange the lost value for added value in hosting (i.e., they're "losing" a few thousand a month in free accounts for our customers, so we'll offer them free hosting to continue to offer that). It's simple barter.

jhancock 17y ago
Very excited to see GitHub continuing to grow and EngineYard and GitHub handling their barter deals so cordially. Both companies are class acts!!!

jcapote 17y ago

That assumes that Github is slow because of their hosting (Engineyard) and not because of how it's written. I dunno, it could go either way for me: "Engineyard drops Github for being too costly" vs "Github needs better hosting, so we're dropping Engineyard"

jhancock 17y ago
Its probably more about costs for both sides. I doubt that EY is _slow_ without regard to cost. Just that instead of rewritting GitHub's web app to deal with inherent performance issues, it makes more sense to throw more hardware at the problem. There does come a point where owning more of your server stack makes more financial sense.
- fnid 17y ago
  It is going to be interesting to see how the performance of Github and their freemium offering changes when they start paying for infrastructure somewhere else. Unless they can smooth talk another host into giving them free infrastructure.
  - jhancock 17y ago
    Based on replies from my support queries about performance, they are aware that paying customers expect better/more-consistent performance than what we've had of late.
    - fnid 17y ago
      I see, so even if they did pay, it still wouldn't be fast enough. So free even has its costs, in terms of dissatisfied customers.

conoryoung 17y ago

It's like when your boss let's you "resign" so you don't have to tell a future employer you were fired. EY was fired.

fnid 17y ago
How can github fire EY when EY was giving them free stuff. Why doesn't github just pay EY for the services they are using? This is another failure of Freemium. GH is only able to provide such large repos because they get free infrastructure. I can't really blame EY for bailing. The partnership is no longer equitable. GH is getting something for nothing. I shouldn't say nothing, it's something, but it's not enough.
- scott_s 17y ago
  If your service doesn't meet my needs, it doesn't matter how cheap it is - including $0. According to my understanding, EngineYard doesn't meet GitHub's needs anymore.
  - mtalantikite 17y ago
    I don't think that's totally fair to say, we've put a considerable amount of effort into GitHub over here at EY, across all of our departments. GitHub offers a great product and it's really amazing to see them grow so fast, but that success also introduces a whole lot of new problems to the situation, many of which can be quite costly to solve. It's just a conclusion that makes the most sense for both sides of the arrangement.
    - marclove 17y ago
      EY is worth every penny they charge because they provide an incredible quality and commitment of service. They are without a doubt the best hosting company I've ever had the pleasure of working with. Not only are they quick with assistance 24/7, but they all know what they're doing. This really shows poorly on the guys at Github. It sounds like EY went out of their way to accommodate the growth rate (I have yet to hear the Github guys dispute that) and just because EY was asking to be paid for all the hard work, Github was unwilling to, for once, be paid...Github ditched them.

fizx 17y ago

Well, EY could make Github fast again by providing more resources. But then, someone would have to pay for the additional resources. EY doesn't think it's worth it, and Github can get the additional resources elsewhere for cheaper (EY is expensive, because you pay for premium support). It seems like it's more about the fit between the companies, than any particular performance issues.

noodle 17y ago

> It sounds odd Engine Yard hosts an entire for-profit company for free just in exchange for some complimentary accounts. why? its fairly high quality, targeted advertising with a built-in demonstration.

anonysock 17y ago

Ok, as far as I can tell, the real story is that Engine Yard relies on a GFS/SAN setup that doesn't scale in the unique way that Github needs. If you think about it, Github is one of the few sites that actually directly uses the filesystem heavily. Everyone else hits scaling issues on the DB first. The sad thing of all of this is it's not really a matter of scaling, and it never has been. Our bottleneck has always been the file system. GFS just... sucks. I'm sorry, but I have to say it. Case in point, your graph. The first rebuild I ran timed out because of GFS. The second one ran fine, took maybe a minute to process, if that. GFS impacts everything... gem build failures due to cloning... GFS. Network graphs taking long time to build... GFS. Caching jobs not completing... GFS. I think you see where I'm going here. There's no plans to deploy the new code to the live servers, and I think the reason is that we're afraid it'll make GFS performance worse, not better. But on the new servers where we don't have to fight GFS, it's amazing.

ezmobius 17y ago
Funny thing is that we told github that gfs would not scale for them over a year ago, we also outlined how to move to a shared nothing chunk server architecture. They didn't take our advice so it's mostly their own architecture decisions that were holding them back with regards to gfs. Anyway there seems to be plenty of airchair quarterback on this one. The real story is that we can't afford to host them for free anymore.
- trevorturk 17y ago
  FWIW - thanks for hosting them for so long. GH is a wonderful service, and I'm sure EY contributed greatly to it's success.
- blasdel 17y ago
  Is it so hard to imagine that they weren't jumping at the chance to write their own custom non-filesystem storage backend?

durin42 17y ago

It's also inaccurate: GitHub doesn't offer the largest free storage quota. That's still Google Code, which offers 1024 mb/project, rather than 300 mb/user.

antonovka 17y ago
Google is also happy to bump up your quotas if you have a genuine need.
- tekkub 17y ago
  As are we, disk space should never be an issue with us unless you're doing something very wrong (hosting videos, etc).
  - jrockway 17y ago
    The GNU Emacs repository uses almost all of my (paid) storage space. I am not doing anything wrong, either, Emacs just has a lot of history and a lot of code.
    - scorpion032 17y ago
      Emacs officially still uses CVS, right?
      - jrockway 17y ago
        bzr. But throwing everything on github is a lot easier than using bzr, so I do that. There is no bzrhub, and I don't have commit-access to GNU Emacs.