5 ms·

GitHub's Architecture, before and after migration

80 points by quellhorst 17y ago

wmf 17y ago
Are we to believe that Rackspace plus outsourced system administration is still cheaper than Engine Yard? Impressive.
- cmelbye 17y ago
  Engine Yard hosting was free. I don't think it was about cutting costs necessarily, but rather it was for improving the performance of the site and opening opportunities for scaling rapidly in the future.
  - simonk 17y ago
    EY told them if they wanted to continue to start paying and they said it was a different by a huge amount.
    - tumult 17y ago
      It's not that simple. EY's filesystem stuff is designed for read-almost-always, as you would expect for most Rails sites. GitHub does a lot of writing. Growing GitHub's setup would have been a huge undertaking for EY (from what I've read of their posts) and you just can't do that stuff for free, or you run out of money. (This is just what I've gleaned from comments. They parted ways amicably. Don't stir up drama.)
    - simonk 17y ago
      Yes, but you can't say if they stayed at EY it would be free.
      - tjogin 17y ago
        It would also have been free to run Github off of an old server out of their garage. Just not very functional.
- nakajima 17y ago
  I'd argue no, at least until you have as good an idea of what your needs are as GitHub did.
- qhoxie 17y ago
  The move to Rackspace will bring about a new backend architecture and a lot more servers, leading to a much improved user experience for everyone. - TPW from Github I don't believe that cost was an issue for them with EY. It was about their system's needs regarding server resources.
- The_Lost_Hacker 17y ago
  Believe it. EY is fucking expensive because they've made people believe it should be.
datums 17y ago
I've setup similar solutions in the past. Nowadays I don't agree with having an entire stack basically sitting idle. I prefer to get those resource involved intelligently. Looking forward to more details and possible reasons for choosing the technology (DRDB, LB software, etc )
- mojombo 17y ago
  We run 12GB of memcache on each slave file server. Hardly idle!
  - datums 17y ago
    I was looking at http://www.anchor.com.au/blog/wp-content/uploads/2009/09/GitHubStorage_Small.png http://www.anchor.com.au/blog/wp-content/uploads/2009/09/Git... which shows file server B as a failover, traffic coming into the active LB. Does the active LB then have a pool which includes other file server B services ?
pilif 17y ago
Their migration seems to have gone quite smoothly. And the whole site feels much faster to me. Congratulations for that. And thanks for sharing this information about your architecture, this was a very interesting read.
colinprince 17y ago
Only 1TB+ of storage for 15+ physical servers?
murrayb 17y ago
I've used Anchor for web hosting (for myself & for clients) and I thoroughly recommend them. Not only is their service great they have technically competent staff. The norm in IT these days seems to be that you have to persevere/fight your way through x layers of support before you get someone who understands tech, at Anchor everytime the person who first spoke to me was able to understand and resolve the issue.

80 points by quellhorst 17y ago

wmf 17y ago

Are we to believe that Rackspace plus outsourced system administration is still cheaper than Engine Yard? Impressive.

cmelbye 17y ago
Engine Yard hosting was free. I don't think it was about cutting costs necessarily, but rather it was for improving the performance of the site and opening opportunities for scaling rapidly in the future.
- simonk 17y ago
  EY told them if they wanted to continue to start paying and they said it was a different by a huge amount.
  - tumult 17y ago
    It's not that simple. EY's filesystem stuff is designed for read-almost-always, as you would expect for most Rails sites. GitHub does a lot of writing. Growing GitHub's setup would have been a huge undertaking for EY (from what I've read of their posts) and you just can't do that stuff for free, or you run out of money. (This is just what I've gleaned from comments. They parted ways amicably. Don't stir up drama.)
  - simonk 17y ago
    Yes, but you can't say if they stayed at EY it would be free.
    - tjogin 17y ago
      It would also have been free to run Github off of an old server out of their garage. Just not very functional.
nakajima 17y ago
I'd argue no, at least until you have as good an idea of what your needs are as GitHub did.
qhoxie 17y ago
The move to Rackspace will bring about a new backend architecture and a lot more servers, leading to a much improved user experience for everyone. - TPW from Github I don't believe that cost was an issue for them with EY. It was about their system's needs regarding server resources.
The_Lost_Hacker 17y ago
Believe it. EY is fucking expensive because they've made people believe it should be.

datums 17y ago

I've setup similar solutions in the past. Nowadays I don't agree with having an entire stack basically sitting idle. I prefer to get those resource involved intelligently. Looking forward to more details and possible reasons for choosing the technology (DRDB, LB software, etc )

mojombo 17y ago
We run 12GB of memcache on each slave file server. Hardly idle!
- datums 17y ago
  I was looking at http://www.anchor.com.au/blog/wp-content/uploads/2009/09/GitHubStorage_Small.png http://www.anchor.com.au/blog/wp-content/uploads/2009/09/Git... which shows file server B as a failover, traffic coming into the active LB. Does the active LB then have a pool which includes other file server B services ?

pilif 17y ago

Their migration seems to have gone quite smoothly. And the whole site feels much faster to me. Congratulations for that. And thanks for sharing this information about your architecture, this was a very interesting read.

colinprince 17y ago

Only 1TB+ of storage for 15+ physical servers?

murrayb 17y ago

I've used Anchor for web hosting (for myself & for clients) and I thoroughly recommend them. Not only is their service great they have technically competent staff. The norm in IT these days seems to be that you have to persevere/fight your way through x layers of support before you get someone who understands tech, at Anchor everytime the person who first spoke to me was able to understand and resolve the issue.