3 ms·

> Until recently, we kept copies of repository data using off-the-shelf, disk-layer replication technologies—namely, RAID and DRBD. We organized our file server

> Until recently, we kept copies of repository data using off-the-shelf, disk-layer replication technologies—namely, RAID and DRBD. We organized our file servers in pairs. Each active file server had a dedicated, online spare connected by a cross-over cable. With all the ink spilled about creative distributed architectures, it's really humbling to see how far they grew with an architecture that simple.

viraptor 11y ago
I'm not surprised really. I never worked at GH scale, but anyway learned early that simple solutions just work. Want a db cluster? Why not active-passive M-M instead. Want a M-M setup? Why not dual server and block replication. Complicated things fail in complicated ways (looking at you, mysql ndb cluster), while simple solutions just work. They may be less efficient, but you'd better have a great use case for spending time on a new, fancy clustering solution - and even better idea how to handle it's state / monitoring / consistency.
- cbsmith 11y ago
  For the real use cases for the fancy clustering solution, the benefits can be huge... and complicated things built with failure in mind actually fail in quite simple ways. DGit is an example in itself. Sure it heavily leverages the complexity of git's versioning/branching/distributed vcs mechanism to get the job done, but compared to a SAN or RAID + DRBD, the failure scenarios are much more straightforward to deal with.
  - viraptor 11y ago
    I agree. As long as you have people who can build it tailored to the service and handling all the errors. GH can afford doing that almost from scratch. Or if you're sure that this is exactly the solution you need and are familiar with the failure scenarios. But from what I've seen in a few places, a lot of people jump to cluster solutions without either a real need or enough people to support it.
    - cbsmith 11y ago
      > But from what I've seen in a few places, a lot of people jump to cluster solutions without either a real need or enough people to support it. Totally.

viraptor 11y ago

I'm not surprised really. I never worked at GH scale, but anyway learned early that simple solutions just work. Want a db cluster? Why not active-passive M-M instead. Want a M-M setup? Why not dual server and block replication. Complicated things fail in complicated ways (looking at you, mysql ndb cluster), while simple solutions just work. They may be less efficient, but you'd better have a great use case for spending time on a new, fancy clustering solution - and even better idea how to handle it's state / monitoring / consistency.

cbsmith 11y ago
For the real use cases for the fancy clustering solution, the benefits can be huge... and complicated things built with failure in mind actually fail in quite simple ways. DGit is an example in itself. Sure it heavily leverages the complexity of git's versioning/branching/distributed vcs mechanism to get the job done, but compared to a SAN or RAID + DRBD, the failure scenarios are much more straightforward to deal with.
- viraptor 11y ago
  I agree. As long as you have people who can build it tailored to the service and handling all the errors. GH can afford doing that almost from scratch. Or if you're sure that this is exactly the solution you need and are familiar with the failure scenarios. But from what I've seen in a few places, a lot of people jump to cluster solutions without either a real need or enough people to support it.
  - cbsmith 11y ago
    > But from what I've seen in a few places, a lot of people jump to cluster solutions without either a real need or enough people to support it. Totally.