12 ms·

No. Our internal source control server is more reliable than our local shared ethernet with cable backup combined with the reliability of github. We've had 43

by batou 12y ago

No. Our internal source control server is more reliable than our local shared ethernet with cable backup combined with the reliability of github. We've had 43 minutes (scheduled and unscheduled combined) downtime in the last 11 years across three VCS technologies hosting internally. If it was github and remote, we're talking about 7 days of downtime in the last 2 years alone. We're definitely in charge and it has consumed virtually no admin effort in that time. The VCS server currently has an uptime of 3 years, 5 days, 20 hours and 38 minutes. It's also shifting 10mbit peaking at 200mbits pretty much all day every day 24/7/365. (CentOS 6, SVN, HP DL380 for ref)

emodendroket 12y ago
I've worked at a place where our public Web site would go down for more than a day at a time like, more than once a month... I think I'll take my chances with Github.
p4wnc6 12y ago
This is wildy uncommon. Borderline unheard of. In every job I've had, internally managed services failed with unscheduled outages at least monthly, sometimes for multiple days at a time. 3 years of up time? If I was hiring for system administrators and a candidate told me they had achieved that, I would just think they were lying.
- batou 12y ago
  We're very good at what we do and plan carefully. No joke. That's it. 2-4s downtime for a 40Gb svn repo migration is what we aim for. To be fair we haven't patched that box's kernel since it was rebooted but we really don't have to bounce kit very often. Most updates are handled through service restarts. Only updates that are an attack vector internally are actually applied too. As for the usual problems of power, we have battery and generator backed support. Redundant NICs on two VLANs and redundant power supplies on two feeds and a RAID array obviously. We also have a large memcache deployment that hasn't been restarted in that timescale as well. Even the memcache processes are that old. Works wonderfully. Also CentOS is pretty damn stable. If this was public facing it would be a slightly different story as we'd be patching the kernel too.
- stonogo 12y ago
  Your systems administrators in every job you've had should be fired. There is nothing particularly astounding about three years of uptime from a correctly-administered internal system.
  - emn13 12y ago
    Yeah - frankly, for most VCS's, it's pretty hard to achieve an uptime as poor as github locally(1). There's just not much that can go wrong; even doing virtually no maintenance you'd do better simply because github is hosted else and effected by all the vagaries of the internet. (1): by uptime I mean time up during office hours under the assumption that dev workstations can run. There's no point in measuring uptime during a power-outage...
darkarmani 12y ago
There is opportunity cost for this. How much does it cost to run a system like that?
- batou 12y ago
  Over 5 years, probably about the same ($3000)

by batou 12y ago

emodendroket 12y ago

I've worked at a place where our public Web site would go down for more than a day at a time like, more than once a month... I think I'll take my chances with Github.

p4wnc6 12y ago

This is wildy uncommon. Borderline unheard of. In every job I've had, internally managed services failed with unscheduled outages at least monthly, sometimes for multiple days at a time. 3 years of up time? If I was hiring for system administrators and a candidate told me they had achieved that, I would just think they were lying.

batou 12y ago
We're very good at what we do and plan carefully. No joke. That's it. 2-4s downtime for a 40Gb svn repo migration is what we aim for. To be fair we haven't patched that box's kernel since it was rebooted but we really don't have to bounce kit very often. Most updates are handled through service restarts. Only updates that are an attack vector internally are actually applied too. As for the usual problems of power, we have battery and generator backed support. Redundant NICs on two VLANs and redundant power supplies on two feeds and a RAID array obviously. We also have a large memcache deployment that hasn't been restarted in that timescale as well. Even the memcache processes are that old. Works wonderfully. Also CentOS is pretty damn stable. If this was public facing it would be a slightly different story as we'd be patching the kernel too.
stonogo 12y ago
Your systems administrators in every job you've had should be fired. There is nothing particularly astounding about three years of uptime from a correctly-administered internal system.
- emn13 12y ago
  Yeah - frankly, for most VCS's, it's pretty hard to achieve an uptime as poor as github locally(1). There's just not much that can go wrong; even doing virtually no maintenance you'd do better simply because github is hosted else and effected by all the vagaries of the internet. (1): by uptime I mean time up during office hours under the assumption that dev workstations can run. There's no point in measuring uptime during a power-outage...

darkarmani 12y ago

There is opportunity cost for this. How much does it cost to run a system like that?

batou 12y ago
Over 5 years, probably about the same ($3000)