4 ms·

It's time we remembered the whole strength of the internet was that it was distributed and we avoided introducing single points of failure. We have ended up usi

It's time we remembered the whole strength of the internet was that it was distributed and we avoided introducing single points of failure. We have ended up using vast amounts of infrastructure for no reason other than developer convenience (often with respect to security), when having local direct connections is often more suitable than shooting everything into the cloud.

lurker14 14y ago
Which is better? Having a day of downtime each year, or not launching at all?
- peterwwillis 14y ago
  Which is better? Using a fallacious comparison to suggest cloud computing is the only viable option, or comparing the pros and cons of different computing models to choose the best one for you?
  - davidkatz 14y ago
    While the argument was perhaps coming on a bit too strong, it's hard to deny the ease of deployment on cloud services. It's probably a safe bet to say that for most early stage startups the cloud is a good move.
    - peterwwillis 14y ago
      I'm at a loss in these discussions. I don't understand this developer-point-of-view. Can you specifically give me examples of why using a cloud provider is better for a startup than, for example, using a couple desktops in your garage? You can't say it's because of backups because the cloud doesn't provide a backup (unless you purchase an extra data backup solution with your cloud provider?). And correct me if i'm wrong, but you still have to set up your development environment on your local computer to write the code, install libraries to test with, etc. What exactly are the steps involved in "deploying" that you couldn't do on your laptop, or a VPS?
      - robin_reala 14y ago
        For what it’s worth, Heroku’s PG Backups addon is free of charge. https://addons.heroku.com/pgbackups https://addons.heroku.com/pgbackups
        bgentry 14y ago
        We also perform our own backups regardless of PG Backups and use Continuous Protection to protect recent data, even in the event of irrecoverable volume failure: https://devcenter.heroku.com/articles/heroku-postgres-documentation#continuous-protection https://devcenter.heroku.com/articles/heroku-postgres-docume...
      - gtaylor 14y ago
        > Can you specifically give me examples of why using a cloud provider is better for a startup than, for example, using a couple desktops in your garage? For us (a small three-man team), the full portfolio of AWS services lets us shove responsibility for some of our infrastructure off to Amazon, which saves us loads of time (EC2, ELB, S3, CloudFront, SQS, Route53, SES, and some light DynamoDB in our case). Even with their repeat EBS issues, we've engineered around the common failure points, and do so with a tiny number of self-managed VMs relative to our traffic. Even if we were to go down every few months, our hosted services do better than a one-man devops team could ever do on his own in his garage, or even in a co-lo. Though AWS is not expensive, we gladly pay up in order to focus on our own software. Sure beats hiring another person, in our case. The common counter-argument is that if we managed it ourselves, that we'd have the ability to resolve outages on our own, because it'd be our responsibility. That is just the thing, we don't want it to be our responsibility, we've got one ops guy (me), and we need to be iterating on our product fast at this point. We'll instead plan for failure, design our architecture to continue operating under some degree of failure, and keep shipping. Not to say that the "cloud" is a silver bullet. It's not. However, especially from the developer point of view, it lets our entire (tiny) team stay more focused on product development. Addendum: I've managed servers in my basement, in co-lo, and in the "cloud". Each of these routes is more (or less) appropriate in various cases, but AWS has been a boon to our particular usage case.
      - 14y ago
- fidotron 14y ago
  It's not just the cloud itself, but your connection into it. I'm more concerned that my connection to the internet is lost so many times a day (traveling underground etc.) and that this is enough to prevent many devices in my vicinity from acting in a coherent way. They should be able to synchronize among themselves without the central intermediary. Obviously this doesn't happen because it's hard, but also companies have a vested interest in piping all sorts of data through them for analytics purposes. This is not in the interest of the users at all.
- prodigal_erik 14y ago
  That depends on how much long-term reputation damage you take from having availability that low before a large audience. Which in turn depends on how important your service really is—"can't play my game" is drastically different than "my landlord didn't receive my payment".
minm 14y ago
Point well made. We operate our tonido relay server across the world using linode, softlayer, herztner and it currently supports couple of million devices and half a million users.In the last 2 years our downtime is less than 30 minutes. It is nil for end users since they migrate to the nearest relay server hosted offered by a different provider. SPOF,security and control are the major issues with Iaas and pass offerings. SPOF, security and control are the major problems for the Iaas and pass.
digeridoo 14y ago
We shifted from services that fail often independently, to services failing rarely all at once. Clearly the former is going to be more noticeable and has greater societal impact, but as a business I'll take the latter any day.
magicalist 14y ago
Huh? Historically, almost all content on the internet has had a single point of failure. Moving "into the cloud" is moving to a distributed model, and generally app engine will protect you from single points of failure. As others have noted, if you distribute your own self-managed servers across the globe, you end up with a very similar system, but now you have to manage it. What appears to be happening here is that you're still vulnerable because the C&C infrastructure ultimately has a single command source, and so can be vulnerable if some code is pushed that affects the whole system. Your homegrown cloud will suffer from the same vulnerability, it may just be more or less easy to manage depending on how specialized your needs are compared to the more general requirements of GAE. Edit: actually, maybe I missed what you're advocating with "local direct connections". This might make more sense from a user's perspective: if everyone ran their own little cloud, a failure may bring down reddit, but not reddit, heroku, pinterest, etc simultaneously. That's actually an interesting point, but I'm not sure if it really matters if they sync up their downtimes (since they would still have downtime, and maybe more or less of it depending on how much they could afford to invest into managing their distributed solution). I'm also not sure if that really solves the problem, since there are other concentrations in the network, just less visible ones (there are a fairly small number of major datacenters around the world, for instance, and managing your own colocated server doesn't matter if the whole building goes dark). I do agree that at the very least we need to maintain an ecosystem of "cloud" providers, however.
- philwelch 14y ago
  It's distributed for you, but with an oligopoly of cloud providers it's considerably less distributed overall.
mhurron 14y ago
The resiliency of the internet has nothing to do with the uptime of any service running at endpoints. The resiliency is about the resiliency of the network, a router goes down, there are paths around it, not necessarily paths to every endpoint hanging off it or the services they offer.

lurker14 14y ago

Which is better? Having a day of downtime each year, or not launching at all?

peterwwillis 14y ago
Which is better? Using a fallacious comparison to suggest cloud computing is the only viable option, or comparing the pros and cons of different computing models to choose the best one for you?
- davidkatz 14y ago
  While the argument was perhaps coming on a bit too strong, it's hard to deny the ease of deployment on cloud services. It's probably a safe bet to say that for most early stage startups the cloud is a good move.
  - peterwwillis 14y ago
    I'm at a loss in these discussions. I don't understand this developer-point-of-view. Can you specifically give me examples of why using a cloud provider is better for a startup than, for example, using a couple desktops in your garage? You can't say it's because of backups because the cloud doesn't provide a backup (unless you purchase an extra data backup solution with your cloud provider?). And correct me if i'm wrong, but you still have to set up your development environment on your local computer to write the code, install libraries to test with, etc. What exactly are the steps involved in "deploying" that you couldn't do on your laptop, or a VPS?
    - robin_reala 14y ago
      For what it’s worth, Heroku’s PG Backups addon is free of charge. https://addons.heroku.com/pgbackups https://addons.heroku.com/pgbackups
      - bgentry 14y ago
        We also perform our own backups regardless of PG Backups and use Continuous Protection to protect recent data, even in the event of irrecoverable volume failure: https://devcenter.heroku.com/articles/heroku-postgres-documentation#continuous-protection https://devcenter.heroku.com/articles/heroku-postgres-docume...
    - gtaylor 14y ago
      > Can you specifically give me examples of why using a cloud provider is better for a startup than, for example, using a couple desktops in your garage? For us (a small three-man team), the full portfolio of AWS services lets us shove responsibility for some of our infrastructure off to Amazon, which saves us loads of time (EC2, ELB, S3, CloudFront, SQS, Route53, SES, and some light DynamoDB in our case). Even with their repeat EBS issues, we've engineered around the common failure points, and do so with a tiny number of self-managed VMs relative to our traffic. Even if we were to go down every few months, our hosted services do better than a one-man devops team could ever do on his own in his garage, or even in a co-lo. Though AWS is not expensive, we gladly pay up in order to focus on our own software. Sure beats hiring another person, in our case. The common counter-argument is that if we managed it ourselves, that we'd have the ability to resolve outages on our own, because it'd be our responsibility. That is just the thing, we don't want it to be our responsibility, we've got one ops guy (me), and we need to be iterating on our product fast at this point. We'll instead plan for failure, design our architecture to continue operating under some degree of failure, and keep shipping. Not to say that the "cloud" is a silver bullet. It's not. However, especially from the developer point of view, it lets our entire (tiny) team stay more focused on product development. Addendum: I've managed servers in my basement, in co-lo, and in the "cloud". Each of these routes is more (or less) appropriate in various cases, but AWS has been a boon to our particular usage case.
    - 14y ago
fidotron 14y ago
It's not just the cloud itself, but your connection into it. I'm more concerned that my connection to the internet is lost so many times a day (traveling underground etc.) and that this is enough to prevent many devices in my vicinity from acting in a coherent way. They should be able to synchronize among themselves without the central intermediary. Obviously this doesn't happen because it's hard, but also companies have a vested interest in piping all sorts of data through them for analytics purposes. This is not in the interest of the users at all.
prodigal_erik 14y ago
That depends on how much long-term reputation damage you take from having availability that low before a large audience. Which in turn depends on how important your service really is—"can't play my game" is drastically different than "my landlord didn't receive my payment".

minm 14y ago

Point well made. We operate our tonido relay server across the world using linode, softlayer, herztner and it currently supports couple of million devices and half a million users.In the last 2 years our downtime is less than 30 minutes. It is nil for end users since they migrate to the nearest relay server hosted offered by a different provider. SPOF,security and control are the major issues with Iaas and pass offerings. SPOF, security and control are the major problems for the Iaas and pass.

digeridoo 14y ago

We shifted from services that fail often independently, to services failing rarely all at once. Clearly the former is going to be more noticeable and has greater societal impact, but as a business I'll take the latter any day.

magicalist 14y ago

Huh? Historically, almost all content on the internet has had a single point of failure. Moving "into the cloud" is moving to a distributed model, and generally app engine will protect you from single points of failure. As others have noted, if you distribute your own self-managed servers across the globe, you end up with a very similar system, but now you have to manage it. What appears to be happening here is that you're still vulnerable because the C&C infrastructure ultimately has a single command source, and so can be vulnerable if some code is pushed that affects the whole system. Your homegrown cloud will suffer from the same vulnerability, it may just be more or less easy to manage depending on how specialized your needs are compared to the more general requirements of GAE. Edit: actually, maybe I missed what you're advocating with "local direct connections". This might make more sense from a user's perspective: if everyone ran their own little cloud, a failure may bring down reddit, but not reddit, heroku, pinterest, etc simultaneously. That's actually an interesting point, but I'm not sure if it really matters if they sync up their downtimes (since they would still have downtime, and maybe more or less of it depending on how much they could afford to invest into managing their distributed solution). I'm also not sure if that really solves the problem, since there are other concentrations in the network, just less visible ones (there are a fairly small number of major datacenters around the world, for instance, and managing your own colocated server doesn't matter if the whole building goes dark). I do agree that at the very least we need to maintain an ecosystem of "cloud" providers, however.

philwelch 14y ago
It's distributed for you, but with an oligopoly of cloud providers it's considerably less distributed overall.

mhurron 14y ago

The resiliency of the internet has nothing to do with the uptime of any service running at endpoints. The resiliency is about the resiliency of the network, a router goes down, there are paths around it, not necessarily paths to every endpoint hanging off it or the services they offer.