3 ms·

Can you explain how can physical hardware concerns be handwaved away? It's not just about the hardware itself, that's the easiest part. It's about building an

Can you explain how can physical hardware concerns be handwaved away? It's not just about the hardware itself, that's the easiest part. It's about building and maintaining the electrical/cooling/network infrastructure the servers require. It's about having multiple locations of that in case your building's power or network goes down/there's a flood/somebody steals all your stuff/whatever else. Or you rent a rack in a colo (or five)... But that's just a step away from going full cloud, you just do a lot of work yourself and get no guarantees and no flexibility. At that point, why not just go full cloud?

endisneigh 3y ago
This is not the 90s - monitoring and alerting are far more robust, and supply chains far more just in time. There’s no reason a modern fleet couldn’t detect failures in hardware and automatically order replacement from Amazon or whatever. This is not to say you won’t need personnel, but you would be surprised how much of the fleet management at AWS and GCP at least are brute forced (as opposed to being completely automatic). To their defense they have a far more complicated and diverse fleet than what I’m describing, which is sort of my point. 99% of customers just need a highly available setup for their stateful boxes (DBs) and some for their stateless services. And the great thing about this setup is you can extend the stateless bit (with the cost of latency) to the cloud more or less infinitely. As far as the rest of your point about cooling and network - it's a good point, but for a workload that isn't a datacenter, in my experience the majority of outages aren't due to that - it's due to misconfiguration. Take a personal house. How often does someone break into the average person's home? Pretty unlikely in a low crime area. Take internet. There are some areas in the United States where there are multiple 1Gbs or even 10Gbs internet providers, you could redundantly network. In any case it's not that the cloud shouldn't be used, it's just interesting the direction things are going. Of course, if money isn't an issue by all means people should use serverless and cloud spanner and be done with it.
- solatic 3y ago
  > This is not to say you won’t need personnel Ding ding ding, we have a winner! Paying premiums for BigCloud means not paying salaries, not worrying about an outage that could happen while your seniors are on vacation, not needing to set up resilient internal processes and controls for datacenter management, not worrying about datacenter management compliance, etc. etc. Needing to hire datacenter personnel is a problem and it is a problem that BigCloud solved.
  - endisneigh 3y ago
    You're thinking about it the wrong way. Sure you're not paying salaries, but now you're paying for experts who understand the cloud and you're paying more for those people compared to datacenter people. At Azure I was surprised how many data center people didn't even have a college degree and how little they were paid. Ultimately the only thing that matters is the total cost. To take true advantage of the cloud you need to use cloud specific technologies - serverless being the greatest example. Cloud VMs are neat and all, but in a sense it's the worst of both worlds. I'm not saying there isn't value in the cloud. I'm saying most people are overpaying. BigCloud didn't solve this any more than paying employees in general to solve your problems. With the era of free money coming to the end, many companies will come to this realization themselves in any case. BigTech clouds have 50%+ margins - something will give eventually. Any serious company is still going to have to pay oncallers, and admins with or without cloud. It's not like using the cloud absolves you from maintenance (pretty much every company with a valuation more than 100 million has an infrastructure team, and uses the cloud. So clearly the cloud doesn't mean you don't have to maintain your own infrastructure). And now we come back to my point - why isn't the fleet easier to maintain to begin with? I'm not even talking about bare-metal necessarily. Say you use VMs. Still a huge hassle.
    - DANmode 3y ago
      > BigTech clouds have 50%+ margins - something will give eventually. Yes, the oligopoly, and the vendor lock-in. Certainly won't be expecting the "advantage" of being able to outsource your sysadmins to somewhere else at the drop of a hat to be the thing that gives.
verdverm 3y ago
On GCP, they will live migrate your running VMs to new racks and you won't likely know unless you look for it in the logs, super nice feature, I once got bit by AWS and a disk physical failure, had to migrate myself (really a nuke and recreate since we couldn't do anything with the VM)
jjav 3y ago
> It's about building and maintaining the electrical/cooling/network infrastructure the servers require. Except at the extreme high end of scale, you never do this yourself. Rent racks in a colocation. It's a very clean separation of duties. You handle all the tech work, they handle all the real estate upkeep. > Or you rent a rack in a colo (or five)... But that's just a step away from going full cloud What? Not even close. When you rent a rack in a colo you have 100% of the benefits of owning the hardware (much higher performance, much lower cost). Just because you let someone else do the A/C and generator maintenance (and other non-computer real estate logistics) doesn't make it anything at all like the cloud.

endisneigh 3y ago

This is not the 90s - monitoring and alerting are far more robust, and supply chains far more just in time. There’s no reason a modern fleet couldn’t detect failures in hardware and automatically order replacement from Amazon or whatever. This is not to say you won’t need personnel, but you would be surprised how much of the fleet management at AWS and GCP at least are brute forced (as opposed to being completely automatic). To their defense they have a far more complicated and diverse fleet than what I’m describing, which is sort of my point. 99% of customers just need a highly available setup for their stateful boxes (DBs) and some for their stateless services. And the great thing about this setup is you can extend the stateless bit (with the cost of latency) to the cloud more or less infinitely. As far as the rest of your point about cooling and network - it's a good point, but for a workload that isn't a datacenter, in my experience the majority of outages aren't due to that - it's due to misconfiguration. Take a personal house. How often does someone break into the average person's home? Pretty unlikely in a low crime area. Take internet. There are some areas in the United States where there are multiple 1Gbs or even 10Gbs internet providers, you could redundantly network. In any case it's not that the cloud shouldn't be used, it's just interesting the direction things are going. Of course, if money isn't an issue by all means people should use serverless and cloud spanner and be done with it.

solatic 3y ago
> This is not to say you won’t need personnel Ding ding ding, we have a winner! Paying premiums for BigCloud means not paying salaries, not worrying about an outage that could happen while your seniors are on vacation, not needing to set up resilient internal processes and controls for datacenter management, not worrying about datacenter management compliance, etc. etc. Needing to hire datacenter personnel is a problem and it is a problem that BigCloud solved.
- endisneigh 3y ago
  You're thinking about it the wrong way. Sure you're not paying salaries, but now you're paying for experts who understand the cloud and you're paying more for those people compared to datacenter people. At Azure I was surprised how many data center people didn't even have a college degree and how little they were paid. Ultimately the only thing that matters is the total cost. To take true advantage of the cloud you need to use cloud specific technologies - serverless being the greatest example. Cloud VMs are neat and all, but in a sense it's the worst of both worlds. I'm not saying there isn't value in the cloud. I'm saying most people are overpaying. BigCloud didn't solve this any more than paying employees in general to solve your problems. With the era of free money coming to the end, many companies will come to this realization themselves in any case. BigTech clouds have 50%+ margins - something will give eventually. Any serious company is still going to have to pay oncallers, and admins with or without cloud. It's not like using the cloud absolves you from maintenance (pretty much every company with a valuation more than 100 million has an infrastructure team, and uses the cloud. So clearly the cloud doesn't mean you don't have to maintain your own infrastructure). And now we come back to my point - why isn't the fleet easier to maintain to begin with? I'm not even talking about bare-metal necessarily. Say you use VMs. Still a huge hassle.
  - DANmode 3y ago
    > BigTech clouds have 50%+ margins - something will give eventually. Yes, the oligopoly, and the vendor lock-in. Certainly won't be expecting the "advantage" of being able to outsource your sysadmins to somewhere else at the drop of a hat to be the thing that gives.

verdverm 3y ago

On GCP, they will live migrate your running VMs to new racks and you won't likely know unless you look for it in the logs, super nice feature, I once got bit by AWS and a disk physical failure, had to migrate myself (really a nuke and recreate since we couldn't do anything with the VM)

jjav 3y ago

> It's about building and maintaining the electrical/cooling/network infrastructure the servers require. Except at the extreme high end of scale, you never do this yourself. Rent racks in a colocation. It's a very clean separation of duties. You handle all the tech work, they handle all the real estate upkeep. > Or you rent a rack in a colo (or five)... But that's just a step away from going full cloud What? Not even close. When you rent a rack in a colo you have 100% of the benefits of owning the hardware (much higher performance, much lower cost). Just because you let someone else do the A/C and generator maintenance (and other non-computer real estate logistics) doesn't make it anything at all like the cloud.