6 ms·

> One last thing about getting dedicated hardware. It's cheaper a lot cheaper. We have machines that give us 2-4x performance that cost less than half as much a

> One last thing about getting dedicated hardware. It's cheaper a lot cheaper. We have machines that give us 2-4x performance that cost less than half as much as their cloud equivalents and we're not even co-locating. I've been saying this for ages, and every time people would fall over backwards trying to defend/prove their cloud mistake... "The cloud is cheaper, faster, and infinitely scalable." Except none of those 3 is true for any real world use case, but a few. The moment a popular site like Reddit switches to the cloud, is the moment it becomes barely usable during certain times of the day.

pablasso 15y ago
I don't think people argue solely on hardware specs, you should count the maintenance and related salaries into the total cost.
- powertower 15y ago
  The majority of that market is getting VPSs or renting dedicated boxes. There is no hardware maintenance or related salaries. Even for multiple boxes, you call up your provider, and order a cage with hardware filled. They'll provide everything. 95% of "Cloud" is just a new marketing term for VPS.
- rhizome 15y ago
  Any technical founder or CTO should be able to administer an application box and have some DBA skills.
bitemyapp 15y ago
I agree with everything you said except for: >The moment a popular site like Reddit switches to the cloud, is the moment it becomes barely usable during certain times of the day. Reddit is largely on EC2, do you even know what you're talking about? EC2 is part of Reddit has so many problems, among others.
ChuckMcM 15y ago
So I'm running operations at Blekko, and just prior to that I was at Google in their eng/ops organization. I had been doing a whole lot of 'total cost of ownership' aka TCO computations around engineering infrastructure both for Google and of course now for Blekko. The conclusion I came to is that for a 'web 2.0' type setup, the break even point was about 500 'machines.' That was in part because a 'machine' today has 8 - 24 'threads' and 2 - 40T of 'storage' and (at the time) 2 - 96G 'memory.' So in terms of 'cloud' you could easy run 10 "instances" on these sorts of machine. So 500 machines might be 5000 'instances' in an AWS type cloud. Its this '10:1' multiplier effect (which is only getting better with bigger machines) and the management techniques of running the same config everywhere, etc. Means your TCO goes up more slowly than the capacity of the resulting infrastructure, so you can 'solve for x' where the two lines cross to identify the break even point. Everything east of that point you're coming out ahead of a 'cloud' based deployment. What is still a challenge however is geographic diversity. If you wanted to put 500 machines 'around the world' so 125 machines in each 90 degrees (approximately) of longitude, the economics of getting 5 - 10 'cabinets' in places around the world can work against you. (you have more negotiating power if you're putting in 100 racks than if you are putting in 10 racks)
- driverdan 15y ago
  This may be true if the available virtual servers fit your configuration needs. If they don't individually provide enough IO bandwidth, storage space, RAM, CPU time, etc and you have to add a second virtual server to make up for the lacking attribute the ratio could change from 10:1 to 20:1, making dedicated servers an even better option. The more custom your configuration the better bare metal is.
- robterrell 15y ago
  You should write this up and post it somewhere. It sounds like something I'd like to read the details of.
- rachelbythebay 15y ago
  Beware the "same config everywhere" approach. It works up to a point, then it turns into a disaster. All you need is one totally broken change like "chmod -x /usr" to really make life interesting. You start bleeding machines and pretty soon you have nowhere left to host your tasks. It's interesting, right? At first, you can handle a couple of totally mixed-up machines. Then it stops scaling and you have to start doing the whole "golden + syncer" approach. Then you go too far and get into a monoculture. When the machines do break, it's impossible for humans to go around and fix them in any reasonable amount of time because there are too many. It's amusing when this happens and the solution put forth is "more administrative controls".
jedberg 15y ago
> The moment a popular site like Reddit switches to the cloud, is the moment it becomes barely usable during certain times of the day. That's completely false. The moment reddit blogged about moving to the cloud, it was * perceived* as being slower. The cloud move happened 7 months before the blog post came out.
- towelrod 15y ago
  I think the OP just meant that once someone like reddit moves into your cloud, your performance goes down. Meaning, the hardware that my little blog is on works great until that hardware also hosting reddit, and then my blog takes a beating because reddit is thrashing away with actual activity.
  - imbriaco 15y ago
    Not if the cloud you're running your workload on is large enough. The Amazon Web Services cloud could absorb many Reddit sized sites without any other customers experiencing an impact. AWS is _enormous_.
    - mnutt 15y ago
      Not in the sense of running out of machines, but at least from my past experiences if your VMs share physical hardware with someone who is using their maximum IO, yours will suffer as well. It makes it more challenging to load test. When there is no contention you can usually 'burst' to use more of the machine's resources, but you can't necessarily trust that you will always have that capacity.
  - res0nat0r 15y ago
    Reddit wasn't just running on one instance on a single physical server in AWS mind you...
asharp 15y ago
Very few workloads need to scale out past the limits of any reasonable cloud provider. Properly built clouds can give you IO faster then a dedicated server, and give you CPUS on demand that far exceed what you would otherwise afford (due to you only needing it occasionally). And any reasonable cloud provider can discount things such that is unreasonable for you to move off. They have a much lower bottom line then a comparable dedicated server provider. Is interesting that this doesn't seem to be the case currently.
pbreit 15y ago
I don't think I've ever heard anyone claim that the cloud was generally cheaper and faster. Only that it is cheaper at small scale and faster to set up.

pablasso 15y ago

I don't think people argue solely on hardware specs, you should count the maintenance and related salaries into the total cost.

powertower 15y ago
The majority of that market is getting VPSs or renting dedicated boxes. There is no hardware maintenance or related salaries. Even for multiple boxes, you call up your provider, and order a cage with hardware filled. They'll provide everything. 95% of "Cloud" is just a new marketing term for VPS.
rhizome 15y ago
Any technical founder or CTO should be able to administer an application box and have some DBA skills.

bitemyapp 15y ago

I agree with everything you said except for: >The moment a popular site like Reddit switches to the cloud, is the moment it becomes barely usable during certain times of the day. Reddit is largely on EC2, do you even know what you're talking about? EC2 is part of Reddit has so many problems, among others.

ChuckMcM 15y ago

So I'm running operations at Blekko, and just prior to that I was at Google in their eng/ops organization. I had been doing a whole lot of 'total cost of ownership' aka TCO computations around engineering infrastructure both for Google and of course now for Blekko. The conclusion I came to is that for a 'web 2.0' type setup, the break even point was about 500 'machines.' That was in part because a 'machine' today has 8 - 24 'threads' and 2 - 40T of 'storage' and (at the time) 2 - 96G 'memory.' So in terms of 'cloud' you could easy run 10 "instances" on these sorts of machine. So 500 machines might be 5000 'instances' in an AWS type cloud. Its this '10:1' multiplier effect (which is only getting better with bigger machines) and the management techniques of running the same config everywhere, etc. Means your TCO goes up more slowly than the capacity of the resulting infrastructure, so you can 'solve for x' where the two lines cross to identify the break even point. Everything east of that point you're coming out ahead of a 'cloud' based deployment. What is still a challenge however is geographic diversity. If you wanted to put 500 machines 'around the world' so 125 machines in each 90 degrees (approximately) of longitude, the economics of getting 5 - 10 'cabinets' in places around the world can work against you. (you have more negotiating power if you're putting in 100 racks than if you are putting in 10 racks)

driverdan 15y ago
This may be true if the available virtual servers fit your configuration needs. If they don't individually provide enough IO bandwidth, storage space, RAM, CPU time, etc and you have to add a second virtual server to make up for the lacking attribute the ratio could change from 10:1 to 20:1, making dedicated servers an even better option. The more custom your configuration the better bare metal is.
robterrell 15y ago
You should write this up and post it somewhere. It sounds like something I'd like to read the details of.
rachelbythebay 15y ago
Beware the "same config everywhere" approach. It works up to a point, then it turns into a disaster. All you need is one totally broken change like "chmod -x /usr" to really make life interesting. You start bleeding machines and pretty soon you have nowhere left to host your tasks. It's interesting, right? At first, you can handle a couple of totally mixed-up machines. Then it stops scaling and you have to start doing the whole "golden + syncer" approach. Then you go too far and get into a monoculture. When the machines do break, it's impossible for humans to go around and fix them in any reasonable amount of time because there are too many. It's amusing when this happens and the solution put forth is "more administrative controls".

jedberg 15y ago

> The moment a popular site like Reddit switches to the cloud, is the moment it becomes barely usable during certain times of the day. That's completely false. The moment reddit blogged about moving to the cloud, it was * perceived* as being slower. The cloud move happened 7 months before the blog post came out.

towelrod 15y ago
I think the OP just meant that once someone like reddit moves into your cloud, your performance goes down. Meaning, the hardware that my little blog is on works great until that hardware also hosting reddit, and then my blog takes a beating because reddit is thrashing away with actual activity.
- imbriaco 15y ago
  Not if the cloud you're running your workload on is large enough. The Amazon Web Services cloud could absorb many Reddit sized sites without any other customers experiencing an impact. AWS is _enormous_.
  - mnutt 15y ago
    Not in the sense of running out of machines, but at least from my past experiences if your VMs share physical hardware with someone who is using their maximum IO, yours will suffer as well. It makes it more challenging to load test. When there is no contention you can usually 'burst' to use more of the machine's resources, but you can't necessarily trust that you will always have that capacity.
- res0nat0r 15y ago
  Reddit wasn't just running on one instance on a single physical server in AWS mind you...

asharp 15y ago

Very few workloads need to scale out past the limits of any reasonable cloud provider. Properly built clouds can give you IO faster then a dedicated server, and give you CPUS on demand that far exceed what you would otherwise afford (due to you only needing it occasionally). And any reasonable cloud provider can discount things such that is unreasonable for you to move off. They have a much lower bottom line then a comparable dedicated server provider. Is interesting that this doesn't seem to be the case currently.

pbreit 15y ago

I don't think I've ever heard anyone claim that the cloud was generally cheaper and faster. Only that it is cheaper at small scale and faster to set up.