6 ms·

Do you have any place you can point for that? Anecdotally, I have found that when we account for... - Human resources costs (payroll, taxes, benefits) [1] -

by throwaway2016a 11y ago

Do you have any place you can point for that? Anecdotally, I have found that when we account for... - Human resources costs (payroll, taxes, benefits) [1] - Time to market for new features / products - Utilizing reserved instances where it makes sense - Appropriately sizing machines We get with AWS... - Faster time to market (accelerated revenue) - Relatively the same cost per month (cheaper in some areas more expensive in others) - Significantly lower initial investment - Increased redundancy (via many small servers vs. a few large servers) / decreased disaster recovery times (and in many cases automated recovery) I'm not saying you're wrong, that's just what I've seen when we run the numbers internally. You may have seen differently which is why I ask. [1] A devops person to manage servers dedicated or not can cost $80-$125k+ a year after benefits and taxes. That is a lot of AWS instances. And we have found we need an IT staff about half the size to manage AWS vs a dedicated data center.

karterk 11y ago
Being on AWS does not necessarily mean that you don't need a devops person - especially at the scale where not being on AWS actually makes a difference to your margins. I have seen quite a few people move off AWS successfully onto bare-metal leased hardware. S3 is just about the only service that's difficult to find an alternative for. Personally, I find using something like DynamoDB no different than using an Oracle DB - it's a vendor lock-in. Unless you had enterprise-level support on AWS (which costs a lot) - if you run into issues with Amazon's proprietary services, then good luck to you. AWS is great to get started, but once you know that you're going to need scale (and lots of infra), it's best to move. I say all of this as someone who has extensive operating experience on AWS. YMMV.
- UK-AL 11y ago
  Oh, with cloud services you defiantly need devops. Without AWS you also need dedicated infrastructure, networking and hardware people as well as devops. You need people who know how to configure cisco networking gear, who understand SAN's, iSCSI, Fibrechannel, Racks, blades lots of stuff even devops people don't think about.
  - latch 11y ago
    No. This is true for colocated, but absolutely false for dedicated.
    - toomuchtodo 11y ago
      You are correct! If you're just colocating, you need your own people to manage your gear. Dedicated equipment is managed by the service provider. Disclaimer: Provided hosting services for ~8 years.
      - UK-AL 11y ago
        Companies that provide managed hosting for hardware tend be just as expensive as cloud providers. These comparisons tend to be, oh look you can buy the the Dell Server for cheap, and shove it in data centre and it's a lot cheaper. If you start going route where you ask the provider to do it, the charges tend to be a lot more. It usually the sort thing where you need to do a phone call before you can even get a quote.
        latch 11y ago
        Again, this is demonstrably false. What's commonly meant by "dedicated hosting" is essentially the same service level you get from EC2 - they they care of the network and hardware, you take care of the software (1). The cost here is 2x-4x lower. In another comment, I quoted E5-2620 v3 w/64GB for $400 which is exactly this type of setup. This is offered by a company that's been in business for longer than AWS has existed and who, in web hosting circles, is well respected. You could definitely go cheaper. You could definitely go to IBM and rackspace and pay as much as EC2, sure...but there are literally thousands of providers in the US, which have been in business for over a decade that'll beat EC2/Rackspace/Softlayer by a wide margin. Some that I've personally used: WebNX (LA), NetDepot (Atlanta), ReliableSite (NY), HiVelocity (Florida). I've also done it on the cheap with OVH (both Quebec and France) and Hetzner (Germany), as well as on the expensive side with Softlayer (you can negotiate softlayer down considerably even on a small order). (1) That's disservice to dedicated hosting, because the quality of the network is often better, and you won't get termination emails from your hosting provider or noisy neighbours the way like you do on EC2.
- throwaway2016a 11y ago
  All valid points. > Being on AWS does not necessarily mean that you don't need a devops person - especially at the scale where not being on AWS actually makes a difference to your margins. Agreed. I didn't mean to imply that. On our scale it means we need 2 instead of 4. And for consulting business, 1 person can handle multiple smaller clients so each client doesn't need to take on the full cost. Incidentally, since moving some of our self-managed servers to AWS services we have seen a drastic decrease in how often our on-call engineer is woken up at 3 AM. Which makes them happy. And happy employees are always a good investment. Admittedly, given enough time we could have made our stuff as resilient as AWS. Also, being able to call Amazon support and get a second eye on things helps. Albeit, the support plan is a bit pricy. In our case $10k a year. > I have seen quite a few people move off AWS successfully onto bare-metal leased hardware. S3 is just about the only service that's difficult to find an alternative for. Personally, I find using something like DynamoDB no different than using an Oracle DB - it's a vendor lock-in. Unless you had enterprise-level support on AWS (which costs a lot) - if you run into issues with Amazon's proprietary services, then good luck to you. Vendor lock-in is a major issue. Which is why we used self-managed databases on AWS for years (vs RDS, Redshift, or DynamoDB). Our conclusion -- after a few years -- was in our case (YMMV) we could accept the vendor lock-in but make sure our code was abstracted in a way that make moving easier. Plus with AWS's new Database Migration System moving databases off (or on) to AWS is pretty easy. Also, there are actually some really good S3 alternatives now. Their names escape me. I'll look them up later. However, I've seen many companies use only S3, Glacier, and CloudFront but not EC2. Your servers don't need to be on AWS to use them, obviously. Almost all AWS services have good open source alternatives. And we have spent the time to make sure system is architected and our code is written in a way that has a clear path to switching. Microservices really help here. > AWS is great to get started, but once you know that you're going to need scale (and lots of infra), it's best to move. I say all of this as someone who has extensive operating experience on AWS. YMMV. Maybe. But once you buy reserved instances in AWS the cost is pretty low. And when hardware fails in AWS it doesn't cost you more. (Assuming your architected without single points of failure) I've found a lot of people did the math before AWS lowered their costs and introduced reserved instances. Either way, this is why I said "cheaper in some ways and more expensive in others"
  - 11y ago
- homero 11y ago
  B2 kills s3
  - alainv 11y ago
    Maybe once it leaves beta.
nirvdrum 11y ago
For my last start-up, I moved off EC2 to a dedicated vSphere cluster with a hosted provider. vSphere has an API, so adapting existing provisioning & deployment code was quite straightforward (I used the fog gem in Ruby). I basically treated the vSphere cluster very similarly to EC2 (small root stores, attached volumes, etc). Granted, I did have to give up the benefit of being in different regions. I found the maintenance burden dropped substantially. It may have just been that I was running on newer hardware, but vSphere has built-in HA features such that VMs will just migrate between hosts when hardware degrades. In the two years I ran that setup, I never lost a VM. I also had dedicated, modern hardware -- no need to worry about CPU steal. I could create instances/VMs of any size I want. If I needed to resize a VM, I could do so without losing all the data on it. As long as I had capacity, I could add new VMs at no extra cost. When I needed more capacity, I'd just call up and have them add a new blade to the cluster, which both increased my redundancy and gave me extra capacity. Essentially, the cost curve, while tied to usage, was much more favorable than linear growth once you get over the base setup costs. On top of that, I had a real console to each of the VMs should anything go wrong. And if I couldn't fix something myself, there was dedicated staff at the colo facility I could just call up. It's not for everyone, but there are options out there that give you an EC2-like environment with many of the benefits of having your own hardware.
- throwaway2016a 11y ago
  The tradeoff for CPU stealing is that now you have multiple services running on the same machine so if the machine goes down you lose them all. On AWS it is very unlikely that two of your VMs end up on the same physical machine. Also, I actually can't recall (in 8 years of AWS) a time where I saw CPU being stolen. Perhaps I've been lucky. I also don't watch for it constantly. We actually did this with our dev servers. Although switch VSphere for VirtualBox on Linux and colo for a server room in our office. It worked great and saved a TON of cost. But we don't need pesky things like 24hr uptime and dedicated bandwidth for dev servers.
  - toomuchtodo 11y ago
    > The tradeoff for CPU stealing is that now you have multiple services running on the same machine so if the machine goes down you lose them all. On AWS it is very unlikely that two of your VMs end up on the same physical machine. Yes. But its also very likely that when AWS has issues, the entire region is going to be having problems, like last Saturday (IAM, EC2, Autoscaling, etc broken badly for 6 hours). > But we don't need pesky things like 24hr uptime and dedicated bandwidth for dev servers. You're not getting this SLA unless you're managing everything yourself and are globally redundant. AWS doesn't provide bandwidth guarantees, nor will you get 100% uptime.
deleted 11y ago
[deleted]
dsp1234 11y ago
Relatively the same cost per month For us, AWS is not even remotely close in cost to a rack, hardware, and staff. The estimated spend for us to duplicate our dedicated rack setup in AWS is 3 times our monthly operational costs. And this does not include using any additional services. And to be clear, this includes operational staff, spare hardware, disaster recovery, etc.

karterk 11y ago

Being on AWS does not necessarily mean that you don't need a devops person - especially at the scale where not being on AWS actually makes a difference to your margins. I have seen quite a few people move off AWS successfully onto bare-metal leased hardware. S3 is just about the only service that's difficult to find an alternative for. Personally, I find using something like DynamoDB no different than using an Oracle DB - it's a vendor lock-in. Unless you had enterprise-level support on AWS (which costs a lot) - if you run into issues with Amazon's proprietary services, then good luck to you. AWS is great to get started, but once you know that you're going to need scale (and lots of infra), it's best to move. I say all of this as someone who has extensive operating experience on AWS. YMMV.

UK-AL 11y ago
Oh, with cloud services you defiantly need devops. Without AWS you also need dedicated infrastructure, networking and hardware people as well as devops. You need people who know how to configure cisco networking gear, who understand SAN's, iSCSI, Fibrechannel, Racks, blades lots of stuff even devops people don't think about.
- latch 11y ago
  No. This is true for colocated, but absolutely false for dedicated.
  - toomuchtodo 11y ago
    You are correct! If you're just colocating, you need your own people to manage your gear. Dedicated equipment is managed by the service provider. Disclaimer: Provided hosting services for ~8 years.
    - UK-AL 11y ago
      Companies that provide managed hosting for hardware tend be just as expensive as cloud providers. These comparisons tend to be, oh look you can buy the the Dell Server for cheap, and shove it in data centre and it's a lot cheaper. If you start going route where you ask the provider to do it, the charges tend to be a lot more. It usually the sort thing where you need to do a phone call before you can even get a quote.
      - latch 11y ago
        Again, this is demonstrably false. What's commonly meant by "dedicated hosting" is essentially the same service level you get from EC2 - they they care of the network and hardware, you take care of the software (1). The cost here is 2x-4x lower. In another comment, I quoted E5-2620 v3 w/64GB for $400 which is exactly this type of setup. This is offered by a company that's been in business for longer than AWS has existed and who, in web hosting circles, is well respected. You could definitely go cheaper. You could definitely go to IBM and rackspace and pay as much as EC2, sure...but there are literally thousands of providers in the US, which have been in business for over a decade that'll beat EC2/Rackspace/Softlayer by a wide margin. Some that I've personally used: WebNX (LA), NetDepot (Atlanta), ReliableSite (NY), HiVelocity (Florida). I've also done it on the cheap with OVH (both Quebec and France) and Hetzner (Germany), as well as on the expensive side with Softlayer (you can negotiate softlayer down considerably even on a small order). (1) That's disservice to dedicated hosting, because the quality of the network is often better, and you won't get termination emails from your hosting provider or noisy neighbours the way like you do on EC2.
throwaway2016a 11y ago
All valid points. > Being on AWS does not necessarily mean that you don't need a devops person - especially at the scale where not being on AWS actually makes a difference to your margins. Agreed. I didn't mean to imply that. On our scale it means we need 2 instead of 4. And for consulting business, 1 person can handle multiple smaller clients so each client doesn't need to take on the full cost. Incidentally, since moving some of our self-managed servers to AWS services we have seen a drastic decrease in how often our on-call engineer is woken up at 3 AM. Which makes them happy. And happy employees are always a good investment. Admittedly, given enough time we could have made our stuff as resilient as AWS. Also, being able to call Amazon support and get a second eye on things helps. Albeit, the support plan is a bit pricy. In our case $10k a year. > I have seen quite a few people move off AWS successfully onto bare-metal leased hardware. S3 is just about the only service that's difficult to find an alternative for. Personally, I find using something like DynamoDB no different than using an Oracle DB - it's a vendor lock-in. Unless you had enterprise-level support on AWS (which costs a lot) - if you run into issues with Amazon's proprietary services, then good luck to you. Vendor lock-in is a major issue. Which is why we used self-managed databases on AWS for years (vs RDS, Redshift, or DynamoDB). Our conclusion -- after a few years -- was in our case (YMMV) we could accept the vendor lock-in but make sure our code was abstracted in a way that make moving easier. Plus with AWS's new Database Migration System moving databases off (or on) to AWS is pretty easy. Also, there are actually some really good S3 alternatives now. Their names escape me. I'll look them up later. However, I've seen many companies use only S3, Glacier, and CloudFront but not EC2. Your servers don't need to be on AWS to use them, obviously. Almost all AWS services have good open source alternatives. And we have spent the time to make sure system is architected and our code is written in a way that has a clear path to switching. Microservices really help here. > AWS is great to get started, but once you know that you're going to need scale (and lots of infra), it's best to move. I say all of this as someone who has extensive operating experience on AWS. YMMV. Maybe. But once you buy reserved instances in AWS the cost is pretty low. And when hardware fails in AWS it doesn't cost you more. (Assuming your architected without single points of failure) I've found a lot of people did the math before AWS lowered their costs and introduced reserved instances. Either way, this is why I said "cheaper in some ways and more expensive in others"
- 11y ago
homero 11y ago
B2 kills s3
- alainv 11y ago
  Maybe once it leaves beta.

nirvdrum 11y ago

For my last start-up, I moved off EC2 to a dedicated vSphere cluster with a hosted provider. vSphere has an API, so adapting existing provisioning & deployment code was quite straightforward (I used the fog gem in Ruby). I basically treated the vSphere cluster very similarly to EC2 (small root stores, attached volumes, etc). Granted, I did have to give up the benefit of being in different regions. I found the maintenance burden dropped substantially. It may have just been that I was running on newer hardware, but vSphere has built-in HA features such that VMs will just migrate between hosts when hardware degrades. In the two years I ran that setup, I never lost a VM. I also had dedicated, modern hardware -- no need to worry about CPU steal. I could create instances/VMs of any size I want. If I needed to resize a VM, I could do so without losing all the data on it. As long as I had capacity, I could add new VMs at no extra cost. When I needed more capacity, I'd just call up and have them add a new blade to the cluster, which both increased my redundancy and gave me extra capacity. Essentially, the cost curve, while tied to usage, was much more favorable than linear growth once you get over the base setup costs. On top of that, I had a real console to each of the VMs should anything go wrong. And if I couldn't fix something myself, there was dedicated staff at the colo facility I could just call up. It's not for everyone, but there are options out there that give you an EC2-like environment with many of the benefits of having your own hardware.

throwaway2016a 11y ago
The tradeoff for CPU stealing is that now you have multiple services running on the same machine so if the machine goes down you lose them all. On AWS it is very unlikely that two of your VMs end up on the same physical machine. Also, I actually can't recall (in 8 years of AWS) a time where I saw CPU being stolen. Perhaps I've been lucky. I also don't watch for it constantly. We actually did this with our dev servers. Although switch VSphere for VirtualBox on Linux and colo for a server room in our office. It worked great and saved a TON of cost. But we don't need pesky things like 24hr uptime and dedicated bandwidth for dev servers.
- toomuchtodo 11y ago
  > The tradeoff for CPU stealing is that now you have multiple services running on the same machine so if the machine goes down you lose them all. On AWS it is very unlikely that two of your VMs end up on the same physical machine. Yes. But its also very likely that when AWS has issues, the entire region is going to be having problems, like last Saturday (IAM, EC2, Autoscaling, etc broken badly for 6 hours). > But we don't need pesky things like 24hr uptime and dedicated bandwidth for dev servers. You're not getting this SLA unless you're managing everything yourself and are globally redundant. AWS doesn't provide bandwidth guarantees, nor will you get 100% uptime.

deleted 11y ago

[deleted]

dsp1234 11y ago

Relatively the same cost per month For us, AWS is not even remotely close in cost to a rack, hardware, and staff. The estimated spend for us to duplicate our dedicated rack setup in AWS is 3 times our monthly operational costs. And this does not include using any additional services. And to be clear, this includes operational staff, spare hardware, disaster recovery, etc.