6 ms·

If saving money and giving your users a better experience are priorities then, in general, moving off AWS is worth considering. When it comes to EC2, spot insta

by latch 11y ago

If saving money and giving your users a better experience are priorities then, in general, moving off AWS is worth considering. When it comes to EC2, spot instances or maybe you're doing it wrong. Conservatively speaking, without bandwidth, you're looking EC2 costing 2-4x more than dedicated while being 2-4x slower. This does depend on specific workload, and the gap has been closing (conversely, I've seen specific workloads be worse than 4x slower). I know RDS is convenient. But learning how to setup and manage your own database is actually a fundamental skill that will serve you well. All learning can be seen as an opportunity cost, but this is one that will let you save money every month, and give your users a better experience.

throwaway2016a 11y ago
Do you have any place you can point for that? Anecdotally, I have found that when we account for... - Human resources costs (payroll, taxes, benefits) [1] - Time to market for new features / products - Utilizing reserved instances where it makes sense - Appropriately sizing machines We get with AWS... - Faster time to market (accelerated revenue) - Relatively the same cost per month (cheaper in some areas more expensive in others) - Significantly lower initial investment - Increased redundancy (via many small servers vs. a few large servers) / decreased disaster recovery times (and in many cases automated recovery) I'm not saying you're wrong, that's just what I've seen when we run the numbers internally. You may have seen differently which is why I ask. [1] A devops person to manage servers dedicated or not can cost $80-$125k+ a year after benefits and taxes. That is a lot of AWS instances. And we have found we need an IT staff about half the size to manage AWS vs a dedicated data center.
- karterk 11y ago
  Being on AWS does not necessarily mean that you don't need a devops person - especially at the scale where not being on AWS actually makes a difference to your margins. I have seen quite a few people move off AWS successfully onto bare-metal leased hardware. S3 is just about the only service that's difficult to find an alternative for. Personally, I find using something like DynamoDB no different than using an Oracle DB - it's a vendor lock-in. Unless you had enterprise-level support on AWS (which costs a lot) - if you run into issues with Amazon's proprietary services, then good luck to you. AWS is great to get started, but once you know that you're going to need scale (and lots of infra), it's best to move. I say all of this as someone who has extensive operating experience on AWS. YMMV.
  - UK-AL 11y ago
    Oh, with cloud services you defiantly need devops. Without AWS you also need dedicated infrastructure, networking and hardware people as well as devops. You need people who know how to configure cisco networking gear, who understand SAN's, iSCSI, Fibrechannel, Racks, blades lots of stuff even devops people don't think about.
    - latch 11y ago
      No. This is true for colocated, but absolutely false for dedicated.
      - toomuchtodo 11y ago
        You are correct! If you're just colocating, you need your own people to manage your gear. Dedicated equipment is managed by the service provider. Disclaimer: Provided hosting services for ~8 years.
        UK-AL 11y ago
        Companies that provide managed hosting for hardware tend be just as expensive as cloud providers. These comparisons tend to be, oh look you can buy the the Dell Server for cheap, and shove it in data centre and it's a lot cheaper. If you start going route where you ask the provider to do it, the charges tend to be a lot more. It usually the sort thing where you need to do a phone call before you can even get a quote.
        latch 11y ago
        Again, this is demonstrably false. What's commonly meant by "dedicated hosting" is essentially the same service level you get from EC2 - they they care of the network and hardware, you take care of the software (1). The cost here is 2x-4x lower. In another comment, I quoted E5-2620 v3 w/64GB for $400 which is exactly this type of setup. This is offered by a company that's been in business for longer than AWS has existed and who, in web hosting circles, is well respected. You could definitely go cheaper. You could definitely go to IBM and rackspace and pay as much as EC2, sure...but there are literally thousands of providers in the US, which have been in business for over a decade that'll beat EC2/Rackspace/Softlayer by a wide margin. Some that I've personally used: WebNX (LA), NetDepot (Atlanta), ReliableSite (NY), HiVelocity (Florida). I've also done it on the cheap with OVH (both Quebec and France) and Hetzner (Germany), as well as on the expensive side with Softlayer (you can negotiate softlayer down considerably even on a small order). (1) That's disservice to dedicated hosting, because the quality of the network is often better, and you won't get termination emails from your hosting provider or noisy neighbours the way like you do on EC2.
- nirvdrum 11y ago
  For my last start-up, I moved off EC2 to a dedicated vSphere cluster with a hosted provider. vSphere has an API, so adapting existing provisioning & deployment code was quite straightforward (I used the fog gem in Ruby). I basically treated the vSphere cluster very similarly to EC2 (small root stores, attached volumes, etc). Granted, I did have to give up the benefit of being in different regions. I found the maintenance burden dropped substantially. It may have just been that I was running on newer hardware, but vSphere has built-in HA features such that VMs will just migrate between hosts when hardware degrades. In the two years I ran that setup, I never lost a VM. I also had dedicated, modern hardware -- no need to worry about CPU steal. I could create instances/VMs of any size I want. If I needed to resize a VM, I could do so without losing all the data on it. As long as I had capacity, I could add new VMs at no extra cost. When I needed more capacity, I'd just call up and have them add a new blade to the cluster, which both increased my redundancy and gave me extra capacity. Essentially, the cost curve, while tied to usage, was much more favorable than linear growth once you get over the base setup costs. On top of that, I had a real console to each of the VMs should anything go wrong. And if I couldn't fix something myself, there was dedicated staff at the colo facility I could just call up. It's not for everyone, but there are options out there that give you an EC2-like environment with many of the benefits of having your own hardware.
  - throwaway2016a 11y ago
    The tradeoff for CPU stealing is that now you have multiple services running on the same machine so if the machine goes down you lose them all. On AWS it is very unlikely that two of your VMs end up on the same physical machine. Also, I actually can't recall (in 8 years of AWS) a time where I saw CPU being stolen. Perhaps I've been lucky. I also don't watch for it constantly. We actually did this with our dev servers. Although switch VSphere for VirtualBox on Linux and colo for a server room in our office. It worked great and saved a TON of cost. But we don't need pesky things like 24hr uptime and dedicated bandwidth for dev servers.
    - toomuchtodo 11y ago
      > The tradeoff for CPU stealing is that now you have multiple services running on the same machine so if the machine goes down you lose them all. On AWS it is very unlikely that two of your VMs end up on the same physical machine. Yes. But its also very likely that when AWS has issues, the entire region is going to be having problems, like last Saturday (IAM, EC2, Autoscaling, etc broken badly for 6 hours). > But we don't need pesky things like 24hr uptime and dedicated bandwidth for dev servers. You're not getting this SLA unless you're managing everything yourself and are globally redundant. AWS doesn't provide bandwidth guarantees, nor will you get 100% uptime.
- deleted 11y ago
  [deleted]
- dsp1234 11y ago
  Relatively the same cost per month For us, AWS is not even remotely close in cost to a rack, hardware, and staff. The estimated spend for us to duplicate our dedicated rack setup in AWS is 3 times our monthly operational costs. And this does not include using any additional services. And to be clear, this includes operational staff, spare hardware, disaster recovery, etc.

by latch 11y ago

throwaway2016a 11y ago

Do you have any place you can point for that? Anecdotally, I have found that when we account for... - Human resources costs (payroll, taxes, benefits) [1] - Time to market for new features / products - Utilizing reserved instances where it makes sense - Appropriately sizing machines We get with AWS... - Faster time to market (accelerated revenue) - Relatively the same cost per month (cheaper in some areas more expensive in others) - Significantly lower initial investment - Increased redundancy (via many small servers vs. a few large servers) / decreased disaster recovery times (and in many cases automated recovery) I'm not saying you're wrong, that's just what I've seen when we run the numbers internally. You may have seen differently which is why I ask. [1] A devops person to manage servers dedicated or not can cost $80-$125k+ a year after benefits and taxes. That is a lot of AWS instances. And we have found we need an IT staff about half the size to manage AWS vs a dedicated data center.

karterk 11y ago
Being on AWS does not necessarily mean that you don't need a devops person - especially at the scale where not being on AWS actually makes a difference to your margins. I have seen quite a few people move off AWS successfully onto bare-metal leased hardware. S3 is just about the only service that's difficult to find an alternative for. Personally, I find using something like DynamoDB no different than using an Oracle DB - it's a vendor lock-in. Unless you had enterprise-level support on AWS (which costs a lot) - if you run into issues with Amazon's proprietary services, then good luck to you. AWS is great to get started, but once you know that you're going to need scale (and lots of infra), it's best to move. I say all of this as someone who has extensive operating experience on AWS. YMMV.
- UK-AL 11y ago
  Oh, with cloud services you defiantly need devops. Without AWS you also need dedicated infrastructure, networking and hardware people as well as devops. You need people who know how to configure cisco networking gear, who understand SAN's, iSCSI, Fibrechannel, Racks, blades lots of stuff even devops people don't think about.
  - latch 11y ago
    No. This is true for colocated, but absolutely false for dedicated.
    - toomuchtodo 11y ago
      You are correct! If you're just colocating, you need your own people to manage your gear. Dedicated equipment is managed by the service provider. Disclaimer: Provided hosting services for ~8 years.
      - UK-AL 11y ago
        Companies that provide managed hosting for hardware tend be just as expensive as cloud providers. These comparisons tend to be, oh look you can buy the the Dell Server for cheap, and shove it in data centre and it's a lot cheaper. If you start going route where you ask the provider to do it, the charges tend to be a lot more. It usually the sort thing where you need to do a phone call before you can even get a quote.
        latch 11y ago
        Again, this is demonstrably false. What's commonly meant by "dedicated hosting" is essentially the same service level you get from EC2 - they they care of the network and hardware, you take care of the software (1). The cost here is 2x-4x lower. In another comment, I quoted E5-2620 v3 w/64GB for $400 which is exactly this type of setup. This is offered by a company that's been in business for longer than AWS has existed and who, in web hosting circles, is well respected. You could definitely go cheaper. You could definitely go to IBM and rackspace and pay as much as EC2, sure...but there are literally thousands of providers in the US, which have been in business for over a decade that'll beat EC2/Rackspace/Softlayer by a wide margin. Some that I've personally used: WebNX (LA), NetDepot (Atlanta), ReliableSite (NY), HiVelocity (Florida). I've also done it on the cheap with OVH (both Quebec and France) and Hetzner (Germany), as well as on the expensive side with Softlayer (you can negotiate softlayer down considerably even on a small order). (1) That's disservice to dedicated hosting, because the quality of the network is often better, and you won't get termination emails from your hosting provider or noisy neighbours the way like you do on EC2.
nirvdrum 11y ago
For my last start-up, I moved off EC2 to a dedicated vSphere cluster with a hosted provider. vSphere has an API, so adapting existing provisioning & deployment code was quite straightforward (I used the fog gem in Ruby). I basically treated the vSphere cluster very similarly to EC2 (small root stores, attached volumes, etc). Granted, I did have to give up the benefit of being in different regions. I found the maintenance burden dropped substantially. It may have just been that I was running on newer hardware, but vSphere has built-in HA features such that VMs will just migrate between hosts when hardware degrades. In the two years I ran that setup, I never lost a VM. I also had dedicated, modern hardware -- no need to worry about CPU steal. I could create instances/VMs of any size I want. If I needed to resize a VM, I could do so without losing all the data on it. As long as I had capacity, I could add new VMs at no extra cost. When I needed more capacity, I'd just call up and have them add a new blade to the cluster, which both increased my redundancy and gave me extra capacity. Essentially, the cost curve, while tied to usage, was much more favorable than linear growth once you get over the base setup costs. On top of that, I had a real console to each of the VMs should anything go wrong. And if I couldn't fix something myself, there was dedicated staff at the colo facility I could just call up. It's not for everyone, but there are options out there that give you an EC2-like environment with many of the benefits of having your own hardware.
- throwaway2016a 11y ago
  The tradeoff for CPU stealing is that now you have multiple services running on the same machine so if the machine goes down you lose them all. On AWS it is very unlikely that two of your VMs end up on the same physical machine. Also, I actually can't recall (in 8 years of AWS) a time where I saw CPU being stolen. Perhaps I've been lucky. I also don't watch for it constantly. We actually did this with our dev servers. Although switch VSphere for VirtualBox on Linux and colo for a server room in our office. It worked great and saved a TON of cost. But we don't need pesky things like 24hr uptime and dedicated bandwidth for dev servers.
  - toomuchtodo 11y ago
    > The tradeoff for CPU stealing is that now you have multiple services running on the same machine so if the machine goes down you lose them all. On AWS it is very unlikely that two of your VMs end up on the same physical machine. Yes. But its also very likely that when AWS has issues, the entire region is going to be having problems, like last Saturday (IAM, EC2, Autoscaling, etc broken badly for 6 hours). > But we don't need pesky things like 24hr uptime and dedicated bandwidth for dev servers. You're not getting this SLA unless you're managing everything yourself and are globally redundant. AWS doesn't provide bandwidth guarantees, nor will you get 100% uptime.
deleted 11y ago
[deleted]
dsp1234 11y ago
Relatively the same cost per month For us, AWS is not even remotely close in cost to a rack, hardware, and staff. The estimated spend for us to duplicate our dedicated rack setup in AWS is 3 times our monthly operational costs. And this does not include using any additional services. And to be clear, this includes operational staff, spare hardware, disaster recovery, etc.