11 ms·

Choosing EC2 and other Amazon offerings means a lot of freedom for startups, it's easy to just create a whole infrastructure at the push of a button. However, t

by Udo 13y ago

Choosing EC2 and other Amazon offerings means a lot of freedom for startups, it's easy to just create a whole infrastructure at the push of a button. However, this comes at a high cost. Amazon is really expensive and once you're locked into their way of doing things, migrating away can be a near-impossible task. I think for some years, companies didn't realize how high the cost of running on EC2/S3 actually is compared to, say, a dedicated rack colocated somewhere - that's kind of a compliment to Amazon tech and marketing. It's still not uncommon to talk to a random startup person and they'll tell you they have hosting costs in the tens of thousands of dollars per year. When you then compare this to what they're actually doing with their servers, it seems to me they're overpaying by several orders of magnitude. I was using EC2 for years and kind of grew to accept (and ignore) its limitations. It was only when I reviewed some dedicated server plans at other providers I (re-)discovered how much raw power you can get elsewhere for the same money. Additionally, I became very frustrated with the lack of pricing transparency and inability to forecast costs - at first glance the Amazon cloud is as open as it gets but the real surprises come when you're running it for a while. No doubt Amazon has caught on to this trend and is fighting to squeeze out some competitors that have sprung up. This pricing change is most likely directly aimed at Hetzner and other inexpensive-but-powerful hosting providers. It means they were either massively overcharging the whole time or they are using profits from other plans to offer Dedicated EC2 at a loss until the competition folds.

Fizzer 13y ago
Are you comparing EC2's hourly rate with dedicated hardware? If so, you're comparing apples and oranges. With EC2, the hourly rate is outrageous for an always-on server. You really need to buy reserved instances for it to be cost-effective. In my case I found I really needed to buy the 3-year heavy utilization reserved instances. The hourly rate is best just for occasional spikes or when you're experimenting with different hardware configurations. And yes, they were massively overcharging for dedicated instances, as they charged $7300 per month just for the privilege of being able to use dedicated instances (that doesn't include the cost of the instance itself.) I always assumed they were doing so just to reserve that feature for their big customers.
- latch 13y ago
  This is a common misrepresentation. Given a unit, reserved instances might be price competitive, but that unit won't be performance competitive. Even with reserved instances, you're talking about 2-4x worse price/performance at the minimum (much, much worse if SSDs are involved). Not to mention the upfront cost... Using reserved instances is the exact opposite of what anyone should be doing. The only way to get price competitive with EC2 is to go the absolute other direction: establish a baseline with dedicated/colo and use spot instances for tasks or to handle peaks.
  - vacri 13y ago
    Speaking of misrepresentation, 'Not to mention the upfront cost'? What are you talking about? All the pricing proposals by AWS include clear references to it, and amortise the cost over the life of the plan. It's not a hidden cost. It's not an 'extra' cost on top of 'the monthly payment you thought you were making'.
    - latch 13y ago
      I'm not, in any way, suggesting that the pricing structure isn't 100% transparent. I'm pointing out that there's an upfront component (say $1614 for a m3.2xlarge) to the reserved instance pricing - which is likely of interest to startups. Amazon calls this an "upfront payment"...I don't think "upfront cost" is a misrepresentation. This cost tends to be better than buying hardware, but it's much higher than what you pay for dedicated (often $0 upfront...or $25-$200).
      - sciurus 13y ago
        "I'm pointing out that there's an upfront component ... which is likely of interest to startups." Yes, this. When you're worried about cash flow, overpaying every month can seem safer than paying the upfront cost to reserve instances.
- vidarh 13y ago
  Of course he is comparing apples and oranges, yet it is how a substantial number of people actually use AWS. Dedicated instances goes some way towards mitigating it, but for a lot of people the short term billing makes AWS look cheap, and their dedicated servers were definitively still not cheap prior to this even just looking at the cost of the instances. A lot of people are completely blinkered when it comes to the cost of using AWS.
vidarh 13y ago
I actually was at one point pushed hard by the CEO at a company I worked at to use EC2 because it "had to" be cheap, despite the fact I had put together numbers for him that clearly demonstrated it was between twice as expensive (purely managed servers with enough headroom to handle short term spikes) to three times expensive (managed servers loaded at 80%-90% of typical daily peak loads plus set up to use EC2 to handle peaks in emergencies). Amazon has done an amazing job in marketing and PR for AWS. The latter is what EC2 is most useful for for me: You can cut costs of your dedicated hosting by going much closer to the wire with the knowledge you can spin up instances at EC2 (and/or other cloud providers) if you should happen to hit the front page somewhere that can drive massive traffic faster / on more transient basis than you can or want to scale up your dedicated setup. Hetzner is great, but fully integrated providers like Softlayer is another growing threat - many providers now offer a sliding scale of some or all from colo, to managed bare metal, to "dedicated virtual" (single VM per physical server, automatically deployed in minutes or hours as a "halfway house" to cloud setups), to cloud images billed similarly to EC2. It'll be interesting to see what IBM tries to do with Softlayer...
_pmf_ 13y ago
> Choosing EC2 and other Amazon offerings Do you have any data on how pure S3 storage is priced relative to other solutions?
twistedpair 13y ago
>When you then compare this to what they're actually doing with their servers, it seems to me they're overpaying by several orders of magnitude. So, you can get that $10,000/yr dedicated server for $10/yr elsewhere? What host is that. I need to check them out.
Bamafan 13y ago
I think for some years, companies didn't realize how high the cost of running on EC2/S3 actually is compared to, say, a dedicated rack colocated somewhere... But this ignores the cost of paying someone to setup/maintain that rack, cost to fix the server when something goes wrong, etc. Setup/maintenance of physical infrastructure has a cost too.
- bfung 13y ago
  +1 A lot of conversation is about the cost of acquisition, but does not compare the prices of maintenance, hardware failure/outages, and recovery. For example, with dedicated hardware and a guessed failure rate, how much does it cost, over time, to 1) replace the hardware, software, and configuration and 2) value lost (could be goodwill, could be $$$) due to outage. With AWS using chef/puppet/salt/etc., it's trivial to boot up new nodes and down the malfunctioning ones. The value of that is super high.
  - Sanddancer 13y ago
    You're acting like you can't do the same thing with dedicated hardware. On the post-install side, chef/puppet/salt/etc work just as well. On the setup/build side, dhcp, pxe, and the hands-free install setups built into every unix distro let you spin up and down machines with minimal hassle. Need a new db machine? Throw the MAC address of the new box in the db pool of your dhcp server and tell it to boot. Need to move that machine into the web server pool? Move that MAC into your web server pool and tell the box to reboot. Combine that with a mixture of hot spares, cold spares, and warm spares, and I'd still bet you'd be cheaper than AWS for any significant sized service.
cstejerean 13y ago
After all the talk about how much cheaper dedicated hosting can be compared to AWS, I looked into dedicated hosting recently. It didn't seem as great as the comments here led me to believe. Rackspace for example starts at $499/month for a server with 4GB of RAM. On AWS I can get a c1.xlarge instance with 7GB of RAM for $435/month, and that's with paying the on-demand price 24x7. Ok, that doesn't include EBS and bandwidth. So let's throw in 1TB of outbound data (less than the 2TB that Rackspace includes, but still more than I'd ever use in a month) and a 150GB EBS volume with 1000 provision IOPS. Let's also use the 1 year heavy utilization price while we're at it. Now I'm looking $342/month with $1800 upfront, which amortizes to $492/month. With AWS I also don't need to stick with paying 24x7 for infrastructure. I don't need to keep my CI infrastructure running on nights and weekends for example, and with AWS I have the flexibility to turn that into cost savings. So I don't see the order of magnitude price difference. So maybe I shouldn't be looking at Rackspace? Hetzner keeps coming up on here, but I don't need a server in Europe. Maybe OVH? Hmm, the comments on Hacker News don't exactly inspire confidence in OVH for something more than a personal server to screw around with. I need to run infrastructure for my business. Do I start hunting for bargain hosting providers on WebHostingTalk? Ok, back to AWS it is for me. EDIT: Thanks for the suggestions to look at Softlayer. A server with 1 CPU and 8GB of RAM starts at $399, which is better than Rackspace, but not exactly orders of magnitude cheaper than the c1.xlarge I mentioned above.
- asb 13y ago
  Looking from Rackspace to OVH is really going straight from one extreme to another. Softlayer is a more sensible middle-ground (though note their list prices are high, you'll almost always be able to get 2x RAM or similar).
  - cstejerean 13y ago
    Thank you for the recommendation, it's definitely a better deal than Rackspace for my needs, but not compelling enough for me to switch from AWS at the moment.
- Udo 13y ago
  I think you picked the most expensive dedicated hosting provider in existence. Have a look at SoftLayer, Hetzner, Incero, or one of the other alternatives. Also, at these prices, it's not unreasonable to just order any old Dell server for $600 and put it into a colo rack somewhere. For reference, the dedicated server I rent for my private projects comes in at €49/month and it has these specs: Intel® Core™ i7-2600 Quad-Core 16 GB DDR 3 RAM 2 x 3 TB SATA 6 Gb/s HDD; 7200 rpm Support is extremely fast and competent, I'm really happy. To try and mirror that package with any kind of Amazon (or Rackspace it seems) offering would be prohibitively costly.
ancarda 13y ago
Recommended reading if your considering co-locating instead of AWS: http://www.codinghorror.com/blog/2012/10/building-servers-for-fun-and-prof-ok-maybe-just-for-fun.html http://www.codinghorror.com/blog/2012/10/building-servers-fo... -- It compares prices (now somewhat out of date) vs performance.
- druiid 13y ago
  I'd say it's vastly out of date now. You can squeeze a LOT into new Dell/HP 1U/2U servers. Their R620 systems can take nearly a terabyte (768GB) of memory per server and the R720 can do the same, but with the 720XD it can do 24 drives in a single 2U space. Top that off with the newer Intel processors and you have a huge amount of hardware available to you in not a very large amount of space.
dasil003 13y ago
There is a tremendous value add from the whole AWS package, so naturally you are going to overpay a lot if you compare on raw power alone. AWS allows you to not purchase backup hardware, it allows you to shape your stack to your load profile on demand rather than purchasing hardware and potentially making a long-term commitment that is wrong, it allows you to handle load spikes very easily and inexpensively, and it allows you control everything from a robust API, and it's big and stable enough where you don't usually have to fight for and be disappointed by support: problems tend to be large and affecting everyone, or small and you just boot a new instance. So what if you're spending $25k a year where a colo'd rack would be only $5k, that's not your major cost center in a startup (or any business for that matter). If you are utilizing the benefits of AWS it could well tip the scales in terms of not needing to have a $150k systems engineer.
x0x0 13y ago
from a former employer who wants to stay anonymous, ec2 bill per month: $97k peak, softlayer: $26k. 1/3 the cost, 2x the performance. And let's not mention emr; a $15k/mo cluster in softlayer did something like 5x the performance of an emr cluster that was costing (and running!) $2k/day.
tchock23 13y ago
On your point about lack of pricing transparency and inability to forecast costs with AWS, check out CloudCheckr. It helped us get much better visibility into our AWS costs. I still feel like I'm overpaying with AWS, but at least I have better visibility into where and how I'm overpaying...
verelo 13y ago
10k, 50k, heck even 100k a year should be nothing to your startup. That's the cost of one good developer. Focus your cost savings in other areas, if 100k is the kind of optimization you need to make (and you're actually spending 100k a year on virtual hardware) then you have other problems.
- arjie 13y ago
  One good developer adds one good developer worth of value. Overpaying 100k is definitely a bad idea because you aren't getting the 100k of value.
  - verelo 13y ago
    Its a fair point, but i suspect that moving 100k of infrastructure from AWS to a colo is going to add at least 1 dev-ops / sys-admin (or sucks in 1 developers time). I'm skeptical that the cost benefit comes in until well above 100k / yr.
toble 13y ago
Agreed. I have been running dedicated servers for years and stayed well away from Amazon due to price calculations, it's such a huge difference in price. When I worked at start-up the lead dev seemed to be heavily bought into the hype of "cloud" hosting, so he used Engine Yard. The founders got freaked out at the cost for what was negligible traffic. We then moved to Amazon, which equally freaked them out. If you don't have money to burn, then consider going down the standard hosting route.

by Udo 13y ago

Fizzer 13y ago

Are you comparing EC2's hourly rate with dedicated hardware? If so, you're comparing apples and oranges. With EC2, the hourly rate is outrageous for an always-on server. You really need to buy reserved instances for it to be cost-effective. In my case I found I really needed to buy the 3-year heavy utilization reserved instances. The hourly rate is best just for occasional spikes or when you're experimenting with different hardware configurations. And yes, they were massively overcharging for dedicated instances, as they charged $7300 per month just for the privilege of being able to use dedicated instances (that doesn't include the cost of the instance itself.) I always assumed they were doing so just to reserve that feature for their big customers.

latch 13y ago
This is a common misrepresentation. Given a unit, reserved instances might be price competitive, but that unit won't be performance competitive. Even with reserved instances, you're talking about 2-4x worse price/performance at the minimum (much, much worse if SSDs are involved). Not to mention the upfront cost... Using reserved instances is the exact opposite of what anyone should be doing. The only way to get price competitive with EC2 is to go the absolute other direction: establish a baseline with dedicated/colo and use spot instances for tasks or to handle peaks.
- vacri 13y ago
  Speaking of misrepresentation, 'Not to mention the upfront cost'? What are you talking about? All the pricing proposals by AWS include clear references to it, and amortise the cost over the life of the plan. It's not a hidden cost. It's not an 'extra' cost on top of 'the monthly payment you thought you were making'.
  - latch 13y ago
    I'm not, in any way, suggesting that the pricing structure isn't 100% transparent. I'm pointing out that there's an upfront component (say $1614 for a m3.2xlarge) to the reserved instance pricing - which is likely of interest to startups. Amazon calls this an "upfront payment"...I don't think "upfront cost" is a misrepresentation. This cost tends to be better than buying hardware, but it's much higher than what you pay for dedicated (often $0 upfront...or $25-$200).
    - sciurus 13y ago
      "I'm pointing out that there's an upfront component ... which is likely of interest to startups." Yes, this. When you're worried about cash flow, overpaying every month can seem safer than paying the upfront cost to reserve instances.
vidarh 13y ago
Of course he is comparing apples and oranges, yet it is how a substantial number of people actually use AWS. Dedicated instances goes some way towards mitigating it, but for a lot of people the short term billing makes AWS look cheap, and their dedicated servers were definitively still not cheap prior to this even just looking at the cost of the instances. A lot of people are completely blinkered when it comes to the cost of using AWS.

vidarh 13y ago

I actually was at one point pushed hard by the CEO at a company I worked at to use EC2 because it "had to" be cheap, despite the fact I had put together numbers for him that clearly demonstrated it was between twice as expensive (purely managed servers with enough headroom to handle short term spikes) to three times expensive (managed servers loaded at 80%-90% of typical daily peak loads plus set up to use EC2 to handle peaks in emergencies). Amazon has done an amazing job in marketing and PR for AWS. The latter is what EC2 is most useful for for me: You can cut costs of your dedicated hosting by going much closer to the wire with the knowledge you can spin up instances at EC2 (and/or other cloud providers) if you should happen to hit the front page somewhere that can drive massive traffic faster / on more transient basis than you can or want to scale up your dedicated setup. Hetzner is great, but fully integrated providers like Softlayer is another growing threat - many providers now offer a sliding scale of some or all from colo, to managed bare metal, to "dedicated virtual" (single VM per physical server, automatically deployed in minutes or hours as a "halfway house" to cloud setups), to cloud images billed similarly to EC2. It'll be interesting to see what IBM tries to do with Softlayer...

_pmf_ 13y ago

> Choosing EC2 and other Amazon offerings Do you have any data on how pure S3 storage is priced relative to other solutions?

twistedpair 13y ago

>When you then compare this to what they're actually doing with their servers, it seems to me they're overpaying by several orders of magnitude. So, you can get that $10,000/yr dedicated server for $10/yr elsewhere? What host is that. I need to check them out.

Bamafan 13y ago

I think for some years, companies didn't realize how high the cost of running on EC2/S3 actually is compared to, say, a dedicated rack colocated somewhere... But this ignores the cost of paying someone to setup/maintain that rack, cost to fix the server when something goes wrong, etc. Setup/maintenance of physical infrastructure has a cost too.

bfung 13y ago
+1 A lot of conversation is about the cost of acquisition, but does not compare the prices of maintenance, hardware failure/outages, and recovery. For example, with dedicated hardware and a guessed failure rate, how much does it cost, over time, to 1) replace the hardware, software, and configuration and 2) value lost (could be goodwill, could be $$$) due to outage. With AWS using chef/puppet/salt/etc., it's trivial to boot up new nodes and down the malfunctioning ones. The value of that is super high.
- Sanddancer 13y ago
  You're acting like you can't do the same thing with dedicated hardware. On the post-install side, chef/puppet/salt/etc work just as well. On the setup/build side, dhcp, pxe, and the hands-free install setups built into every unix distro let you spin up and down machines with minimal hassle. Need a new db machine? Throw the MAC address of the new box in the db pool of your dhcp server and tell it to boot. Need to move that machine into the web server pool? Move that MAC into your web server pool and tell the box to reboot. Combine that with a mixture of hot spares, cold spares, and warm spares, and I'd still bet you'd be cheaper than AWS for any significant sized service.

cstejerean 13y ago

After all the talk about how much cheaper dedicated hosting can be compared to AWS, I looked into dedicated hosting recently. It didn't seem as great as the comments here led me to believe. Rackspace for example starts at $499/month for a server with 4GB of RAM. On AWS I can get a c1.xlarge instance with 7GB of RAM for $435/month, and that's with paying the on-demand price 24x7. Ok, that doesn't include EBS and bandwidth. So let's throw in 1TB of outbound data (less than the 2TB that Rackspace includes, but still more than I'd ever use in a month) and a 150GB EBS volume with 1000 provision IOPS. Let's also use the 1 year heavy utilization price while we're at it. Now I'm looking $342/month with $1800 upfront, which amortizes to $492/month. With AWS I also don't need to stick with paying 24x7 for infrastructure. I don't need to keep my CI infrastructure running on nights and weekends for example, and with AWS I have the flexibility to turn that into cost savings. So I don't see the order of magnitude price difference. So maybe I shouldn't be looking at Rackspace? Hetzner keeps coming up on here, but I don't need a server in Europe. Maybe OVH? Hmm, the comments on Hacker News don't exactly inspire confidence in OVH for something more than a personal server to screw around with. I need to run infrastructure for my business. Do I start hunting for bargain hosting providers on WebHostingTalk? Ok, back to AWS it is for me. EDIT: Thanks for the suggestions to look at Softlayer. A server with 1 CPU and 8GB of RAM starts at $399, which is better than Rackspace, but not exactly orders of magnitude cheaper than the c1.xlarge I mentioned above.

asb 13y ago
Looking from Rackspace to OVH is really going straight from one extreme to another. Softlayer is a more sensible middle-ground (though note their list prices are high, you'll almost always be able to get 2x RAM or similar).
- cstejerean 13y ago
  Thank you for the recommendation, it's definitely a better deal than Rackspace for my needs, but not compelling enough for me to switch from AWS at the moment.
Udo 13y ago
I think you picked the most expensive dedicated hosting provider in existence. Have a look at SoftLayer, Hetzner, Incero, or one of the other alternatives. Also, at these prices, it's not unreasonable to just order any old Dell server for $600 and put it into a colo rack somewhere. For reference, the dedicated server I rent for my private projects comes in at €49/month and it has these specs: Intel® Core™ i7-2600 Quad-Core 16 GB DDR 3 RAM 2 x 3 TB SATA 6 Gb/s HDD; 7200 rpm Support is extremely fast and competent, I'm really happy. To try and mirror that package with any kind of Amazon (or Rackspace it seems) offering would be prohibitively costly.

ancarda 13y ago

Recommended reading if your considering co-locating instead of AWS: http://www.codinghorror.com/blog/2012/10/building-servers-for-fun-and-prof-ok-maybe-just-for-fun.html http://www.codinghorror.com/blog/2012/10/building-servers-fo... -- It compares prices (now somewhat out of date) vs performance.

druiid 13y ago
I'd say it's vastly out of date now. You can squeeze a LOT into new Dell/HP 1U/2U servers. Their R620 systems can take nearly a terabyte (768GB) of memory per server and the R720 can do the same, but with the 720XD it can do 24 drives in a single 2U space. Top that off with the newer Intel processors and you have a huge amount of hardware available to you in not a very large amount of space.

dasil003 13y ago

There is a tremendous value add from the whole AWS package, so naturally you are going to overpay a lot if you compare on raw power alone. AWS allows you to not purchase backup hardware, it allows you to shape your stack to your load profile on demand rather than purchasing hardware and potentially making a long-term commitment that is wrong, it allows you to handle load spikes very easily and inexpensively, and it allows you control everything from a robust API, and it's big and stable enough where you don't usually have to fight for and be disappointed by support: problems tend to be large and affecting everyone, or small and you just boot a new instance. So what if you're spending $25k a year where a colo'd rack would be only $5k, that's not your major cost center in a startup (or any business for that matter). If you are utilizing the benefits of AWS it could well tip the scales in terms of not needing to have a $150k systems engineer.

x0x0 13y ago

from a former employer who wants to stay anonymous, ec2 bill per month: $97k peak, softlayer: $26k. 1/3 the cost, 2x the performance. And let's not mention emr; a $15k/mo cluster in softlayer did something like 5x the performance of an emr cluster that was costing (and running!) $2k/day.

tchock23 13y ago

On your point about lack of pricing transparency and inability to forecast costs with AWS, check out CloudCheckr. It helped us get much better visibility into our AWS costs. I still feel like I'm overpaying with AWS, but at least I have better visibility into where and how I'm overpaying...

verelo 13y ago

10k, 50k, heck even 100k a year should be nothing to your startup. That's the cost of one good developer. Focus your cost savings in other areas, if 100k is the kind of optimization you need to make (and you're actually spending 100k a year on virtual hardware) then you have other problems.

arjie 13y ago
One good developer adds one good developer worth of value. Overpaying 100k is definitely a bad idea because you aren't getting the 100k of value.
- verelo 13y ago
  Its a fair point, but i suspect that moving 100k of infrastructure from AWS to a colo is going to add at least 1 dev-ops / sys-admin (or sucks in 1 developers time). I'm skeptical that the cost benefit comes in until well above 100k / yr.

toble 13y ago

Agreed. I have been running dedicated servers for years and stayed well away from Amazon due to price calculations, it's such a huge difference in price. When I worked at start-up the lead dev seemed to be heavily bought into the hype of "cloud" hosting, so he used Engine Yard. The founders got freaked out at the cost for what was negligible traffic. We then moved to Amazon, which equally freaked them out. If you don't have money to burn, then consider going down the standard hosting route.