4 ms·

The Economics of AWS

41 points by phsr 17y ago

DanBlake 17y ago
Using EC2 for anything but overflow is silly. The costs are extraordinarily high if you are doing decent volume ( read- if you have more than 3-5 servers ) Bandwidth is ridiculously expensive with them. You can get 3-4x cheaper per megabit going dedicated. Servers are crazy expensive. Compare the most powerful machine they have vs something on 10tb / gigenet / theplanet for the price. You will definitely end up with a more powerful machine for half the price on either. The only real advantage to using ec2 is the hourly billing. Make its perfect for overflow, but thats about it. I read about alot of startups that use ec2 for things like webservers or other servers that have 100% reliance. Dont get me wrong. Some of Amazons offerings are great. The CDS and S3 can take the bullshit out of dealing with the complexitys of each and might save you a network engineer or 2. However, for EC2 there is no excuse. Its no more difficult to setup a normal dedicated server than it is a EC2 box. /endrant
- teej 17y ago
  You seem to have missed a pretty big feature of EC2 - servers on demand. There is no thought or overhead in spinning up a new server. 10 seconds in your console of choice and you have a new virtual server, ready to go. That, combined with instance customization, mean server instantiation can be done by a developer on her lunch break, instead of taking hours and attention away from the ops guy.
  - DanBlake 17y ago
    You can justify how important that is to you I suppose. Any decent dedicated provider (eg- theplanet.com ) Will email you your server details 5 minutes after you order the box, unless you are getting some weird equipment or off the bend OS. The bottom line is you are paying 2, sometimes 3 times as much for that convenience of "instantly" powering it on. How often do you that? How often do most users do that? If your business is at all like most, Your need to power on a new server instantly isnt mission critical. Even if it is, You should have your main servers running OFF of ec2. By all means, use ec2 or any other VPS to handle your overflow.
    - using_ec2 17y ago
      I think that you are missing the value of the opposite proposition - that you are able to turn it off and stop paying. This is an obvious value to someone who needs to be able to scale sideways, but having essentially no cost for provisioning and de-provisioning allows you to do things for development, testing and operations you cannot do without great expense outside of it. I worked at a number of banks 10 years ago and we can get the same functionality now for a few bucks that we did then by buying far too many spare E3500s. We have a decent sized EC2 architecture right now and EBS snapshotting + provisioning on demand means that we can get all of the benefits of multiply redundant hardware at a fraction of the cost. - We never need to upgrade software in place. Software gets freshly installed on a new cluster that duplicates our existing cluster and gets a increasing portion of traffic. Failover is as simple as a proxy cutover. The freedom to do config changes or 3rd party software upgrades and be able to just swap instances in and out enables us to experiment often and safely with production changes. - Slave gets corrupted or falls out of sync with the master? A machine suffers a degraded disk? Bad net interface? Don't fix, replace. - We can spin up an entire test stack that precisely mirrors our prod stack to stress test with significant cash outlay. Since this cost is low, we can stress test often and test the effect of s/w and config changes on performance easily. Stress tests the H/W outlays associated with them are traditionally hugely expensive if you want to test exactly like you run. - Significant database migrations can take place on a slave that is easily promoted, snapshotted and cloned without having to deal with any long term ping ponging between instances. - Copies of databases can be instantly provisioned and destroyed for data analysis needs. If analysis needs peak at 5x in a week, no one has to wait. Run 5 copies and shut them all off when you're done. These are just a few quick things that we do with regularity. And I'm sure there are lots more opportunities to exploit these capabilities than we are thinking of. Also - in terms of cost - don't forget to compare against reserved instance pricing.
      - DanBlake 17y ago
        I did state in the very first post that hourly billing is the only asset. Thats why its perfect for overflow. I can spin up a double extra large and use it for a few hours when I need it, and kill it afterwards. Thats it though. I use r1soft for redundancy and the costs for that are nominal. That sort of handles any hardware issues for me. I can get back up and running in about 10 minutes from a catastrophic failure. All the stuff you mention below really doesnt matter to me, and I cant imagine it matters that much to many people. All these tests you mention could easily be done off site, or mirrored to run locally. Still not worth paying 3-4x for these random "abilitys" For things that matter- Bandwidth, Processing Power, Ram- Dedicated servers have EC2 Beat, hands down. More power, Less cost.
- blasdel 17y ago
  It's actually harder to set up your software on an EC2 instance, but just the first time -- it demands a pretty high level of sophistication. OTOH, do you have API keys for your Dell salesman and your co-location facilities? I'd love to see those provisioning scripts!
- gfodor 17y ago
  I agree it's more expensive, but I think you are undervaluing the instant provisioning capability EC2 gives you and overestimating the impact of the increased cost. The way I look at it is if all these services were free, EC2 would win based upon features. So, the question is, does cost matter? For us, a single engineer's salary costs more than our entire yearly EC2 bill. And, we're not even using cost saving measures like elasticity and reserved instances. And, EC2 actually drives costs down in ways hosting does not. First, as you mentioned, it dramatically reduces the need for IT staff. (Though it doesn't eliminate it, of course.) Second, and IMHO, more importantly, the constraints EC2 imposes on you forces you to build fault tolerant, shared-nothing systems. I'm sure many folks here have worked at companies with hosted servers to find machines that are chock full of random cron jobs, services, and so on. The instant provisioning capability of EC2 combined with the constraints makes it very hard for you to have these one off "god boxes" your system is hinging on. Beyond that, as things get too messy, you have the freedom to automate and shut-down and bring up fresh nodes instantly with no increase in cost. It's hard to me to estimate the amount of man-hours we've saved and disasters we've avoided due to this capability. It's allowed us to cleanly "refactor" our entire cluster over time since we can reprovision nodes on demand as we improve the services or architecture on those nodes.
  - DanBlake 17y ago
    I dont buy that ec2 relieves the employee need. Its just as difficult, if not MORE difficult to setup a ec2 instance than buying a dedicated server. S3, SDB, CDS, etc.. Yes, those make things easier. EC2 Does not.
    - teej 17y ago
      "Its just as difficult, if not MORE difficult to setup a ec2 instance than buying a dedicated server." Is this from experience? If so, please share it.
      - DanBlake 17y ago
        Sure. I use EC2 to handle our overflow during peak hours. I had to literally look up guides to dealing with it. Its not exactly straight forward how the IP system works, and figuring out all the ec2 specific crap, images and storing the image in S3, etc. etc.
- javery 17y ago
  I moved up from a small instance to a medium instance on a server that I never thought I would need more power on, it took me less than 5 minutes to make the switch. (unattach the IP and EBS and attach to the new instance) This is why I pay for EC2. My dedicated boxes are on a yearly contract and when it comes to move it's a couple hours to move data, IP addresses, etc.
  - DanBlake 17y ago
    The amount you pay for a "small" instance on EC2 is the same price as a machine you can get with gigenet that is more powerful than the "Large" instance. Hence the major problem with ec2- Cost. Yes, you can instantly scale up to a less powerful server than you would have got in the first place from a dedicated provider. Also, You do know you can do things like upgrade the processor and ram on a dedicated, usually within a hour also. You just send in a ticket. What server provider makes you do a contract?
    - javery 17y ago
      The contract isn't required, but I use it to get the best deal. And if you do the reserved instance the EC2 instances get pretty cheap. Upgrading hardware involves more downtime, and it's not anywhere near as easy to scale back down if I need to.
      - DanBlake 17y ago
        If you are signing contracts to get a deal, it sounds like you are price conscious. You should see in comments above where I spec out the needs for a typical website. However, Assuming you only need 1.2ghz of single core processing power and nobody needs to visit your website, ec2 is perfect. It only starts sucking when you actually have to serve files and do it quickly.
        javery 17y ago
        I haven't had a problem with speed yet, but then again I use CloudFront and don't serve files directly from the server. For me its worth the money for the flexibility and the ability to scale as needed. I might be able to save a little money if I went with blended dedicated/EC2 but honestly the extra headache and latency for cross-communication wouldn't be worth the hassle.
- moe 17y ago
  Oh well, the old cost argument again. As a satisfied EC2 customer [who also happens to be fully able to use a calculator] I'm happy to take these apart, again and again. Bandwidth is ridiculously expensive with them. You can get 3-4x cheaper per megabit going dedicated. Yes. But that does not normally matter. Bandwidth has become ridiculously cheap either way; a terabyte sets you back a measly hundred bucks on amazon. And if you're pushing more than a few dozen of them then most of it is very likely going over a CDN (or a set of separate dedicated servers) anyways. Remember sites pushing 2-digit terabytes of dynamic html are very rare. And everything else can be trivially offloaded (e.g. to one of the isp's you advocate). Servers are crazy expensive. Compare the most powerful machine they have vs something on 10tb / gigenet / theplanet for the price. You will definitely end up with a more powerful machine for half the price on either. Well, I did and came back underwhelmed - as usual. Yes, you can go cheaper for a comparable spec, sometimes as much as 50 bucks. But again - that only matters when you're a small shop with 1-3 servers, looking to cut the penny. Once your business grows out of that the hosting fees turn into a rounding error either way, and the advantages EC2 offers are nothing to sneeze at (ever looked what an solution equal to EBS costs elsewhere?). Its no more difficult to setup a normal dedicated server than it is a EC2 box. That's a red herring. Provisioning is automated in any reasonable sized deployment and you face the exact same problems with dedicated hardware that you have with amazon. Plus the hassles of having to babysit said hardware... There is exactly one argument against amazon today and that is when you need hosts with lots of memory. RAM is expensive in the cloud, for obvious reasons - to a degree where stuffing your own hardware with 64G starts to look like a bargain. But the thing about this argument is: Very few applications really need that amount of RAM in a single box. In most cases the root cause for such a requirement is simply a flaw in your architecture.
  - dnsworks 17y ago
    Yes. But that does not normally matter. Bandwidth has become ridiculously cheap either way; a terabyte sets you back a measly hundred bucks on amazon. And if you're pushing more than a few dozen of them then most of it is very likely going over a CDN (or a set of separate dedicated servers) anyways. That's not really a safe assumption. I work with several real-world, profitable sites whose monthly bandwidth usage range from 50Mbps to 2Gbps at 95th percentile, and they push enough custom content that CDNs are overly complex and still not priced competitively. Their respective business models would not be viable if they ran on Amazon. I just helped one of them acquire a pair of GigE uplinks, one of which they're paying $1,500/month for the other they're paying $3k/month for. Assuming they can see 800Mbps out of each link, that's $7 and $14/TB of xfe. This compared to Amazon's "measly $100" ..
senthil_rajasek 17y ago
I agree that the ec2 is more expensive but... What are your alternatives?
- DanBlake 17y ago
  Dedicated, Colo, Other VPS's if you must. Shop around.
  - senthil_rajasek 17y ago
    bringing up a new ec2 instance takes about .5 secs
    - DanBlake 17y ago
      Well, you cant beat that logic. Spend away sheeple.
      - jon_dahl 17y ago
        You're coming on a little strong, man. I appreciate your basic point, but you're calling people sheep because they weigh the pros and cons of cloud hosting differently than you.
  - deleted 17y ago
    [deleted]
siverson914 17y ago
Storage pricing is getting more competitive to Diomede Storage, but Diomede is still more than 50% less expensive, even for very large installations.

41 points by phsr 17y ago

Using EC2 for anything but overflow is silly. The costs are extraordinarily high if you are doing decent volume ( read- if you have more than 3-5 servers ) Bandwidth is ridiculously expensive with them. You can get 3-4x cheaper per megabit going dedicated. Servers are crazy expensive. Compare the most powerful machine they have vs something on 10tb / gigenet / theplanet for the price. You will definitely end up with a more powerful machine for half the price on either. The only real advantage to using ec2 is the hourly billing. Make its perfect for overflow, but thats about it. I read about alot of startups that use ec2 for things like webservers or other servers that have 100% reliance. Dont get me wrong. Some of Amazons offerings are great. The CDS and S3 can take the bullshit out of dealing with the complexitys of each and might save you a network engineer or 2. However, for EC2 there is no excuse. Its no more difficult to setup a normal dedicated server than it is a EC2 box. /endrant

teej 17y ago
You seem to have missed a pretty big feature of EC2 - servers on demand. There is no thought or overhead in spinning up a new server. 10 seconds in your console of choice and you have a new virtual server, ready to go. That, combined with instance customization, mean server instantiation can be done by a developer on her lunch break, instead of taking hours and attention away from the ops guy.
- DanBlake 17y ago
  You can justify how important that is to you I suppose. Any decent dedicated provider (eg- theplanet.com ) Will email you your server details 5 minutes after you order the box, unless you are getting some weird equipment or off the bend OS. The bottom line is you are paying 2, sometimes 3 times as much for that convenience of "instantly" powering it on. How often do you that? How often do most users do that? If your business is at all like most, Your need to power on a new server instantly isnt mission critical. Even if it is, You should have your main servers running OFF of ec2. By all means, use ec2 or any other VPS to handle your overflow.
  - using_ec2 17y ago
    I think that you are missing the value of the opposite proposition - that you are able to turn it off and stop paying. This is an obvious value to someone who needs to be able to scale sideways, but having essentially no cost for provisioning and de-provisioning allows you to do things for development, testing and operations you cannot do without great expense outside of it. I worked at a number of banks 10 years ago and we can get the same functionality now for a few bucks that we did then by buying far too many spare E3500s. We have a decent sized EC2 architecture right now and EBS snapshotting + provisioning on demand means that we can get all of the benefits of multiply redundant hardware at a fraction of the cost. - We never need to upgrade software in place. Software gets freshly installed on a new cluster that duplicates our existing cluster and gets a increasing portion of traffic. Failover is as simple as a proxy cutover. The freedom to do config changes or 3rd party software upgrades and be able to just swap instances in and out enables us to experiment often and safely with production changes. - Slave gets corrupted or falls out of sync with the master? A machine suffers a degraded disk? Bad net interface? Don't fix, replace. - We can spin up an entire test stack that precisely mirrors our prod stack to stress test with significant cash outlay. Since this cost is low, we can stress test often and test the effect of s/w and config changes on performance easily. Stress tests the H/W outlays associated with them are traditionally hugely expensive if you want to test exactly like you run. - Significant database migrations can take place on a slave that is easily promoted, snapshotted and cloned without having to deal with any long term ping ponging between instances. - Copies of databases can be instantly provisioned and destroyed for data analysis needs. If analysis needs peak at 5x in a week, no one has to wait. Run 5 copies and shut them all off when you're done. These are just a few quick things that we do with regularity. And I'm sure there are lots more opportunities to exploit these capabilities than we are thinking of. Also - in terms of cost - don't forget to compare against reserved instance pricing.
    - DanBlake 17y ago
      I did state in the very first post that hourly billing is the only asset. Thats why its perfect for overflow. I can spin up a double extra large and use it for a few hours when I need it, and kill it afterwards. Thats it though. I use r1soft for redundancy and the costs for that are nominal. That sort of handles any hardware issues for me. I can get back up and running in about 10 minutes from a catastrophic failure. All the stuff you mention below really doesnt matter to me, and I cant imagine it matters that much to many people. All these tests you mention could easily be done off site, or mirrored to run locally. Still not worth paying 3-4x for these random "abilitys" For things that matter- Bandwidth, Processing Power, Ram- Dedicated servers have EC2 Beat, hands down. More power, Less cost.
blasdel 17y ago
It's actually harder to set up your software on an EC2 instance, but just the first time -- it demands a pretty high level of sophistication. OTOH, do you have API keys for your Dell salesman and your co-location facilities? I'd love to see those provisioning scripts!
gfodor 17y ago
I agree it's more expensive, but I think you are undervaluing the instant provisioning capability EC2 gives you and overestimating the impact of the increased cost. The way I look at it is if all these services were free, EC2 would win based upon features. So, the question is, does cost matter? For us, a single engineer's salary costs more than our entire yearly EC2 bill. And, we're not even using cost saving measures like elasticity and reserved instances. And, EC2 actually drives costs down in ways hosting does not. First, as you mentioned, it dramatically reduces the need for IT staff. (Though it doesn't eliminate it, of course.) Second, and IMHO, more importantly, the constraints EC2 imposes on you forces you to build fault tolerant, shared-nothing systems. I'm sure many folks here have worked at companies with hosted servers to find machines that are chock full of random cron jobs, services, and so on. The instant provisioning capability of EC2 combined with the constraints makes it very hard for you to have these one off "god boxes" your system is hinging on. Beyond that, as things get too messy, you have the freedom to automate and shut-down and bring up fresh nodes instantly with no increase in cost. It's hard to me to estimate the amount of man-hours we've saved and disasters we've avoided due to this capability. It's allowed us to cleanly "refactor" our entire cluster over time since we can reprovision nodes on demand as we improve the services or architecture on those nodes.
- DanBlake 17y ago
  I dont buy that ec2 relieves the employee need. Its just as difficult, if not MORE difficult to setup a ec2 instance than buying a dedicated server. S3, SDB, CDS, etc.. Yes, those make things easier. EC2 Does not.
  - teej 17y ago
    "Its just as difficult, if not MORE difficult to setup a ec2 instance than buying a dedicated server." Is this from experience? If so, please share it.
    - DanBlake 17y ago
      Sure. I use EC2 to handle our overflow during peak hours. I had to literally look up guides to dealing with it. Its not exactly straight forward how the IP system works, and figuring out all the ec2 specific crap, images and storing the image in S3, etc. etc.
javery 17y ago
I moved up from a small instance to a medium instance on a server that I never thought I would need more power on, it took me less than 5 minutes to make the switch. (unattach the IP and EBS and attach to the new instance) This is why I pay for EC2. My dedicated boxes are on a yearly contract and when it comes to move it's a couple hours to move data, IP addresses, etc.
- DanBlake 17y ago
  The amount you pay for a "small" instance on EC2 is the same price as a machine you can get with gigenet that is more powerful than the "Large" instance. Hence the major problem with ec2- Cost. Yes, you can instantly scale up to a less powerful server than you would have got in the first place from a dedicated provider. Also, You do know you can do things like upgrade the processor and ram on a dedicated, usually within a hour also. You just send in a ticket. What server provider makes you do a contract?
  - javery 17y ago
    The contract isn't required, but I use it to get the best deal. And if you do the reserved instance the EC2 instances get pretty cheap. Upgrading hardware involves more downtime, and it's not anywhere near as easy to scale back down if I need to.
    - DanBlake 17y ago
      If you are signing contracts to get a deal, it sounds like you are price conscious. You should see in comments above where I spec out the needs for a typical website. However, Assuming you only need 1.2ghz of single core processing power and nobody needs to visit your website, ec2 is perfect. It only starts sucking when you actually have to serve files and do it quickly.
      - javery 17y ago
        I haven't had a problem with speed yet, but then again I use CloudFront and don't serve files directly from the server. For me its worth the money for the flexibility and the ability to scale as needed. I might be able to save a little money if I went with blended dedicated/EC2 but honestly the extra headache and latency for cross-communication wouldn't be worth the hassle.
moe 17y ago
Oh well, the old cost argument again. As a satisfied EC2 customer [who also happens to be fully able to use a calculator] I'm happy to take these apart, again and again. Bandwidth is ridiculously expensive with them. You can get 3-4x cheaper per megabit going dedicated. Yes. But that does not normally matter. Bandwidth has become ridiculously cheap either way; a terabyte sets you back a measly hundred bucks on amazon. And if you're pushing more than a few dozen of them then most of it is very likely going over a CDN (or a set of separate dedicated servers) anyways. Remember sites pushing 2-digit terabytes of dynamic html are very rare. And everything else can be trivially offloaded (e.g. to one of the isp's you advocate). Servers are crazy expensive. Compare the most powerful machine they have vs something on 10tb / gigenet / theplanet for the price. You will definitely end up with a more powerful machine for half the price on either. Well, I did and came back underwhelmed - as usual. Yes, you can go cheaper for a comparable spec, sometimes as much as 50 bucks. But again - that only matters when you're a small shop with 1-3 servers, looking to cut the penny. Once your business grows out of that the hosting fees turn into a rounding error either way, and the advantages EC2 offers are nothing to sneeze at (ever looked what an solution equal to EBS costs elsewhere?). Its no more difficult to setup a normal dedicated server than it is a EC2 box. That's a red herring. Provisioning is automated in any reasonable sized deployment and you face the exact same problems with dedicated hardware that you have with amazon. Plus the hassles of having to babysit said hardware... There is exactly one argument against amazon today and that is when you need hosts with lots of memory. RAM is expensive in the cloud, for obvious reasons - to a degree where stuffing your own hardware with 64G starts to look like a bargain. But the thing about this argument is: Very few applications really need that amount of RAM in a single box. In most cases the root cause for such a requirement is simply a flaw in your architecture.
- dnsworks 17y ago
  Yes. But that does not normally matter. Bandwidth has become ridiculously cheap either way; a terabyte sets you back a measly hundred bucks on amazon. And if you're pushing more than a few dozen of them then most of it is very likely going over a CDN (or a set of separate dedicated servers) anyways. That's not really a safe assumption. I work with several real-world, profitable sites whose monthly bandwidth usage range from 50Mbps to 2Gbps at 95th percentile, and they push enough custom content that CDNs are overly complex and still not priced competitively. Their respective business models would not be viable if they ran on Amazon. I just helped one of them acquire a pair of GigE uplinks, one of which they're paying $1,500/month for the other they're paying $3k/month for. Assuming they can see 800Mbps out of each link, that's $7 and $14/TB of xfe. This compared to Amazon's "measly $100" ..

senthil_rajasek 17y ago

I agree that the ec2 is more expensive but... What are your alternatives?

DanBlake 17y ago
Dedicated, Colo, Other VPS's if you must. Shop around.
- senthil_rajasek 17y ago
  bringing up a new ec2 instance takes about .5 secs
  - DanBlake 17y ago
    Well, you cant beat that logic. Spend away sheeple.
    - jon_dahl 17y ago
      You're coming on a little strong, man. I appreciate your basic point, but you're calling people sheep because they weigh the pros and cons of cloud hosting differently than you.
- deleted 17y ago
  [deleted]

siverson914 17y ago

Storage pricing is getting more competitive to Diomede Storage, but Diomede is still more than 50% less expensive, even for very large installations.