7 ms·

What I really wish the cloud providers would do is reduce network egress costs. They seem insanely expensive when compared to dedicated servers.

thrownaway2424 11y ago
You're getting a fundamentally different quality of network from AWS or GCE than you get from Linode or similarly-sized dedicated provider. It's a good idea to question whether you need the network services of the large cloud providers, but it's not a very good idea to compare them as if they were apples and apples.
- iofj 11y ago
  Aside from the fact that total egress for AWS is biggger and thus (slightly) more ddos resistant ... what exactly is the quality difference here ? There's plenty of dedi providers that are big enough to prevent real outages though. And it doesn't protect you as a customer, it just protects you from other Amazon customers getting ddossed, but Amazon doesn't keep your website reachable if you're getting ddossed (no dedi providers do).
  - thrownaway2424 11y ago
    AWS has at least 23 points of presence and Google has at least 70. I think you are seriously underestimating the size of Amazon's or Google's networks.
    - iofj 11y ago
      That doesn't protect you as much as you think. Ddos resiliency really depends on the amount of unused (!) available capacity, in absolute value. And Amazon, I guarantee you, doesn't have that much unused network capacity. That would be very, very expensive at their scale.
      - cherioo 11y ago
        A lot of the time ddos also depends on number of locations, because typically only one customer is attacked, and typically that one customer is only in a few location. Also, I'm not sure I buy the argument Amazon has small unused capacity in absolute value, relative maybe. They could also have 5% across 23 locations so if your application is distributed it can have even better resiliency.
      - thrownaway2424 11y ago
        Again, I think you are _dramatically_ underestimating the size of these networks.
      - biot 11y ago
        Amazon AWS is 10 times bigger than the next 14 competitors combined. [0] Amazon's unused network capacity is likely far larger than any other single provider's total available capacity, possibly with the exception of Azure. If it's expensive for Amazon to have multiple Tbps of unused capacity, imagine how expensive it is for any other provider. To match the absolute spare bandwidth of Amazon having only 1/10 extra network capacity, another cloud provider might need to have keep its network utilization at only 5%. Maintaining a network capable of serving over 20x your current utilization "just in case of DDoS" is bloody expensive. [0] http://www.businessinsider.com/nobody-gets-fired-for-buying-amazon-2015-5?op=1 http://www.businessinsider.com/nobody-gets-fired-for-buying-...
        vgt 11y ago
        Google's SDN is seriously massive. Petabit of bisectional bandwidth: http://googlecloudplatform.blogspot.com/2015/06/A-Look-Inside-Googles-Data-Center-Networks.html http://googlecloudplatform.blogspot.com/2015/06/A-Look-Insid... One major difference between Google and AWS is that Google will carry your packets between data centers on its backbone by default. Google will also carry packets as close to the customer as possible, whereas AWS will dump it off as quickly as possible. So, even between Google Cloud and AWS it's not an apples to apples comparison.
        11y ago
fpgaminer 11y ago
Agreed. I've heard some people say that those prices are kept high to encourage keeping data locked within the platform; forcing people to use the provider's services versus shipping data off to other platforms. I'm not sure if I believe that or not, but it's an interesting theory. Honestly, the high bandwidth costs have killed a number of ideas I would have liked to tinker with on AWS. I find the cost of EC2 to be quite good, and storage cost is tolerable, but $.10/GB of bandwidth-out is horrific.
- Vendan 11y ago
  Seriously, I pay $20 for a VPS with good stats and 3TB of outbound bandwidth per month. It's with Linode, who have been apparently hard hit, but I've suffered about 10 minutes of downtime over the course of the last 2 years, and all of that was in the past week, when the DDOS crap started. Since then it's been fine.
  - noir_lord 11y ago
    Apart from the breach and global password resets today. http://www.zdnet.com/article/cloud-firm-linode-resets-user-passwords-after-suspected-hack/ http://www.zdnet.com/article/cloud-firm-linode-resets-user-p... I have been a big fan of linode in the past, I still have a fair amount with them but honestly at this point I'm seriously considering AWS even at the increased cost (which compared to my time isn't that large really and reputational cost which is).
    - meowface 11y ago
      This is the third time they've had a critical breach in the past 5 or so years, and that's excluding all the DDoS downtime they've had in that time span. I stopped using them after their second breach, and frankly am glad I did. I'm very happy with Digital Ocean.
      - jonathanoliver 11y ago
        I just wish DO hardware has a bit faster (or not so over-subscribed?). All of our tests show Linode beat DO hands down for at similar price points.
PaulHoule 11y ago
It hasn't bothered me. I quit Softlayer because every time I sent a change order in something awful would happen. At one point the issue tracking system broke in a way that I could not put in new tickets and got to talk to four different people until I talked to a wizard who punched a few commands into the SQL monitor and told me he saw something "amusing". I had a "near miss" at data loss because one of their techs botched adding another hard drive to the machine, plus I was dealing with an expensive and balky backup system so I immediately moved my data into S3 then all the servers into EC2. Softlayer had a crack sales guy call me to try to get me back and I told him I had a day job and a night job and I don't have time to talk to minions to fix the problems they make for me. He brought up the egress cost issue and I told him flatly that "I make $1000 a month in ad revenue and I pay $30 in egress charges so I don't care." My business situation has changed in many ways since then but I'd say that my egress charges tend to run between 5-10% of my total spend so it is not a concern for me. If there is any AWS service that I don't like the value of it is RDS and I am mostly off it since I have been using SSD-backed instances and first running local copies of MySQL and then ditched MySQL.
- click170 11y ago
  Can I ask when you moved off of soft layer? I ask because my employer is considering them as an option and this isn't the first time I've heard softlayer horror stories.
  - johnnycarcin 11y ago
    Not the OP but we have had a hell of a time with SoftLayer and have been doing everything we can to move. If you have a large ops team (or a team that isn't busy) and you go with a well planned physical server build-out (make sure you get them in different pods) you _might_ be ok. We have been working on a POC in AWS and it's such a breath of fresh air. Things work as advertised, the provisioning process is quick and everything can easily be done with an API call. No more waiting for support tickets. The freedom you get with your network routing in an AWS VPC is worth it to me. At the risk of sounding too much like an AWS fanboy we are actually looking at scrapping it all and just bringing everything in-house running on our own physical boxes with some type of hypervisor on top.
mwcampbell 11y ago
For me, the worthwhile comparison isn't between cloud and dedicated, but between AWS/Google/Azure on the one hand and DigitalOcean/Linode/Vultr on the other. The former charge for all outgoing transfer by the GB, while the latter include generous amounts of transfer per month with each VM plan.
mattbillenstein 11y ago
App on AWS, data on S3, Varnish caches on Linode, Digital Ocean, etc. Route53 for managing geo-DNS.
- fideloper 11y ago
  Not sure I get the logic of Varnish being on different providers. Are your static assets also hosted on Linode/DO, in addition to Varnish?
  - toomuchtodo 11y ago
    Varnish caches dynamic app data. Your static assets should be in a CDN, served outside of your infrastructure.
    - fideloper 11y ago
      So to put what I'm cofused about plainly (my fault on the confusion I'm sure :D ): It sounds like the reason for having that outside of your primary infrastructure (or more accurately, inside a cheaper bandwidth host) are lower bandwidth costs with the trade-off of some slower requests getting sent to your origin server when the cached resource expires/is invalidated.
      - toomuchtodo 11y ago
        Exactly! You're architecting your infrastructure to take advantage of the most efficient cost option at each layer.
    - mattbillenstein 11y ago
      So mainly thinking of Varnish as the CDN -- paying a 3rd party CDN is pricey.
simonebrunozzi 11y ago
Why should they? That's how they lock you in.
twistedpair 11y ago
Jeff's got to make money somewhere. Remember, ingress is free and unlimited. This has made business models like DropCam viable where essentially the ISP and AWS pay for all the ingress, and DropCam only has to pay for the processing and storage.

thrownaway2424 11y ago

You're getting a fundamentally different quality of network from AWS or GCE than you get from Linode or similarly-sized dedicated provider. It's a good idea to question whether you need the network services of the large cloud providers, but it's not a very good idea to compare them as if they were apples and apples.

iofj 11y ago
Aside from the fact that total egress for AWS is biggger and thus (slightly) more ddos resistant ... what exactly is the quality difference here ? There's plenty of dedi providers that are big enough to prevent real outages though. And it doesn't protect you as a customer, it just protects you from other Amazon customers getting ddossed, but Amazon doesn't keep your website reachable if you're getting ddossed (no dedi providers do).
- thrownaway2424 11y ago
  AWS has at least 23 points of presence and Google has at least 70. I think you are seriously underestimating the size of Amazon's or Google's networks.
  - iofj 11y ago
    That doesn't protect you as much as you think. Ddos resiliency really depends on the amount of unused (!) available capacity, in absolute value. And Amazon, I guarantee you, doesn't have that much unused network capacity. That would be very, very expensive at their scale.
    - cherioo 11y ago
      A lot of the time ddos also depends on number of locations, because typically only one customer is attacked, and typically that one customer is only in a few location. Also, I'm not sure I buy the argument Amazon has small unused capacity in absolute value, relative maybe. They could also have 5% across 23 locations so if your application is distributed it can have even better resiliency.
    - thrownaway2424 11y ago
      Again, I think you are _dramatically_ underestimating the size of these networks.
    - biot 11y ago
      Amazon AWS is 10 times bigger than the next 14 competitors combined. [0] Amazon's unused network capacity is likely far larger than any other single provider's total available capacity, possibly with the exception of Azure. If it's expensive for Amazon to have multiple Tbps of unused capacity, imagine how expensive it is for any other provider. To match the absolute spare bandwidth of Amazon having only 1/10 extra network capacity, another cloud provider might need to have keep its network utilization at only 5%. Maintaining a network capable of serving over 20x your current utilization "just in case of DDoS" is bloody expensive. [0] http://www.businessinsider.com/nobody-gets-fired-for-buying-amazon-2015-5?op=1 http://www.businessinsider.com/nobody-gets-fired-for-buying-...
      - vgt 11y ago
        Google's SDN is seriously massive. Petabit of bisectional bandwidth: http://googlecloudplatform.blogspot.com/2015/06/A-Look-Inside-Googles-Data-Center-Networks.html http://googlecloudplatform.blogspot.com/2015/06/A-Look-Insid... One major difference between Google and AWS is that Google will carry your packets between data centers on its backbone by default. Google will also carry packets as close to the customer as possible, whereas AWS will dump it off as quickly as possible. So, even between Google Cloud and AWS it's not an apples to apples comparison.
        11y ago

fpgaminer 11y ago

Agreed. I've heard some people say that those prices are kept high to encourage keeping data locked within the platform; forcing people to use the provider's services versus shipping data off to other platforms. I'm not sure if I believe that or not, but it's an interesting theory. Honestly, the high bandwidth costs have killed a number of ideas I would have liked to tinker with on AWS. I find the cost of EC2 to be quite good, and storage cost is tolerable, but $.10/GB of bandwidth-out is horrific.

Vendan 11y ago
Seriously, I pay $20 for a VPS with good stats and 3TB of outbound bandwidth per month. It's with Linode, who have been apparently hard hit, but I've suffered about 10 minutes of downtime over the course of the last 2 years, and all of that was in the past week, when the DDOS crap started. Since then it's been fine.
- noir_lord 11y ago
  Apart from the breach and global password resets today. http://www.zdnet.com/article/cloud-firm-linode-resets-user-passwords-after-suspected-hack/ http://www.zdnet.com/article/cloud-firm-linode-resets-user-p... I have been a big fan of linode in the past, I still have a fair amount with them but honestly at this point I'm seriously considering AWS even at the increased cost (which compared to my time isn't that large really and reputational cost which is).
  - meowface 11y ago
    This is the third time they've had a critical breach in the past 5 or so years, and that's excluding all the DDoS downtime they've had in that time span. I stopped using them after their second breach, and frankly am glad I did. I'm very happy with Digital Ocean.
    - jonathanoliver 11y ago
      I just wish DO hardware has a bit faster (or not so over-subscribed?). All of our tests show Linode beat DO hands down for at similar price points.

PaulHoule 11y ago

It hasn't bothered me. I quit Softlayer because every time I sent a change order in something awful would happen. At one point the issue tracking system broke in a way that I could not put in new tickets and got to talk to four different people until I talked to a wizard who punched a few commands into the SQL monitor and told me he saw something "amusing". I had a "near miss" at data loss because one of their techs botched adding another hard drive to the machine, plus I was dealing with an expensive and balky backup system so I immediately moved my data into S3 then all the servers into EC2. Softlayer had a crack sales guy call me to try to get me back and I told him I had a day job and a night job and I don't have time to talk to minions to fix the problems they make for me. He brought up the egress cost issue and I told him flatly that "I make $1000 a month in ad revenue and I pay $30 in egress charges so I don't care." My business situation has changed in many ways since then but I'd say that my egress charges tend to run between 5-10% of my total spend so it is not a concern for me. If there is any AWS service that I don't like the value of it is RDS and I am mostly off it since I have been using SSD-backed instances and first running local copies of MySQL and then ditched MySQL.

click170 11y ago
Can I ask when you moved off of soft layer? I ask because my employer is considering them as an option and this isn't the first time I've heard softlayer horror stories.
- johnnycarcin 11y ago
  Not the OP but we have had a hell of a time with SoftLayer and have been doing everything we can to move. If you have a large ops team (or a team that isn't busy) and you go with a well planned physical server build-out (make sure you get them in different pods) you _might_ be ok. We have been working on a POC in AWS and it's such a breath of fresh air. Things work as advertised, the provisioning process is quick and everything can easily be done with an API call. No more waiting for support tickets. The freedom you get with your network routing in an AWS VPC is worth it to me. At the risk of sounding too much like an AWS fanboy we are actually looking at scrapping it all and just bringing everything in-house running on our own physical boxes with some type of hypervisor on top.

mwcampbell 11y ago

For me, the worthwhile comparison isn't between cloud and dedicated, but between AWS/Google/Azure on the one hand and DigitalOcean/Linode/Vultr on the other. The former charge for all outgoing transfer by the GB, while the latter include generous amounts of transfer per month with each VM plan.

mattbillenstein 11y ago

App on AWS, data on S3, Varnish caches on Linode, Digital Ocean, etc. Route53 for managing geo-DNS.

fideloper 11y ago
Not sure I get the logic of Varnish being on different providers. Are your static assets also hosted on Linode/DO, in addition to Varnish?
- toomuchtodo 11y ago
  Varnish caches dynamic app data. Your static assets should be in a CDN, served outside of your infrastructure.
  - fideloper 11y ago
    So to put what I'm cofused about plainly (my fault on the confusion I'm sure :D ): It sounds like the reason for having that outside of your primary infrastructure (or more accurately, inside a cheaper bandwidth host) are lower bandwidth costs with the trade-off of some slower requests getting sent to your origin server when the cached resource expires/is invalidated.
    - toomuchtodo 11y ago
      Exactly! You're architecting your infrastructure to take advantage of the most efficient cost option at each layer.
  - mattbillenstein 11y ago
    So mainly thinking of Varnish as the CDN -- paying a 3rd party CDN is pricey.

simonebrunozzi 11y ago

Why should they? That's how they lock you in.

twistedpair 11y ago

Jeff's got to make money somewhere. Remember, ingress is free and unlimited. This has made business models like DropCam viable where essentially the ISP and AWS pay for all the ingress, and DropCam only has to pay for the processing and storage.