6 ms·

Even at a smaller scale it is a little nerve-wracking to know be so reliant on one provider. If AWS tanks there's a fair amount of code that'd need to be change

Even at a smaller scale it is a little nerve-wracking to know be so reliant on one provider. If AWS tanks there's a fair amount of code that'd need to be changed just to switch over to Azure or GCE. Failover with, e.g., email providers is easy enough, but the entire cloud stack (for lack of better terms) is a completely different ballgame.

tylersmith 9y ago
AWS provides a lot of features that are exclusive to their platform and can't be drop-in replaced on other providers like Azure of GCE. ELB, EFS, S3, ASGs, etc. They'd need to be replaced at the application level for other platforms. That could be a huge commitment for a decent sized system.
- conradk 9y ago
  I don't know about ELB, EFS and ASG but: - S3 has a public protocol and many 3rd party providers support it (OpenIO, Scality, Ceph, Minio, etc), - EFS could be replaced with something like DRDB or GlusterFS, or DigitalOcean's block storage or Google Cloud's networked disks. - ELB could be replaced easily with similar services from other providers [1] if you use Kubernetes (I don't know if all have a LoadBalancer type though) I would be more concerned about firewall/vpc rules, because I have no idea how those could be migrated without risk of forgetting some. Lock-in seems not that high in the end though and even less so if you use an open source container orchestration stack because they abstract most of these things away. [1] https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#external-load-balancer-providers https://kubernetes.io/docs/tasks/access-application-cluster/...
  - Matt3o12_ 9y ago
    One good way is to have automated tests which make sure that those rules actually work. What if all your AWS rules suddenly get deleted. How are you supposed to know if you have not forgotten any.
    - Artemis2 9y ago
      Terraform (https://www.terraform.io https://www.terraform.io), which we use, is a neat way to abstract this configuration data. We keep the configuration files in git, and can do GitHub pull requests over our whole infrastructure and apply the configuration with confidence (to change the existing setup or re-create it from scratch). This works for multiple cloud providers and is great for security purposes (all changes are auditable, no configuration drift). Heap has a great blog post on Terraform : https://heap.engineering/terraform-gotchas/ https://heap.engineering/terraform-gotchas/
      - dsandip 9y ago
        Essentially, a declarative configuration for infra is what you are getting at. You can take this further, using containers and orchestration tech, to abstract your application behind a declarative configuration making it infra agnostic (as it should be iMO). Obviously, not getting locked into any cloud provider services is a pre-requisite for this. Check out such an implementation here[1](full disclosure - I work here)) [1]: https://docs.hasura.io/0.14/ref/project-configuration-and-status/index.html https://docs.hasura.io/0.14/ref/project-configuration-and-st...
PaulKeeble 9y ago
It is one of the issues with choosing the cloud providers and taking their stack. They are hoping the cost of swapping once bought into their way is too costly to a competitor who can offer similar service cheaper. Lockin used to be considered bad but something changed with cloud providers and ops/developers don't seem to care as much anymore.
- wiz21c 9y ago
  Maybe because pricing by, say Amazon, is published on their web site and therefore, the same for everyone ? Whereas before, when you were with one supplier, he could make specific price for you and leverage its position to make you pay more ? dunno...
  - arethuza 9y ago
    I'd be very surprised if big users of AWS or Azure pay the rates advertised on the public web sites.
    - latch 9y ago
      Like most large business, AWS has a sales department and sales engineers that behave like you expect. Rates are absolutely negotiable.
      - sokoloff 9y ago
        Having negotiated both, AWS is negotiable within a range that is much tighter than the range for enterprise sales of Cisco, EMC, etc gear. (Or AWS has better negotiators, but I've never gotten a call "Hey, Qx is about to end and I need to hit my numbers, so is there anything we can pull forward" from an AWS rep.)
      - patta54 9y ago
        Our AWS rep is nice and cheery. He'll come into our office twice a year and bring sales engineers to hear about our upcoming projects. There's one lead developer on our team who keeps imagining systems that use half a dozen AWS services for "big data". The AWS dudes always end up talking to him the most and they definitely bait him with various pitches and, of course, feed his ego. Good thing that he's so disorganized and delayed that he never has a chance to waste company money on all that bullshit.
awkwarddaturtle 9y ago
It's amazing how the promise of "decentralized" internet has turned into centralized datacenters. P2P networks, each computer being a "data store" on the internet, no one entity can control data, etc to modern day centralized cloud where a couple of players control so much. There has been a cultural shift. In the early 2000s, the idea of storing your data somewhere else would have been weird. But now, people don't care about keeping their data on apple/google/etcs data centers. I think it has to do with the fact that computer/internet illiterate people are now the majority whereas in the 90s/early 2000s, it was generally the computer literate on the internet.
- kirykl 9y ago
  I was pretty befuddled when my company IT switched from self hosted storage to commercial cloud accounts for incredibly sensitive info. I think the reasoning was cloud accounts are easier for the masses than mapping a drive and accessing over VPN
  - ams6110 9y ago
    Also someone else to blame when the sensitive info is exposed.
patta54 9y ago
I warn other developers at my company about this. When new projects spin up they're often very excited about using new Amazon services and will make any excuse to choose an AWS product over a stable open source solution. If I were a manager, I'd be very worried over the vendor lock-in. I don't understand the preference for AWS over open source in many cases. Their services are "reliable", but they often have minute restrictions that will eventually bite you. You also end up having to pay for something you could get for free. Why use SNS/SQS when there are free pubsub/message buses out there? Most of the other devs justify this with the argument of not having to maintain the software themselves. "But RabbitMQ might crash! We don't have to worry about that with AWS!" Anyway, I typically minimize the AWS services I use (S3, EC2, ECS) so I don't dread the day AWS blows up or, more likely, some VP or exec says we're moving to GCP/Azure because we got a better deal.
- kevan 9y ago
  >Why use SNS/SQS when there are free pubsub/message buses out there? Free is never really free. There's always a tradeoff in engineering time and money when you choose to run your own stack instead of paying to use a stable, well-established service. Oftentimes running your own will be cheaper overall, but you have to do that cost-benefit comparison for yourself.
- plandis 9y ago
  You're also forgetting that if you set up something on your own you also have all the hardware concerns as well. You need to procure hosts, provision them properly, deploy them, monitor them, scale them, fix them. That infrastructure cost doesn't go to zero but it is significantly reduced using a cloud provider.
  - patta54 9y ago
    I'm not arguing against cloud platforms in general; just the irrational use of very specialized services they offer. I can run a containerized service that uses open source packages on any of the cloud computing platforms. Now if I used Athena, SQS/SNS, DynamoDB, ELB, Lambda, EC2 that would make me very nervous, and I see other devs designing these stacks all the time. I guess I shouldn't care as much, because I'm not going to be the one to migrate that when the company gets a better deal from another platform service.

tylersmith 9y ago

AWS provides a lot of features that are exclusive to their platform and can't be drop-in replaced on other providers like Azure of GCE. ELB, EFS, S3, ASGs, etc. They'd need to be replaced at the application level for other platforms. That could be a huge commitment for a decent sized system.

conradk 9y ago
I don't know about ELB, EFS and ASG but: - S3 has a public protocol and many 3rd party providers support it (OpenIO, Scality, Ceph, Minio, etc), - EFS could be replaced with something like DRDB or GlusterFS, or DigitalOcean's block storage or Google Cloud's networked disks. - ELB could be replaced easily with similar services from other providers [1] if you use Kubernetes (I don't know if all have a LoadBalancer type though) I would be more concerned about firewall/vpc rules, because I have no idea how those could be migrated without risk of forgetting some. Lock-in seems not that high in the end though and even less so if you use an open source container orchestration stack because they abstract most of these things away. [1] https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#external-load-balancer-providers https://kubernetes.io/docs/tasks/access-application-cluster/...
- Matt3o12_ 9y ago
  One good way is to have automated tests which make sure that those rules actually work. What if all your AWS rules suddenly get deleted. How are you supposed to know if you have not forgotten any.
  - Artemis2 9y ago
    Terraform (https://www.terraform.io https://www.terraform.io), which we use, is a neat way to abstract this configuration data. We keep the configuration files in git, and can do GitHub pull requests over our whole infrastructure and apply the configuration with confidence (to change the existing setup or re-create it from scratch). This works for multiple cloud providers and is great for security purposes (all changes are auditable, no configuration drift). Heap has a great blog post on Terraform : https://heap.engineering/terraform-gotchas/ https://heap.engineering/terraform-gotchas/
    - dsandip 9y ago
      Essentially, a declarative configuration for infra is what you are getting at. You can take this further, using containers and orchestration tech, to abstract your application behind a declarative configuration making it infra agnostic (as it should be iMO). Obviously, not getting locked into any cloud provider services is a pre-requisite for this. Check out such an implementation here[1](full disclosure - I work here)) [1]: https://docs.hasura.io/0.14/ref/project-configuration-and-status/index.html https://docs.hasura.io/0.14/ref/project-configuration-and-st...

PaulKeeble 9y ago

It is one of the issues with choosing the cloud providers and taking their stack. They are hoping the cost of swapping once bought into their way is too costly to a competitor who can offer similar service cheaper. Lockin used to be considered bad but something changed with cloud providers and ops/developers don't seem to care as much anymore.

wiz21c 9y ago
Maybe because pricing by, say Amazon, is published on their web site and therefore, the same for everyone ? Whereas before, when you were with one supplier, he could make specific price for you and leverage its position to make you pay more ? dunno...
- arethuza 9y ago
  I'd be very surprised if big users of AWS or Azure pay the rates advertised on the public web sites.
  - latch 9y ago
    Like most large business, AWS has a sales department and sales engineers that behave like you expect. Rates are absolutely negotiable.
    - sokoloff 9y ago
      Having negotiated both, AWS is negotiable within a range that is much tighter than the range for enterprise sales of Cisco, EMC, etc gear. (Or AWS has better negotiators, but I've never gotten a call "Hey, Qx is about to end and I need to hit my numbers, so is there anything we can pull forward" from an AWS rep.)
    - patta54 9y ago
      Our AWS rep is nice and cheery. He'll come into our office twice a year and bring sales engineers to hear about our upcoming projects. There's one lead developer on our team who keeps imagining systems that use half a dozen AWS services for "big data". The AWS dudes always end up talking to him the most and they definitely bait him with various pitches and, of course, feed his ego. Good thing that he's so disorganized and delayed that he never has a chance to waste company money on all that bullshit.

awkwarddaturtle 9y ago

It's amazing how the promise of "decentralized" internet has turned into centralized datacenters. P2P networks, each computer being a "data store" on the internet, no one entity can control data, etc to modern day centralized cloud where a couple of players control so much. There has been a cultural shift. In the early 2000s, the idea of storing your data somewhere else would have been weird. But now, people don't care about keeping their data on apple/google/etcs data centers. I think it has to do with the fact that computer/internet illiterate people are now the majority whereas in the 90s/early 2000s, it was generally the computer literate on the internet.

kirykl 9y ago
I was pretty befuddled when my company IT switched from self hosted storage to commercial cloud accounts for incredibly sensitive info. I think the reasoning was cloud accounts are easier for the masses than mapping a drive and accessing over VPN
- ams6110 9y ago
  Also someone else to blame when the sensitive info is exposed.

patta54 9y ago

I warn other developers at my company about this. When new projects spin up they're often very excited about using new Amazon services and will make any excuse to choose an AWS product over a stable open source solution. If I were a manager, I'd be very worried over the vendor lock-in. I don't understand the preference for AWS over open source in many cases. Their services are "reliable", but they often have minute restrictions that will eventually bite you. You also end up having to pay for something you could get for free. Why use SNS/SQS when there are free pubsub/message buses out there? Most of the other devs justify this with the argument of not having to maintain the software themselves. "But RabbitMQ might crash! We don't have to worry about that with AWS!" Anyway, I typically minimize the AWS services I use (S3, EC2, ECS) so I don't dread the day AWS blows up or, more likely, some VP or exec says we're moving to GCP/Azure because we got a better deal.

kevan 9y ago
>Why use SNS/SQS when there are free pubsub/message buses out there? Free is never really free. There's always a tradeoff in engineering time and money when you choose to run your own stack instead of paying to use a stable, well-established service. Oftentimes running your own will be cheaper overall, but you have to do that cost-benefit comparison for yourself.
plandis 9y ago
You're also forgetting that if you set up something on your own you also have all the hardware concerns as well. You need to procure hosts, provision them properly, deploy them, monitor them, scale them, fix them. That infrastructure cost doesn't go to zero but it is significantly reduced using a cloud provider.
- patta54 9y ago
  I'm not arguing against cloud platforms in general; just the irrational use of very specialized services they offer. I can run a containerized service that uses open source packages on any of the cloud computing platforms. Now if I used Athena, SQS/SNS, DynamoDB, ELB, Lambda, EC2 that would make me very nervous, and I see other devs designing these stacks all the time. I guess I shouldn't care as much, because I'm not going to be the one to migrate that when the company gets a better deal from another platform service.