11 ms·

This article is very hand-wavy (I agree with some of the comments here), but as a heavy AWS user here are things that bother me 1. Networking -> I completely a

This article is very hand-wavy (I agree with some of the comments here), but as a heavy AWS user here are things that bother me 1. Networking -> I completely agree with the OP here, The networking on AWS needs to be better. I don't want the strongest machine just to have a better transfer rate. It makes complete sense to have a micro machine for some services, but if those services are accessed or access other HTTP/s services, it will be unnecessarily slow 2. Pricing -> I have the privilege of working at a company that can afford to pay 3 year in advance. Even if you do that though Amazon will keep you on the machine types you purchased and not on the newest parallel machine types. In which case if you paid 3 year in advance you are often "stuck" on previous generation instance types. 3. Someone mentioned CloudFormation here. I completely ditched it in favor of terraform, CloudFormation looks like a tool from the 90s after using Terraform (which itself isn't clear of flaws as well) 4. In terms of VPC and networking I completely disagree with the OP, the networking and security settings on Amazon are great (if you understand them). You can define instances that have absolutely no access to/from the outside world. If you build a secure service, some of your services living in a "sandbox" makes total sense 5. App Engine -> The amount of complaints I heard about this service over the last couple of years are just insane. I heard from multiple users of the platform that it sucks. While I have absolutely zero experience with it myself, I tend to listen to people that suffer from it daily.

scrollaway 10y ago
App Engine is a joke. It was an interesting and novel idea at the time but it is not something you want to pick up today (or last year, or the year before that, or the one before that either). GAE is a massive lock-in with outdated software in a world where there's so many better, cheaper, non-lock-in alternatives, even within Google itself.
- anysz 10y ago
  Care to share these alternatives ?
  - scrollaway 10y ago
    Um, AWS? :) Within Google, either of the GCEs (https://cloud.google.com/compute/ https://cloud.google.com/compute/ https://cloud.google.com/container-engine/ https://cloud.google.com/container-engine/) depending on your architecture.
    - hehheh 10y ago
      Those aren't really alternatives to GAE IMO unless you would also say that a laptop is an alternative. A real GAE alternative would be a hosted solution that doesn't require you to install an OS or even know what OS it's using.
      - scrollaway 10y ago
        Such alternatives would have the same massive downsides as GAE (outdated software by design, lock-in by design). I'm not saying "Don't use GAE because it's expensive/google/whatever". I'm saying "Don't use GAE or anything like it".
      - NetStrikeForce 10y ago
        I think both AWS and Azure have pretty strong PaaS services where you just deploy your [nodejs|python|php] app. Heroku also comes to mind...
        jacques_chester 10y ago
        AWS has Elastic Beanstalk. Azure is still finding their own PaaS pathway. Cloud Foundry gives you the "just push" experience for both of these (as well as vSphere, OpenStack, GCP and more to come). It's open source, with the IP owned by an independent foundation. Disclosure: I work for Pivotal, we donate the majority of engineering on Cloud Foundry.
      - 010a 10y ago
        GKE/Kubernetes does fill that requirement; its not as simple as GAE, but its one step closer to bare metal.
- themihai 10y ago
  I can't agree more!
- dragonwriter 10y ago
  > App Engine is a joke. It was an interesting and novel idea at the time but it is not something you want to pick up today [...] in a world where there's so many better, cheaper, non-lock-in alternatives, even within Google itself. As I see it, at the time App Engine was introduced, there were a lot more comparable PaaS offerings (maybe not with the same languages support, but the style of PaaS that GAE is was more common.) Competition from GAE and Heroku (which originally was focused on a similar PaaS offering) seems to have driven most of the other alternatives to pivot to something else or fail entirely. Right now, there's very few close substitutes; there's lots of "alternatives" that require more infrastructure work (e.g., IaaS and similar offerings like GCE, GKE, EC2, etc.) or are off in the other direction ("severless" function hosting like Lambda or GCF.) And in some cases these might be better options than an GAE-style PaaS, but they aren't really direct substitutes, and they leave a space better served by GAE.
- bduerst 10y ago
  Wait, how? I've used it before for one-off web projects, but didn't think it was that bad.
  - macNchz 10y ago
    Having inherited a large legacy codebase running on App Engine and powering a successful business, I see the biggest danger with App Engine as being the lock-in. We're on App Engine and we're not going anywhere, because it'd be too big of a project to move somewhere else. There's one open source project I know of that lets you run an App Engine project on another server (Appscale), but it's hard to know how useable it would be without the overhead of getting everything set up, and surely there are plenty of dark corners you'd encounter along the way. Since App Engine comes with a proprietary data store, ORM and task queue built in, skipping the Appscale compatibility thing and just porting a large application to run on another platform would be a herculean effort. The data store models and everywhere in the application code that they're queried would need to be re-written for a different ORM, all of the data would have to be migrated to a different store, and all the background tasks and methods that trigger them would need to be rebuilt. This would be thousands and thousands of lines of hand-edited code changes, before even thinking about how much time would need to be spent verifying all of the changes. At least with Heroku the magic is mostly around deploying and scaling the various pieces, rather than providing a lot of application level libraries that lock you into the platform. Porting to another platform likely would require a bunch of configuration management and some deployment scripts, but fairly limited changes to the application code itself.
    - jacques_chester 10y ago
      The easiest path off Heroku to self-hosted is probably Cloud Foundry: one of the inspirations was Heroku and several of the Cloud Foundry buildpacks are soft forks of Heroku's buildpacks. Generally we find that any buildpack written for Heroku will work with Cloud Foundry without modification. With a little extra engineering you can create a buildpack that will run in a fully disconnected environment. Disclosure: I work on the Cloud Foundry Buildpacks team on behalf of Pivotal.
    - 10y ago
- robryk 10y ago
  > (...) non-lock-in alternatives (...) There is an opensource reimplementation of App Engine's interface. That said, I have no clue about its completeness or buglessness. https://github.com/AppScale/appscale https://github.com/AppScale/appscale
- boulos 10y ago
  Do you feel the same way about App Engine Flexible Environment (neé Managed VMs)? That is, is your complaint that GAE Standard offers you all these services (like Memcache, Task Queues, Datastore) or that you hate the sandbox it's in? The GAE team has been (admittedly slowly) pushing the services that were GAE only into being fully-consumable "Cloud Platform" services (e.g., Cloud Datastore is the same Datastore that's been "part of" App Engine forever, but now accessible from anywhere). It's your choice to use Datastore, and I respect that you would chose to avoid it, but the basic PaaS idea of "here's some code, run it for me as a web endpoint" is still compelling even today. Disclosure: I work on Google Cloud.
  - scrollaway 10y ago
    My impression of GAE is that on top of the core issues of the platform's (lock in by design, outdated by design), it is also very much unloved by Google and has received very few updates. I stayed away from it so I could be wrong. But to answer your question, I don't know anything about that Amazon service but I would feel the same way about similarly-designed competing product. IMHO, no informed person would ever pick proprietary, lock-in-heavy PAAS solutions.
    - boulos 10y ago
      But picking at that a bit, if I were to run a simple off-the-shelf Django setup against MySQL, I'd use GAE (if not Standard than certainly Flexible). My code wouldn't be any different than deploying it to a raw VM, so I'm not locked in and could deploy it on DO, AWS or anywhere else in a heartbeat. In the meantime, I don't have to deal with setting up load balancing, autoscaling, DNS per module, etc. so I get some real value from that. Is your concern the lock-in from the code standpoint, or operational "lock-in" that once you've got all this set up for you, you'd have to go replicate it to get out? If the latter, aren't you saying you're going to do that on Day 1 in the "avoid PaaS" case?
      - scrollaway 10y ago
        Why would you use GAE in this scenario, over, say, GCE or even AWS? And at a smaller scale, DO would be far cheaper.
jfindley 10y ago
Agree WRT network performance, the basic instances are unreasonably slow (only 70 Mbps for a micro), and requiring users to pay for a big 8 core server for wirespeed Gbe, and a huge 40 core beast for 10Gbe (and rather poor 10gbe at that) is totally unreasonable, and borders on price gouging (if such a term can be used for this market sector). The cross-AZ network performance is terrible much of the time, the jitter and loss are unpredictable and often far higher than you'd expect and generally trying to run high performance network services on AWS is deeply unpleasant in multiple directions at once. WRT to VPC and security groups, I wonder if the OP might be talking about some of the surprising and frustrating limitations with them. E.g. VPCs are limited to fairly small subnet blocks, within which you have virtually no control. You can specify simple static routes - but only for subnets outside your VPC block. If you want to run a routing protocol in your VPC (e.g. you want your docker containers to be reachable via BGP), you get stuck pretty quickly - if the BGP subnet is within the VPC block, you can't route to them within the VPC, as the router won't permit more specific routes within the BGP block. If it's not within the VPC block, you won't be able to get to it via VPNs, peering connections, etc. So you're stuck either way. Security groups are also limited - the number of rules you can have is pretty small, and they are a weird mix of stateless and stateful - they are stateful if the src/dest is an individual IP or group, but stateless for 0.0.0.0/0. And As you can't match TCP states like related or established, properly securing services when you have rules including 0.0.0.0/0 can be surprisingly difficult to achieve. These are but a few examples of the limitations of the ways in which networking and security are implemented on AWS - there are plenty plenty more.
- vacri 10y ago
  > Security groups are also limited - the number of rules you can have is pretty small You can have multiple security groups now, though. Previously you were limited to just the one.
eeeeeeeeeeeee 10y ago
The argument in the original article against VPC sounded more like "it's confusing, so it's not worthwhile." VPC is confusing if you don't understand basic networking, but most people can pick it up, and people coming from a physical datacenter, moving to the cloud, will get it almost immediately.
uptown 10y ago
Can you use VPC to create an environment that blocks outside access to Internet users completely, but connects to a primary hosting environment hosted elsewhere so that systems operating in both locations can communicate freely without concern for the AWS environment being exploited to access your primary server farm?
- vacri 10y ago
  Yes, you can have VPN endpoints to your VPC, where the other end is in whatever external network you want. And if you don't want any external communication at all (barring your VPN), just remove the 'IGW' from your VPC (or make a new VPC without one). Or modify the VPC's routing tables. Or don't assign any public IPs to anything in the VPC. Or probably a few other methods :)
- NetStrikeForce 10y ago
  This is one of the main reasons we created Wormhole Network[1]. You can block all incoming traffic to your VPC and still be able to freely communicate with your resources in other locations. The good thing is that you can easily add and remove devices / users to your network. We require an agent to be deployed, which is SoftEther's VPN client (free, open source, known VPN client). [1] https://wormhole.network https://wormhole.network
vacri 10y ago
> looks like a tool from the 90s I see this criticism a lot for various things... and the things in question never look like the tools and/or websites I saw in the 90s. Instead, they look like things we'd dream of having in the 90s.
user5994461 10y ago
4. In terms of VPC and networking. I am definitely not satisfied with AWS. - On GCE the security groups are created automatically and managed by the "role" of instances. On AWS there is no link between instances and security groups. I currently have to emulate the working of GCE over AWS with extremely complex ansible scripts just so that an instance called "repository" can actually be assigned to a group called "repository". - ELB (load balancer on AWS) cannot have a fixed IP (we've been waiting for that for YEARS). An ELB can only be accessed over a DNS name, and the underlying IP can change every 60 second. (Have fun with applications which are caching DNS). By comparison, GCE has had load balancers with fixed IPs forever. - AWS is a region nightmare. Many resources only exist in one region and cannot be accessed or even acknowledged to exist from another region. e.g. an AMI only exists in a single region. It's not possible to create a host with it in another region. It's not even possible to take that existing AMI and push it to another region. (As far as a region is concerned, other regions don't exist). All services on AWS are acutely region centric. I am currently expanding my infra to multiple regions and there are many obstacles to overcome. The networking and interconnections is definitely one of them. By comparison Google is not region centric like that. It's a lot easier to manage at planet scale. --- Just three major pain points on the top of my head.

scrollaway 10y ago

App Engine is a joke. It was an interesting and novel idea at the time but it is not something you want to pick up today (or last year, or the year before that, or the one before that either). GAE is a massive lock-in with outdated software in a world where there's so many better, cheaper, non-lock-in alternatives, even within Google itself.

anysz 10y ago
Care to share these alternatives ?
- scrollaway 10y ago
  Um, AWS? :) Within Google, either of the GCEs (https://cloud.google.com/compute/ https://cloud.google.com/compute/ https://cloud.google.com/container-engine/ https://cloud.google.com/container-engine/) depending on your architecture.
  - hehheh 10y ago
    Those aren't really alternatives to GAE IMO unless you would also say that a laptop is an alternative. A real GAE alternative would be a hosted solution that doesn't require you to install an OS or even know what OS it's using.
    - scrollaway 10y ago
      Such alternatives would have the same massive downsides as GAE (outdated software by design, lock-in by design). I'm not saying "Don't use GAE because it's expensive/google/whatever". I'm saying "Don't use GAE or anything like it".
    - NetStrikeForce 10y ago
      I think both AWS and Azure have pretty strong PaaS services where you just deploy your [nodejs|python|php] app. Heroku also comes to mind...
      - jacques_chester 10y ago
        AWS has Elastic Beanstalk. Azure is still finding their own PaaS pathway. Cloud Foundry gives you the "just push" experience for both of these (as well as vSphere, OpenStack, GCP and more to come). It's open source, with the IP owned by an independent foundation. Disclosure: I work for Pivotal, we donate the majority of engineering on Cloud Foundry.
    - 010a 10y ago
      GKE/Kubernetes does fill that requirement; its not as simple as GAE, but its one step closer to bare metal.
themihai 10y ago
I can't agree more!
dragonwriter 10y ago
> App Engine is a joke. It was an interesting and novel idea at the time but it is not something you want to pick up today [...] in a world where there's so many better, cheaper, non-lock-in alternatives, even within Google itself. As I see it, at the time App Engine was introduced, there were a lot more comparable PaaS offerings (maybe not with the same languages support, but the style of PaaS that GAE is was more common.) Competition from GAE and Heroku (which originally was focused on a similar PaaS offering) seems to have driven most of the other alternatives to pivot to something else or fail entirely. Right now, there's very few close substitutes; there's lots of "alternatives" that require more infrastructure work (e.g., IaaS and similar offerings like GCE, GKE, EC2, etc.) or are off in the other direction ("severless" function hosting like Lambda or GCF.) And in some cases these might be better options than an GAE-style PaaS, but they aren't really direct substitutes, and they leave a space better served by GAE.
bduerst 10y ago
Wait, how? I've used it before for one-off web projects, but didn't think it was that bad.
- macNchz 10y ago
  Having inherited a large legacy codebase running on App Engine and powering a successful business, I see the biggest danger with App Engine as being the lock-in. We're on App Engine and we're not going anywhere, because it'd be too big of a project to move somewhere else. There's one open source project I know of that lets you run an App Engine project on another server (Appscale), but it's hard to know how useable it would be without the overhead of getting everything set up, and surely there are plenty of dark corners you'd encounter along the way. Since App Engine comes with a proprietary data store, ORM and task queue built in, skipping the Appscale compatibility thing and just porting a large application to run on another platform would be a herculean effort. The data store models and everywhere in the application code that they're queried would need to be re-written for a different ORM, all of the data would have to be migrated to a different store, and all the background tasks and methods that trigger them would need to be rebuilt. This would be thousands and thousands of lines of hand-edited code changes, before even thinking about how much time would need to be spent verifying all of the changes. At least with Heroku the magic is mostly around deploying and scaling the various pieces, rather than providing a lot of application level libraries that lock you into the platform. Porting to another platform likely would require a bunch of configuration management and some deployment scripts, but fairly limited changes to the application code itself.
  - jacques_chester 10y ago
    The easiest path off Heroku to self-hosted is probably Cloud Foundry: one of the inspirations was Heroku and several of the Cloud Foundry buildpacks are soft forks of Heroku's buildpacks. Generally we find that any buildpack written for Heroku will work with Cloud Foundry without modification. With a little extra engineering you can create a buildpack that will run in a fully disconnected environment. Disclosure: I work on the Cloud Foundry Buildpacks team on behalf of Pivotal.
  - 10y ago
robryk 10y ago
> (...) non-lock-in alternatives (...) There is an opensource reimplementation of App Engine's interface. That said, I have no clue about its completeness or buglessness. https://github.com/AppScale/appscale https://github.com/AppScale/appscale
boulos 10y ago
Do you feel the same way about App Engine Flexible Environment (neé Managed VMs)? That is, is your complaint that GAE Standard offers you all these services (like Memcache, Task Queues, Datastore) or that you hate the sandbox it's in? The GAE team has been (admittedly slowly) pushing the services that were GAE only into being fully-consumable "Cloud Platform" services (e.g., Cloud Datastore is the same Datastore that's been "part of" App Engine forever, but now accessible from anywhere). It's your choice to use Datastore, and I respect that you would chose to avoid it, but the basic PaaS idea of "here's some code, run it for me as a web endpoint" is still compelling even today. Disclosure: I work on Google Cloud.
- scrollaway 10y ago
  My impression of GAE is that on top of the core issues of the platform's (lock in by design, outdated by design), it is also very much unloved by Google and has received very few updates. I stayed away from it so I could be wrong. But to answer your question, I don't know anything about that Amazon service but I would feel the same way about similarly-designed competing product. IMHO, no informed person would ever pick proprietary, lock-in-heavy PAAS solutions.
  - boulos 10y ago
    But picking at that a bit, if I were to run a simple off-the-shelf Django setup against MySQL, I'd use GAE (if not Standard than certainly Flexible). My code wouldn't be any different than deploying it to a raw VM, so I'm not locked in and could deploy it on DO, AWS or anywhere else in a heartbeat. In the meantime, I don't have to deal with setting up load balancing, autoscaling, DNS per module, etc. so I get some real value from that. Is your concern the lock-in from the code standpoint, or operational "lock-in" that once you've got all this set up for you, you'd have to go replicate it to get out? If the latter, aren't you saying you're going to do that on Day 1 in the "avoid PaaS" case?
    - scrollaway 10y ago
      Why would you use GAE in this scenario, over, say, GCE or even AWS? And at a smaller scale, DO would be far cheaper.

jfindley 10y ago

Agree WRT network performance, the basic instances are unreasonably slow (only 70 Mbps for a micro), and requiring users to pay for a big 8 core server for wirespeed Gbe, and a huge 40 core beast for 10Gbe (and rather poor 10gbe at that) is totally unreasonable, and borders on price gouging (if such a term can be used for this market sector). The cross-AZ network performance is terrible much of the time, the jitter and loss are unpredictable and often far higher than you'd expect and generally trying to run high performance network services on AWS is deeply unpleasant in multiple directions at once. WRT to VPC and security groups, I wonder if the OP might be talking about some of the surprising and frustrating limitations with them. E.g. VPCs are limited to fairly small subnet blocks, within which you have virtually no control. You can specify simple static routes - but only for subnets outside your VPC block. If you want to run a routing protocol in your VPC (e.g. you want your docker containers to be reachable via BGP), you get stuck pretty quickly - if the BGP subnet is within the VPC block, you can't route to them within the VPC, as the router won't permit more specific routes within the BGP block. If it's not within the VPC block, you won't be able to get to it via VPNs, peering connections, etc. So you're stuck either way. Security groups are also limited - the number of rules you can have is pretty small, and they are a weird mix of stateless and stateful - they are stateful if the src/dest is an individual IP or group, but stateless for 0.0.0.0/0. And As you can't match TCP states like related or established, properly securing services when you have rules including 0.0.0.0/0 can be surprisingly difficult to achieve. These are but a few examples of the limitations of the ways in which networking and security are implemented on AWS - there are plenty plenty more.

vacri 10y ago
> Security groups are also limited - the number of rules you can have is pretty small You can have multiple security groups now, though. Previously you were limited to just the one.

eeeeeeeeeeeee 10y ago

The argument in the original article against VPC sounded more like "it's confusing, so it's not worthwhile." VPC is confusing if you don't understand basic networking, but most people can pick it up, and people coming from a physical datacenter, moving to the cloud, will get it almost immediately.

uptown 10y ago

Can you use VPC to create an environment that blocks outside access to Internet users completely, but connects to a primary hosting environment hosted elsewhere so that systems operating in both locations can communicate freely without concern for the AWS environment being exploited to access your primary server farm?

vacri 10y ago
Yes, you can have VPN endpoints to your VPC, where the other end is in whatever external network you want. And if you don't want any external communication at all (barring your VPN), just remove the 'IGW' from your VPC (or make a new VPC without one). Or modify the VPC's routing tables. Or don't assign any public IPs to anything in the VPC. Or probably a few other methods :)
NetStrikeForce 10y ago
This is one of the main reasons we created Wormhole Network[1]. You can block all incoming traffic to your VPC and still be able to freely communicate with your resources in other locations. The good thing is that you can easily add and remove devices / users to your network. We require an agent to be deployed, which is SoftEther's VPN client (free, open source, known VPN client). [1] https://wormhole.network https://wormhole.network

vacri 10y ago

> looks like a tool from the 90s I see this criticism a lot for various things... and the things in question never look like the tools and/or websites I saw in the 90s. Instead, they look like things we'd dream of having in the 90s.

user5994461 10y ago

4. In terms of VPC and networking. I am definitely not satisfied with AWS. - On GCE the security groups are created automatically and managed by the "role" of instances. On AWS there is no link between instances and security groups. I currently have to emulate the working of GCE over AWS with extremely complex ansible scripts just so that an instance called "repository" can actually be assigned to a group called "repository". - ELB (load balancer on AWS) cannot have a fixed IP (we've been waiting for that for YEARS). An ELB can only be accessed over a DNS name, and the underlying IP can change every 60 second. (Have fun with applications which are caching DNS). By comparison, GCE has had load balancers with fixed IPs forever. - AWS is a region nightmare. Many resources only exist in one region and cannot be accessed or even acknowledged to exist from another region. e.g. an AMI only exists in a single region. It's not possible to create a host with it in another region. It's not even possible to take that existing AMI and push it to another region. (As far as a region is concerned, other regions don't exist). All services on AWS are acutely region centric. I am currently expanding my infra to multiple regions and there are many obstacles to overcome. The networking and interconnections is definitely one of them. By comparison Google is not region centric like that. It's a lot easier to manage at planet scale. --- Just three major pain points on the top of my head.