9 ms·

You're only thinking about the _input_. Technically, yes, I can host an express app on lambda just like I could by other means, but the problem is that it can't

You're only thinking about the _input_. Technically, yes, I can host an express app on lambda just like I could by other means, but the problem is that it can't really _do_ anything. Unless you're performing a larger job or something you probably need to read/write data from somewhere and connecting to a normal database is too slow for most use-cases. Connecting to AWS managed services (s3, kinesis, dynamodb, sns) don't have this overhead so you can actually perform some task that involves reading/writing data. Lambda is basically just glue code to connect AWS services together. It's not a general purpose platform. Think "IFTTT for AWS"

scarface74 8y ago
The API I just deployed using this connects to a regular old Aurora MySQL instance using the standard MySQL driver. It’s a standard CRUD REST API used by our website. The only thing slow is infamous cold start time when running within a VPC because it has to create an ENI. AWS pinky promised they were going to fix this soon.
KorematsuFred 8y ago
But that is the whole point of using cloud services that are tightly integrated with each other. I can not do it as efficiently as Amazon myself can not be called "propriety lock-in".
- chrischen 8y ago
  Said efficiencies are not due to Amazon, just that the services are colocated in the same facility. If I put the node service and a database on the same box I'd get the same performance, and actually probably better since Amazon would still have them on separate physical hardware.
  - scarface74 8y ago
    It’s not about performance, then I have to support all of that infrastructure myself.
    - chrischen 8y ago
      I'm talking about his statement: "Connecting to AWS managed services (s3, kinesis, dynamodb, sns) don't have this overhead so you can actually perform some task that involves reading/writing data." That is due to network and colocation efficiencies. The overhead of managing such services yourself is another matter.
      - scarface74 8y ago
        Not just the network overhead, the maintenance and setup overhead. I can spin up an entire full stack in multiple accounts just by creating a CloudFormation template. I’ve done stress testing by spinning up and tearing down multiple VMs played with different size databases, autoscaled read replicas for performance. Ran a spot fleet, etc. When you need things now you don’t have time to requisition hardware and get it sent to your colo.
        chrischen 8y ago
        As far as spinning up and down, a lot of this is solved with docker, while also being relatively platform independent.
        scarface74 8y ago
        So Docker allows me to scale up MySQL Read replicas instantaneously? And I still have to manage infrastructure.
        chrischen 8y ago
        Well, you can use a container service or use EC2 still.
        scarface74 8y ago
        And then you still have more stuff to manage now based on the slim chance that one day years down the road you might rip your entire multi Az redundant infrastructure, your databases, etc with all of the read replicas to another provider.... And this doesn’t count all of the third party hosted services. Aurora (Mysql) redundantly writes your data to six different storage devices across multiple availability zones. The read replicas read from the same disks. As soon as you bring up a read replica, the data is already there. You can’t do that with a standard Mysql read replica.
matchagaucho 8y ago
Our Lambda deployments handle REST API Gateway calls, SQS events, and Step functions. Basically the entire middleware of a classic 3-tier stack. Except for some proprietary light AWS proxy code, the bulk of the Lambdas delegate to pre-existing Java POJO classes. The cold start issues and VPC configuration were a painful learning curve, but nothing I would consider proprietary to AWS. Those are universal deployment tasks.
hn_throwaway_99 8y ago
> Unless you're performing a larger job or something you probably need to read/write data from somewhere and connecting to a normal database is too slow for most use-cases. This is false. I've seen entire Lambda APIs backed by MySQL on large, consumer-facing apps and websites. As another poster pointed out, the cold-start-in-a-VPC is a major PITA, but it can (mostly) be worked around.
- scarface74 8y ago
  And there is always DynamoDB where you aren’t in a VPC and Serverless Aurora where you don’t have to worry about the typical database connections and you can use the http based Data APIs.
  - redisman 8y ago
    Dynamo is really the hardest lock-in in the ecosystem for me. Serverless Aurora is still easy to kill with too much concurrency/bad connection management compared to Dynamo
    - scarface74 8y ago
      That’s theoretically where the Data API comes in.
  - meekins 8y ago
    How is the Aurora Serverless Data API now? On preview release it was a bit sketchy: horrible latencies (pretty much ruining any benefit you could get from avoiding the in-VPC cold start) and a dangerous sql-query-as-a-string API (no prepared statements or placeholders for query params that would get automatically escaped IIRC).
    - scarface74 8y ago
      Unfortunately, we require the ability to load/unload directly from S3 and Aurora serverless doesn’t support that. We haven’t been able to do anymore than a POC.
staticassertion 8y ago
OK. So you connect to Postgres on RDS - cloud agnostic. You connect to S3, and: a) You can build an abstraction service if you care about vendor lock-in so much b) It has an API that plenty of open source projects are compatible with (I believe Google's storage is compatible as well) Maybe you use something like SQS or SNS. Bummer, those are gonna "lock you in". But I've personally migrated between queueing solutions before and it shouldn't be a big deal to do so. It's really easy to avoid lockin, lambda really doesn't make it any harder than EC2 at all.
- scarface74 8y ago
  Have you ever asked the business folks or your investors did they care about your “levels of abstraction”? What looks better on your review? I created a facade over our messaging system or I implemented this feature that brought in revenue/increased customer retention/got us closer to the upper right quadrant of Gartner’s magic square?
  - staticassertion 8y ago
    I'm not sure I see your point. What is it you think I'm advocating for? My only point is that vendor lock-in is not a significant issue on AWS, and that it requires virtually no effort to avoid it.
    - tomnipotent 8y ago
      > and that it requires virtually no effort to avoid it Of course it requires effort. A lot of effort, not to mention headcount. The entire value of cloud-managed services is what it saves you vs. the trade-off's, and it's disingenuous to pretend that's not the case.
      - staticassertion 8y ago
        Sorry, I don't agree, and I feel like I provided evidence why in my first post. To summarize, choosing services like Postgres and S3 doesn't lock you in. SQS and SNS might, but I think it's an exaggerated cost, and that has nothing to do with Lambdas (EC2 instances are just as likely to use SQS or SNS - moreso, given that SQS wasn't supported for Lambdas until recently). There are tradeoffs, of course. Cost at scale is the really big one - at some point it's cheaper to bring ops/ hardware in-house. I just don't agree that lock-in is a huge issue, and I really disagree with the idea that lambdas make lock-in harder.
vorticalbox 8y ago
We have been connecting to mongo db from without lambda for the past year and sure you don't get single digit latency but r/w data happens under 30ms in most cases, we even use paramastore to pull all secrets and it's still without that time frame.
- nailer 8y ago
  Where you connecting to Mongo from?
  - gtsteve 8y ago
    You can run your Lambda function within the same network as your other servers. It just appears as a new IP address inside your network and can call whatever you permit it to.
  - vorticalbox 8y ago
    our MongoDB is running on mlabs.
    - nailer 8y ago
      So but you mention you're connecting without lambda - so where are you connecting from?
      - giorgioz 8y ago
        he probably meant 'within' instead of 'without'. myself to use aws-serverless-express and connect my lambda hosted in us-east-virginia to a mlab (now Mongo Atlas) mongo database hosted in the same us-east-virginia amazon region

scarface74 8y ago

The API I just deployed using this connects to a regular old Aurora MySQL instance using the standard MySQL driver. It’s a standard CRUD REST API used by our website. The only thing slow is infamous cold start time when running within a VPC because it has to create an ENI. AWS pinky promised they were going to fix this soon.

KorematsuFred 8y ago

But that is the whole point of using cloud services that are tightly integrated with each other. I can not do it as efficiently as Amazon myself can not be called "propriety lock-in".

chrischen 8y ago
Said efficiencies are not due to Amazon, just that the services are colocated in the same facility. If I put the node service and a database on the same box I'd get the same performance, and actually probably better since Amazon would still have them on separate physical hardware.
- scarface74 8y ago
  It’s not about performance, then I have to support all of that infrastructure myself.
  - chrischen 8y ago
    I'm talking about his statement: "Connecting to AWS managed services (s3, kinesis, dynamodb, sns) don't have this overhead so you can actually perform some task that involves reading/writing data." That is due to network and colocation efficiencies. The overhead of managing such services yourself is another matter.
    - scarface74 8y ago
      Not just the network overhead, the maintenance and setup overhead. I can spin up an entire full stack in multiple accounts just by creating a CloudFormation template. I’ve done stress testing by spinning up and tearing down multiple VMs played with different size databases, autoscaled read replicas for performance. Ran a spot fleet, etc. When you need things now you don’t have time to requisition hardware and get it sent to your colo.
      - chrischen 8y ago
        As far as spinning up and down, a lot of this is solved with docker, while also being relatively platform independent.
        scarface74 8y ago
        So Docker allows me to scale up MySQL Read replicas instantaneously? And I still have to manage infrastructure.
        chrischen 8y ago
        Well, you can use a container service or use EC2 still.
        scarface74 8y ago
        And then you still have more stuff to manage now based on the slim chance that one day years down the road you might rip your entire multi Az redundant infrastructure, your databases, etc with all of the read replicas to another provider.... And this doesn’t count all of the third party hosted services. Aurora (Mysql) redundantly writes your data to six different storage devices across multiple availability zones. The read replicas read from the same disks. As soon as you bring up a read replica, the data is already there. You can’t do that with a standard Mysql read replica.

matchagaucho 8y ago

Our Lambda deployments handle REST API Gateway calls, SQS events, and Step functions. Basically the entire middleware of a classic 3-tier stack. Except for some proprietary light AWS proxy code, the bulk of the Lambdas delegate to pre-existing Java POJO classes. The cold start issues and VPC configuration were a painful learning curve, but nothing I would consider proprietary to AWS. Those are universal deployment tasks.

hn_throwaway_99 8y ago

> Unless you're performing a larger job or something you probably need to read/write data from somewhere and connecting to a normal database is too slow for most use-cases. This is false. I've seen entire Lambda APIs backed by MySQL on large, consumer-facing apps and websites. As another poster pointed out, the cold-start-in-a-VPC is a major PITA, but it can (mostly) be worked around.

scarface74 8y ago
And there is always DynamoDB where you aren’t in a VPC and Serverless Aurora where you don’t have to worry about the typical database connections and you can use the http based Data APIs.
- redisman 8y ago
  Dynamo is really the hardest lock-in in the ecosystem for me. Serverless Aurora is still easy to kill with too much concurrency/bad connection management compared to Dynamo
  - scarface74 8y ago
    That’s theoretically where the Data API comes in.
- meekins 8y ago
  How is the Aurora Serverless Data API now? On preview release it was a bit sketchy: horrible latencies (pretty much ruining any benefit you could get from avoiding the in-VPC cold start) and a dangerous sql-query-as-a-string API (no prepared statements or placeholders for query params that would get automatically escaped IIRC).
  - scarface74 8y ago
    Unfortunately, we require the ability to load/unload directly from S3 and Aurora serverless doesn’t support that. We haven’t been able to do anymore than a POC.

staticassertion 8y ago

OK. So you connect to Postgres on RDS - cloud agnostic. You connect to S3, and: a) You can build an abstraction service if you care about vendor lock-in so much b) It has an API that plenty of open source projects are compatible with (I believe Google's storage is compatible as well) Maybe you use something like SQS or SNS. Bummer, those are gonna "lock you in". But I've personally migrated between queueing solutions before and it shouldn't be a big deal to do so. It's really easy to avoid lockin, lambda really doesn't make it any harder than EC2 at all.

scarface74 8y ago
Have you ever asked the business folks or your investors did they care about your “levels of abstraction”? What looks better on your review? I created a facade over our messaging system or I implemented this feature that brought in revenue/increased customer retention/got us closer to the upper right quadrant of Gartner’s magic square?
- staticassertion 8y ago
  I'm not sure I see your point. What is it you think I'm advocating for? My only point is that vendor lock-in is not a significant issue on AWS, and that it requires virtually no effort to avoid it.
  - tomnipotent 8y ago
    > and that it requires virtually no effort to avoid it Of course it requires effort. A lot of effort, not to mention headcount. The entire value of cloud-managed services is what it saves you vs. the trade-off's, and it's disingenuous to pretend that's not the case.
    - staticassertion 8y ago
      Sorry, I don't agree, and I feel like I provided evidence why in my first post. To summarize, choosing services like Postgres and S3 doesn't lock you in. SQS and SNS might, but I think it's an exaggerated cost, and that has nothing to do with Lambdas (EC2 instances are just as likely to use SQS or SNS - moreso, given that SQS wasn't supported for Lambdas until recently). There are tradeoffs, of course. Cost at scale is the really big one - at some point it's cheaper to bring ops/ hardware in-house. I just don't agree that lock-in is a huge issue, and I really disagree with the idea that lambdas make lock-in harder.

vorticalbox 8y ago

We have been connecting to mongo db from without lambda for the past year and sure you don't get single digit latency but r/w data happens under 30ms in most cases, we even use paramastore to pull all secrets and it's still without that time frame.

nailer 8y ago
Where you connecting to Mongo from?
- gtsteve 8y ago
  You can run your Lambda function within the same network as your other servers. It just appears as a new IP address inside your network and can call whatever you permit it to.
- vorticalbox 8y ago
  our MongoDB is running on mlabs.
  - nailer 8y ago
    So but you mention you're connecting without lambda - so where are you connecting from?
    - giorgioz 8y ago
      he probably meant 'within' instead of 'without'. myself to use aws-serverless-express and connect my lambda hosted in us-east-virginia to a mlab (now Mongo Atlas) mongo database hosted in the same us-east-virginia amazon region