4 ms·

I used to be complain about the same thing and even asked someone who was head of BD for Serverless at AWS what they recommended, and didn't get an answer to my

by jpdb 6y ago

I used to be complain about the same thing and even asked someone who was head of BD for Serverless at AWS what they recommended, and didn't get an answer to my satisfaction. After working with more and more serverless applications (despite the development pains, the business value was still justified) I realized that local development was difficult because I was coupling my code to the delivery. This is similar to the way you shouldn't couple your code to your database implementation. Instead, you can write a function that takes parameters from elsewhere and call your business logic there. It definitely adds a bit more work, but it alleviates quite a bit of pain that comes with Lambda local development. Disclaimer: I work at AWS, however not for any service or marketing team. Opinions are my own.

jacques_chester 6y ago
> I realized that local development was difficult because I was coupling my code to the delivery. Of interest, I've spent some free time crunching on CNCF survey data over the past few months. Some of the strongest correlations are between particular serverless offerings and particular delivery offerings. If you use Azure Functions then I know you are more likely to use Azure Devops than anything else. Same for Lambda + CodePipeline and Google Cloud Functions + Cloud Build.
- oblio 6y ago
  I think his point was that you should be able to run and test the Lambda code independently of Lambda. After all the entry point is just a method with some parameters, you can replicate that locally.
  - jacques_chester 6y ago
    ... ah.
  - losteric 6y ago
    Yes, this is a great way of doing things - I have no problems TDD'ing business logic hosted in Lambda, because the business logic is fully decoupled from the execution environment. SAM should be for high-fidelity E2E integration testing.
- myrloc 6y ago
  I’d be interested in seeing more of your results, no matter how raw they are.
  - jacques_chester 6y ago
    My email address is in my profile. Personal is best, I've been crunching it on my own time.
deleted 6y ago
[deleted]
nickjj 6y ago
> Instead, you can write a function that takes parameters from elsewhere and call your business logic there. This is what I tried to do initially after experiencing the dev pain for only a few minutes. But unfortunately this doesn't work very well in anything but the most trivial case because as soon as your lambda has a 3rd party package dependency you need to install that dependency somehow. For example, let's say you have a Python lambda that does some stuff, writes the result to postgres and then sends a webhook out using the requests library. That means your code needs access to a postgres database library and the requests library to send a webhook response. Suddenly you need to pollute your dev environment with these dependencies to even run the thing outside of lambda and every dev needs to follow a 100 step README file to get these dependencies installed and now we're back to pre-Docker days. Or you spin up your own Docker container with a volume mount and manage all of the complexity on your own. It seems criminal to create your own Dockerfile just to develop the business logic of a lambda where you only use that Dockerfile for development. Then there's the whole problem of running your split out business logic without it being triggered from a lambda. Do you just write boiler plate scripts that read the same JSON files, and set up command line parsing code in the same way as sam local invoke does to pass in params? Then there's also the problem of wanting one of your non-Serverless services to invoke a lambda in development so you can actually test what happens when you call it in your main web app but instead of calling sam local invoke, you really want that service's code to be more like how it would run in production where it's triggered by an SNS publish message. Now you need to somehow figure out how to mock out SNS in development. Serverless is madness from start to finish.
- lstamour 6y ago
  Unless I’be misunderstood, every knock against serverless above has actually been a knock against the complexity of having tiny, de-coupled cloud native services and how difficult it can be to mock... to which the answer is often “don’t mock, start by using real services” and then when that is less reliable or you need unit tests, then mock the data you expect. In the case of SNS, mock a message with the correct SNS signature, or go one layer deeper, stub out SNS validation logic and just unit test the function assuming the response is valid or invalid? In the case of Postgres, you could use an ORM that supports SQLite for dependency-free development but at a compatibility cost... worst case you might need to have your local machine talk to AWS and host it’s own LetsEncrypt certificate and open NAT port... but one can hope it doesn’t come to that...? Even so... that’s not exactly a knock against serverless itself, is it?
steve_adams_86 6y ago
This principle works with front end development too. Crazy build times on large applications can be alleviated if your approach is to build logic in isolation, then do final testing and fitting in the destination. It’s hard to do this when surrounding code doesn’t accommodate the approach, but it’s great way to design an application if you have the choice. I really love sandboxing features before moving them into the application. Everything from design to testing can be so much faster and fun without the distractions of the build system and the rest of the application.

by jpdb 6y ago

jacques_chester 6y ago

> I realized that local development was difficult because I was coupling my code to the delivery. Of interest, I've spent some free time crunching on CNCF survey data over the past few months. Some of the strongest correlations are between particular serverless offerings and particular delivery offerings. If you use Azure Functions then I know you are more likely to use Azure Devops than anything else. Same for Lambda + CodePipeline and Google Cloud Functions + Cloud Build.

oblio 6y ago
I think his point was that you should be able to run and test the Lambda code independently of Lambda. After all the entry point is just a method with some parameters, you can replicate that locally.
- jacques_chester 6y ago
  ... ah.
- losteric 6y ago
  Yes, this is a great way of doing things - I have no problems TDD'ing business logic hosted in Lambda, because the business logic is fully decoupled from the execution environment. SAM should be for high-fidelity E2E integration testing.
myrloc 6y ago
I’d be interested in seeing more of your results, no matter how raw they are.
- jacques_chester 6y ago
  My email address is in my profile. Personal is best, I've been crunching it on my own time.

deleted 6y ago

[deleted]

nickjj 6y ago

> Instead, you can write a function that takes parameters from elsewhere and call your business logic there. This is what I tried to do initially after experiencing the dev pain for only a few minutes. But unfortunately this doesn't work very well in anything but the most trivial case because as soon as your lambda has a 3rd party package dependency you need to install that dependency somehow. For example, let's say you have a Python lambda that does some stuff, writes the result to postgres and then sends a webhook out using the requests library. That means your code needs access to a postgres database library and the requests library to send a webhook response. Suddenly you need to pollute your dev environment with these dependencies to even run the thing outside of lambda and every dev needs to follow a 100 step README file to get these dependencies installed and now we're back to pre-Docker days. Or you spin up your own Docker container with a volume mount and manage all of the complexity on your own. It seems criminal to create your own Dockerfile just to develop the business logic of a lambda where you only use that Dockerfile for development. Then there's the whole problem of running your split out business logic without it being triggered from a lambda. Do you just write boiler plate scripts that read the same JSON files, and set up command line parsing code in the same way as sam local invoke does to pass in params? Then there's also the problem of wanting one of your non-Serverless services to invoke a lambda in development so you can actually test what happens when you call it in your main web app but instead of calling sam local invoke, you really want that service's code to be more like how it would run in production where it's triggered by an SNS publish message. Now you need to somehow figure out how to mock out SNS in development. Serverless is madness from start to finish.

lstamour 6y ago
Unless I’be misunderstood, every knock against serverless above has actually been a knock against the complexity of having tiny, de-coupled cloud native services and how difficult it can be to mock... to which the answer is often “don’t mock, start by using real services” and then when that is less reliable or you need unit tests, then mock the data you expect. In the case of SNS, mock a message with the correct SNS signature, or go one layer deeper, stub out SNS validation logic and just unit test the function assuming the response is valid or invalid? In the case of Postgres, you could use an ORM that supports SQLite for dependency-free development but at a compatibility cost... worst case you might need to have your local machine talk to AWS and host it’s own LetsEncrypt certificate and open NAT port... but one can hope it doesn’t come to that...? Even so... that’s not exactly a knock against serverless itself, is it?

steve_adams_86 6y ago

This principle works with front end development too. Crazy build times on large applications can be alleviated if your approach is to build logic in isolation, then do final testing and fitting in the destination. It’s hard to do this when surrounding code doesn’t accommodate the approach, but it’s great way to design an application if you have the choice. I really love sandboxing features before moving them into the application. Everything from design to testing can be so much faster and fun without the distractions of the build system and the rest of the application.