2 ms·
> Instead, you can write a function that takes parameters from elsewhere and call your business logic there. This is what I tried to do initially after experie
by nickjj 6y ago
> Instead, you can write a function that takes parameters from elsewhere and call your business logic there.
This is what I tried to do initially after experiencing the dev pain for only a few minutes.
But unfortunately this doesn't work very well in anything but the most trivial case because as soon as your lambda has a 3rd party package dependency you need to install that dependency somehow.
For example, let's say you have a Python lambda that does some stuff, writes the result to postgres and then sends a webhook out using the requests library.
That means your code needs access to a postgres database library and the requests library to send a webhook response.
Suddenly you need to pollute your dev environment with these dependencies to even run the thing outside of lambda and every dev needs to follow a 100 step README file to get these dependencies installed and now we're back to pre-Docker days.
Or you spin up your own Docker container with a volume mount and manage all of the complexity on your own. It seems criminal to create your own Dockerfile just to develop the business logic of a lambda where you only use that Dockerfile for development.
Then there's the whole problem of running your split out business logic without it being triggered from a lambda. Do you just write boiler plate scripts that read the same JSON files, and set up command line parsing code in the same way as sam local invoke does to pass in params?
Then there's also the problem of wanting one of your non-Serverless services to invoke a lambda in development so you can actually test what happens when you call it in your main web app but instead of calling sam local invoke, you really want that service's code to be more like how it would run in production where it's triggered by an SNS publish message. Now you need to somehow figure out how to mock out SNS in development.
Serverless is madness from start to finish.
- lstamour 6y agoUnless I’be misunderstood, every knock against serverless above has actually been a knock against the complexity of having tiny, de-coupled cloud native services and how difficult it can be to mock... to which the answer is often “don’t mock, start by using real services” and then when that is less reliable or you need unit tests, then mock the data you expect. In the case of SNS, mock a message with the correct SNS signature, or go one layer deeper, stub out SNS validation logic and just unit test the function assuming the response is valid or invalid? In the case of Postgres, you could use an ORM that supports SQLite for dependency-free development but at a compatibility cost... worst case you might need to have your local machine talk to AWS and host it’s own LetsEncrypt certificate and open NAT port... but one can hope it doesn’t come to that...? Even so... that’s not exactly a knock against serverless itself, is it?
- monadic2 6y agoWell sure, but then your development (and test) environment is tied to live services and you now you have two problems. It really, really sucks when an unrelated service goes down during an existing incident and you never mocked.
- nickjj 6y ago> In the case of SNS, mock a message with the correct SNS signature, or go one layer deeper, stub out SNS validation logic. SAM already provides a way to mock out what SNS would send to your function so that the function can use the same code path in both cases. Basically mocking the signature. This is good to make sure your function is running the same code in both dev and prod and lets you trigger a function in development without needing SNS. But the problem is locally invoking the function with the SAM CLI tool is the trigger mechanism where you pass in that mocked out SNS event, but in reality that only works for running that function in complete isolation in development. In practice, what you'd really likely want to do is call it from another local service so you can test how your web app works (the thing really calling your lambda at the end of the day). This involves calling SNS publish in your service's code base to trigger the lambda. That means really setting up an SNS topic and deploying your lambda to AWS or calling some API compatible mock of SNS because if you execute a different code path then you have no means to test the most important part of your code in dev. > In the case of Postgres, you could use an ORM that supports SQLite for dependency-free development but at a compatibility cos The DB is mostly easy. You can throw it into a docker-compose.yml file and use the same version as you run on RDS with like 5 lines of yaml and little system requirements. Then use the same code in both dev and prod while changing the connection string with an environment variable. > That’s not exactly a knock against serverless itself, is it? It is for everything surrounding how lambdas are triggered and run. But yes, you'd run into the DB, S3, etc. issues with any tech choice.
- lstamour 6y agoSo there’s an argument that the future deployment model is actually Kubernetes Operators, which means you could have test code that deploys and sets up AWS APIs... thus if your code responds to the trigger, it’s up to another bit of code to make sure the trigger is installed and works as expected against AWS APIs? And yes, I think the problem here are APIs you use in multiple places but can’t easily run yourself in a production-friendly way. Until AWS builds and supports Docker containers to run their APIs locally, I don’t see how this improves... end to end testing of AWS requires AWS? ;-)