4 ms·

The impossible to replicate the production environment I think was referred to impossible to replicate locally, on your machine. At my job I use AWS serverless

The impossible to replicate the production environment I think was referred to impossible to replicate locally, on your machine. At my job I use AWS serverless services and I get a lot of frustration not being able to test and debug code offline. Having each time to upload some code to debug it is time consuming. Also you have to rely only on logs to debug, you obviously cannot use a debugger, and thus the solution is to insert a ton of print statements in the code and remove them, which is not a problem to me (I usually do that even in code that I debug locally) but the service to read these logs (ColoudWatch) is not great at all, you don't even have all your logs in one place, it's a mess. I think serverless is overrated, sure it maybe the right tool for a simple project, but when the complexity grows it's best to use other more classical solutions.

k__ 6y ago
Thundra offers online and offline debugging. Also, AWS SAM CLI allows to run things locally.
rualca 6y ago
> At my job I use AWS serverless services and I get a lot of frustration not being able to test and debug code offline. AWS's Serverless Application Model explicitly supports local testing. https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-using-debugging.html https://docs.aws.amazon.com/serverless-application-model/lat... Which problems did you experienced?
- vp8989 6y ago
  "AWS's Serverless Application Model explicitly supports local testing." It doesn't provide any additional capability that was already there. Ie, I could already run my arbitrary function from the CLI by building a simple wrapper that reads a JSON file and sends it in. Or wrap it in a simple HTTP server. The debugging story using that tool is worse than how I would already do it pre-SAM and they don't even have documentation for every runtime they officially support: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-using-debugging.html https://docs.aws.amazon.com/serverless-application-model/lat... It doesn't allow you to test any of the interesting bits, which is what the blog post was alluding to. - What happens if I have an SQS Lambda with 100 concurrency and a few of the containers get into a bad state? Do the other ones keep going or does the whole thing grind to a halt? - What happens if I have a Lambda that consumes from 50 Kinesis shards and some of the containers get into a bad state? - What happens if my Lambda throws an exception, what happens to my SQS message or Kinesis record? If you outsource the plumbing you can't actually test the system locally you can only test "units" of it and in a lot of cases you either have to do remote tests with print debugging to figure out what's going on or what is more common is blindly assume the plumbing works how you intend, ship it to Prod and then fix problems as they occur.
  - rualca 6y ago
    > It doesn't allow you to run all the other services like DynamoDB, SNS, S3, etc. locally. Sounds like you're mixing up how a test pyramid works. AWS SAM supports running unit tests locally. Your remarks refer to integration tests. Integration tests are expected to be performed on environments that mimic either subsets of the production environment, or the whole production environment. https://docs.aws.amazon.com/whitepapers/latest/serverless-architectures-lambda/integration-testing.html https://docs.aws.amazon.com/whitepapers/latest/serverless-ar...
    - spdionis 6y ago
      What about manual tests? You know, just running the app on your machine and check if it works/how it works. You're forced to either share the aws resources with other devs or provision separate resources for each dev which is a big PITA.
      - viraptor 6y ago
        As long as you use terraform/cloudformation/sam, it should be fairly simple. You deploy everything with your name as a suffix. Most of the resources will be billed on usage only as well, so you don't even pay more than a few cents for testing like that.
    - earthboundkid 6y ago
      It’s only an “integration” test because you split things up too much. In a monolith, it would all be unit tests against a mock DB.
- Dunedan 6y ago
  SAM is able to run AWS Lambda functions locally and to emulate some API Gateway behavior. It doesn't allow you to run all the other services like DynamoDB, SNS, S3, etc. locally. As Serverless applications often rely on such additional AWS services, the ability of SAM to run AWS Lambda functions locally doesn't really help for running such applications locally.
  - deleted 6y ago
    [deleted]
  - zerubeus 6y ago
    You guys don't even take the time to do a simple search... https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg... You can have pretty much all the services locally if you want you can even have a step function locally
    - Dunedan 6y ago
      DynamoDB and AWS Step Functions are notable exceptions to the fact that AWS services are usually not available locally. The vast majority of AWS services isn't available outside of AWS.
      - zerubeus 6y ago
        I have a local tool for every AWS service just tell what are you looking for, for example I use https://github.com/softwaremill/elasticmq https://github.com/softwaremill/elasticmq to emulate SQS locally
        viraptor 6y ago
        There's also https://github.com/localstack/localstack https://github.com/localstack/localstack for the heavier services.
Dunedan 6y ago
I agree. Iteration speed and the ability to debug applications are the biggest downsides for me as well when developing Serverless applications. To make debugging at least a bit easier, I prefer doing structured logging (e.g. by utilizing AWS Lambda Powertools [1]) to get the ability to query the logs efficiently with CloudWatch Insights [2]. Also AWS X-Ray [3] is really valuable to understand how a Serverless application behaves. [1]: https://github.com/awslabs/aws-lambda-powertools-python https://github.com/awslabs/aws-lambda-powertools-python [2]: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Ana... [3]: https://aws.amazon.com/xray/ https://aws.amazon.com/xray/
ed25519FUUU 6y ago
> At my job I use AWS serverless services and I get a lot of frustration not being able to test and debug code offline. For my purposes debugging is as simple as crafting a payload and feeding it through the handler locally. I match credentials locally with what my lambda runs as. Since it’s a monolith function it behaves basically the same way, including the usage of a debugger. To make life a little easier any time there’s an exception it persists that ordinal payload so I can just replay it locally. But I agree that serverless is mostly overrated for most use cases.
neurostimulant 6y ago
Ugh, I rewrote my serverless service for exactly these reasons. It felt magical the first time where you can deploy a service without spinning up a server so I put up with all these annoyances. But then docker gains popularity and suddenly there is no practical advantage in using serverless anymore, at least for me. I rewrote the service in django, packaged it as a docker image, then run it on any platform that can run docker. On the other hand, I use cloudflare worker on a bunch of my projects. Unlike aws serverless, cloudflare worker is actually useful because it can do many things that can't be done by traditional backend, such as running and intercepting requests on the edge servers that sit between your server and your visitors.
- Dunedan 6y ago
  What you do with CloudFlare Workers can be done on AWS with Lambda@Edge [1]. [1]: https://docs.aws.amazon.com/lambda/latest/dg/lambda-edge.html https://docs.aws.amazon.com/lambda/latest/dg/lambda-edge.htm...
  - neurostimulant 6y ago
    I can't comment on lambda@edge as it wasn't available when I was still using aws lambda a few years ago. My current use case is probably supported on lambda@edge if I were still on aws as I'm mostly only use them to implement smart caching and routing.
zerubeus 6y ago
I think you never heard about SAM/Serverless Framework, this is why you suffer.

k__ 6y ago

Thundra offers online and offline debugging. Also, AWS SAM CLI allows to run things locally.

rualca 6y ago

> At my job I use AWS serverless services and I get a lot of frustration not being able to test and debug code offline. AWS's Serverless Application Model explicitly supports local testing. https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-using-debugging.html https://docs.aws.amazon.com/serverless-application-model/lat... Which problems did you experienced?

vp8989 6y ago
"AWS's Serverless Application Model explicitly supports local testing." It doesn't provide any additional capability that was already there. Ie, I could already run my arbitrary function from the CLI by building a simple wrapper that reads a JSON file and sends it in. Or wrap it in a simple HTTP server. The debugging story using that tool is worse than how I would already do it pre-SAM and they don't even have documentation for every runtime they officially support: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-using-debugging.html https://docs.aws.amazon.com/serverless-application-model/lat... It doesn't allow you to test any of the interesting bits, which is what the blog post was alluding to. - What happens if I have an SQS Lambda with 100 concurrency and a few of the containers get into a bad state? Do the other ones keep going or does the whole thing grind to a halt? - What happens if I have a Lambda that consumes from 50 Kinesis shards and some of the containers get into a bad state? - What happens if my Lambda throws an exception, what happens to my SQS message or Kinesis record? If you outsource the plumbing you can't actually test the system locally you can only test "units" of it and in a lot of cases you either have to do remote tests with print debugging to figure out what's going on or what is more common is blindly assume the plumbing works how you intend, ship it to Prod and then fix problems as they occur.
- rualca 6y ago
  > It doesn't allow you to run all the other services like DynamoDB, SNS, S3, etc. locally. Sounds like you're mixing up how a test pyramid works. AWS SAM supports running unit tests locally. Your remarks refer to integration tests. Integration tests are expected to be performed on environments that mimic either subsets of the production environment, or the whole production environment. https://docs.aws.amazon.com/whitepapers/latest/serverless-architectures-lambda/integration-testing.html https://docs.aws.amazon.com/whitepapers/latest/serverless-ar...
  - spdionis 6y ago
    What about manual tests? You know, just running the app on your machine and check if it works/how it works. You're forced to either share the aws resources with other devs or provision separate resources for each dev which is a big PITA.
    - viraptor 6y ago
      As long as you use terraform/cloudformation/sam, it should be fairly simple. You deploy everything with your name as a suffix. Most of the resources will be billed on usage only as well, so you don't even pay more than a few cents for testing like that.
  - earthboundkid 6y ago
    It’s only an “integration” test because you split things up too much. In a monolith, it would all be unit tests against a mock DB.
Dunedan 6y ago
SAM is able to run AWS Lambda functions locally and to emulate some API Gateway behavior. It doesn't allow you to run all the other services like DynamoDB, SNS, S3, etc. locally. As Serverless applications often rely on such additional AWS services, the ability of SAM to run AWS Lambda functions locally doesn't really help for running such applications locally.
- deleted 6y ago
  [deleted]
- zerubeus 6y ago
  You guys don't even take the time to do a simple search... https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg... You can have pretty much all the services locally if you want you can even have a step function locally
  - Dunedan 6y ago
    DynamoDB and AWS Step Functions are notable exceptions to the fact that AWS services are usually not available locally. The vast majority of AWS services isn't available outside of AWS.
    - zerubeus 6y ago
      I have a local tool for every AWS service just tell what are you looking for, for example I use https://github.com/softwaremill/elasticmq https://github.com/softwaremill/elasticmq to emulate SQS locally
      - viraptor 6y ago
        There's also https://github.com/localstack/localstack https://github.com/localstack/localstack for the heavier services.

Dunedan 6y ago

I agree. Iteration speed and the ability to debug applications are the biggest downsides for me as well when developing Serverless applications. To make debugging at least a bit easier, I prefer doing structured logging (e.g. by utilizing AWS Lambda Powertools [1]) to get the ability to query the logs efficiently with CloudWatch Insights [2]. Also AWS X-Ray [3] is really valuable to understand how a Serverless application behaves. [1]: https://github.com/awslabs/aws-lambda-powertools-python https://github.com/awslabs/aws-lambda-powertools-python [2]: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Ana... [3]: https://aws.amazon.com/xray/ https://aws.amazon.com/xray/

ed25519FUUU 6y ago

> At my job I use AWS serverless services and I get a lot of frustration not being able to test and debug code offline. For my purposes debugging is as simple as crafting a payload and feeding it through the handler locally. I match credentials locally with what my lambda runs as. Since it’s a monolith function it behaves basically the same way, including the usage of a debugger. To make life a little easier any time there’s an exception it persists that ordinal payload so I can just replay it locally. But I agree that serverless is mostly overrated for most use cases.

neurostimulant 6y ago

Ugh, I rewrote my serverless service for exactly these reasons. It felt magical the first time where you can deploy a service without spinning up a server so I put up with all these annoyances. But then docker gains popularity and suddenly there is no practical advantage in using serverless anymore, at least for me. I rewrote the service in django, packaged it as a docker image, then run it on any platform that can run docker. On the other hand, I use cloudflare worker on a bunch of my projects. Unlike aws serverless, cloudflare worker is actually useful because it can do many things that can't be done by traditional backend, such as running and intercepting requests on the edge servers that sit between your server and your visitors.

Dunedan 6y ago
What you do with CloudFlare Workers can be done on AWS with Lambda@Edge [1]. [1]: https://docs.aws.amazon.com/lambda/latest/dg/lambda-edge.html https://docs.aws.amazon.com/lambda/latest/dg/lambda-edge.htm...
- neurostimulant 6y ago
  I can't comment on lambda@edge as it wasn't available when I was still using aws lambda a few years ago. My current use case is probably supported on lambda@edge if I were still on aws as I'm mostly only use them to implement smart caching and routing.

zerubeus 6y ago

I think you never heard about SAM/Serverless Framework, this is why you suffer.