3 ms·

For AWS specifically, I prefer to have an AWS account specifically dedicated to each service + stage. For example, if I have an image service that handles s3 up

For AWS specifically, I prefer to have an AWS account specifically dedicated to each service + stage. For example, if I have an image service that handles s3 uploads (say Lambda, S3, Cloudfront and API Gateway), then I'd deploy a "test" environment to a dedicated AWS account and run tests against that. Since it's fully serverless, it only costs a few pennies to test (or free). I try not to develop locally at all anymore. If you're looking for more practical advice, perhaps this will help: https://dev.to/aws-builders/developing-against-the-cloud-55o4 https://dev.to/aws-builders/developing-against-the-cloud-55o...

cortesoft 5y ago
That means that every person who runs the tests needs credentials for that AWS account. That obviously won’t work for an open source project. Even for a company project, how do you distribute those secrets? It adds friction for developers getting their local dev environment setup. Not only that, but you now need network access to run tests. A network blip or a third party service outage now makes your tests fail. There is also the possibility that an aborted test run might leave state in s3 that you are now paying for. Someone hits Ctrl-c during a test run and now you have a huge AWS bill.
- astuyvenberg 5y ago
  > That obviously won’t work for an open source project On the contrary - I was an employee at Serverless Inc, working on the Serverless Framework for the last two years, we used this pattern extensively (and very successfully) in our open source repos. You can even find an example here which provisions real live AWS infrastructure: https://github.com/serverless/dashboard-plugin/tree/master/integration-testing https://github.com/serverless/dashboard-plugin/tree/master/i... We used part of our enterprise SaaS product to provision temporary credentials via STS and an assumable role, and it works great. You could do the same thing with something like HC Vault. For Lambda, S3, DynamoDB, the perpetual free tier means we've never paid to run our own tests. API Gateway isn't free (after 1 year), but it's still pennies per month. We've had several cases where tests stuck around a long time, but a billing alert and occasionally some CloudFormation stack cleanup takes care of that. We still have offline unit tests which test business logic, but everything else runs against the cloud - even our development environments ship code straight to lambda.
  - deleted 5y ago
    [deleted]
Supermancho 5y ago
Why spend money on AWS you don't have to? Use Minio for S3 locally (or on your build server). Local development is the easiest way to avoid wasting money and resources on debugging/development.
- deleted 5y ago
  [deleted]
- astuyvenberg 5y ago
  Speaking from personal experience, our team wasted far more money tinkering with local dev environments and trying to replicate the cloud than we ever did simply using it to develop. The blog post in the parent comment lays out our experience and my thoughts, but because of the pretty generous free tier, I don't think we've ever paid a penny for a build/dev/test AWS account.

cortesoft 5y ago

That means that every person who runs the tests needs credentials for that AWS account. That obviously won’t work for an open source project. Even for a company project, how do you distribute those secrets? It adds friction for developers getting their local dev environment setup. Not only that, but you now need network access to run tests. A network blip or a third party service outage now makes your tests fail. There is also the possibility that an aborted test run might leave state in s3 that you are now paying for. Someone hits Ctrl-c during a test run and now you have a huge AWS bill.

astuyvenberg 5y ago
> That obviously won’t work for an open source project On the contrary - I was an employee at Serverless Inc, working on the Serverless Framework for the last two years, we used this pattern extensively (and very successfully) in our open source repos. You can even find an example here which provisions real live AWS infrastructure: https://github.com/serverless/dashboard-plugin/tree/master/integration-testing https://github.com/serverless/dashboard-plugin/tree/master/i... We used part of our enterprise SaaS product to provision temporary credentials via STS and an assumable role, and it works great. You could do the same thing with something like HC Vault. For Lambda, S3, DynamoDB, the perpetual free tier means we've never paid to run our own tests. API Gateway isn't free (after 1 year), but it's still pennies per month. We've had several cases where tests stuck around a long time, but a billing alert and occasionally some CloudFormation stack cleanup takes care of that. We still have offline unit tests which test business logic, but everything else runs against the cloud - even our development environments ship code straight to lambda.
- deleted 5y ago
  [deleted]

Supermancho 5y ago

Why spend money on AWS you don't have to? Use Minio for S3 locally (or on your build server). Local development is the easiest way to avoid wasting money and resources on debugging/development.

deleted 5y ago
[deleted]
astuyvenberg 5y ago
Speaking from personal experience, our team wasted far more money tinkering with local dev environments and trying to replicate the cloud than we ever did simply using it to develop. The blog post in the parent comment lays out our experience and my thoughts, but because of the pretty generous free tier, I don't think we've ever paid a penny for a build/dev/test AWS account.