7 ms·

i wish aws...and to be honest, most cloud providers put more effort into the development experience. they have local dynamo, but i believe that's the only offi

by chrsig 5y ago

i wish aws...and to be honest, most cloud providers put more effort into the development experience. they have local dynamo, but i believe that's the only official appliance they provide. I think it's especially necessary for the proprietary products -- dynamo being a great example. I'll take a moment to call out snowflake for this as well -- it's really frustrating to not be able to quickly (within milliseconds) and easily (i shouldn't need to get credentials from elsewhere in my org) setup & teardown databases while under test.

emersonrsantos 5y ago
I used Serverless framework to develop locally many times, it works very well for my case.
thrauat 5y ago
i think cloud providers have little incentive to help you develop and test everything locally, since they benefit more from onboarding you quickly into using their infrastructure, which they can much better monetize on. it seems to me that a completely local cloud stack would be threatening to cloud providers also in that you can then start running non-critical applications on-prem at no cost.
- chrsig 5y ago
  i don't think that's a real threat -- i don't think development environments are likely a huge income source for them, either. i think they'd be better served by letting me develop more quickly, so I can help my company grow, and presumably need (and have the budget for!) more resources in our production environment. if i'm mistaken, i think it's reasonable to have some sort of built in time limit to how long the appliance will run for, or some other way of preventing it being used for non-development purposes.
GusEggert 5y ago
There is also Step Functions Local (https://docs.aws.amazon.com/step-functions/latest/dg/sfn-local.html https://docs.aws.amazon.com/step-functions/latest/dg/sfn-loc..., it is also integrated with LocalStack). I built it as a side project when I was at AWS, motivated by the same frustrations. The best way to get these prioritized is to request it through your account manager/SA/etc. AWS does listen closely to customer feature requests.
mcbain 5y ago
It isn’t what you are asking for, but I was a bit surprised/happy to discover that the Ruby AWS SDK has a good stub client implementation: https://aws.amazon.com/blogs/developer/advanced-client-stubbing-in-the-aws-sdk-for-ruby-version-3/ https://aws.amazon.com/blogs/developer/advanced-client-stubb... I found it useful when updating some existing old ruby scripts we had. (Of course they had little to no testing before that.)
jacurtis 5y ago
In addition to local dynamo, they also have local AWS Lambda execution via AWS SAM (Serverless Application Model). You invoke it via the command line `sam local invoke [OPTIONS] [FUNCTION_LOGICAL_ID]` Behind the scenes it is running an attached docker container and feeding you the output. I found that it works actually very well and mimics the cloud invocation environment perfectly. The problem is that it stubs out any other services you run. So if your Lambda adds something to SQS or SNS or SES or S3 or anything else, it simply stubs those out. So advanced functions do get difficult to test locally. But This Localstack emulator may solve those problems (but I haven't used it yet) Details: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-local-invoke.html https://docs.aws.amazon.com/serverless-application-model/lat...
- mcintyre1994 5y ago
  I’ve used localstack to mock things like Dynamo, S3 and Step Functions for dev/integration testing of lambdas built with SAM with great results. I actually didn’t know that SAM stubbed those out by default and haven’t used it for a while, but I found that if you point your AWS library at localstack:4566 then it’ll run it all against localstack correctly.
- Hikikomori 5y ago
  They also support local stepfunctions.
sverhagen 5y ago
It frustrates me how they always go for the biggest common denominator there, the lowest level shared by everyone, but what's hardly used by professional users: don't let me download a jar file, please give me Maven coordinates. Don't tell me how to click in the console to get an EC2 instance, please give me an example Terraform snippet. Don't just show me how AWS CLI returns a blob of JSON when I assume role, please tell me how to use aws-mfa properly. I understand and respect their hands off approach to these more opinionated tools. But ergo, I tend to skip over the Amazon properties when finding what I need in the search results! PS: Localstack is awesome! Also in combination with Testcontainers.
- whummer 5y ago
  +100 Another example to add to your list - the new AWS CLI v2: > don't force me to download a 30MB binary with embedded Python interpreter baked in, please just give us a pip package instead! ;)
  - Sanguinaire 5y ago
    Or better yet, rewrite it in Go to avoid both the fat blob and python dependency management at the same time!
    - whummer 5y ago
      Touché!
    - chrsig 5y ago
      i mean, the go binary would still probably be a bit large, but having a statically linked binary to copy around would be welcome.
    - onetom 5y ago
      Or put the API definitions into some database (sqlite, datascript + nodejs or babashka or nbb), then you can decide what http library or mocked http solution would you like to use, eg https://github.com/oliyh/martian https://github.com/oliyh/martian Most of the code is just a thin veneer over that information...
zokier 5y ago
I general I do agree, but otoh I'm not sure if local appliances are the solution. I believe AWS' vision is that development happens against real cloud resources, which seems neat on paper but to that to really work provisioning resources needs to be super smooth to easily create developer sandboxes. CloudFormation definitely is not it.

by chrsig 5y ago

emersonrsantos 5y ago

I used Serverless framework to develop locally many times, it works very well for my case.

thrauat 5y ago

i think cloud providers have little incentive to help you develop and test everything locally, since they benefit more from onboarding you quickly into using their infrastructure, which they can much better monetize on. it seems to me that a completely local cloud stack would be threatening to cloud providers also in that you can then start running non-critical applications on-prem at no cost.

chrsig 5y ago
i don't think that's a real threat -- i don't think development environments are likely a huge income source for them, either. i think they'd be better served by letting me develop more quickly, so I can help my company grow, and presumably need (and have the budget for!) more resources in our production environment. if i'm mistaken, i think it's reasonable to have some sort of built in time limit to how long the appliance will run for, or some other way of preventing it being used for non-development purposes.

GusEggert 5y ago

There is also Step Functions Local (https://docs.aws.amazon.com/step-functions/latest/dg/sfn-local.html https://docs.aws.amazon.com/step-functions/latest/dg/sfn-loc..., it is also integrated with LocalStack). I built it as a side project when I was at AWS, motivated by the same frustrations. The best way to get these prioritized is to request it through your account manager/SA/etc. AWS does listen closely to customer feature requests.

mcbain 5y ago

It isn’t what you are asking for, but I was a bit surprised/happy to discover that the Ruby AWS SDK has a good stub client implementation: https://aws.amazon.com/blogs/developer/advanced-client-stubbing-in-the-aws-sdk-for-ruby-version-3/ https://aws.amazon.com/blogs/developer/advanced-client-stubb... I found it useful when updating some existing old ruby scripts we had. (Of course they had little to no testing before that.)

jacurtis 5y ago

In addition to local dynamo, they also have local AWS Lambda execution via AWS SAM (Serverless Application Model). You invoke it via the command line `sam local invoke [OPTIONS] [FUNCTION_LOGICAL_ID]` Behind the scenes it is running an attached docker container and feeding you the output. I found that it works actually very well and mimics the cloud invocation environment perfectly. The problem is that it stubs out any other services you run. So if your Lambda adds something to SQS or SNS or SES or S3 or anything else, it simply stubs those out. So advanced functions do get difficult to test locally. But This Localstack emulator may solve those problems (but I haven't used it yet) Details: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-local-invoke.html https://docs.aws.amazon.com/serverless-application-model/lat...

mcintyre1994 5y ago
I’ve used localstack to mock things like Dynamo, S3 and Step Functions for dev/integration testing of lambdas built with SAM with great results. I actually didn’t know that SAM stubbed those out by default and haven’t used it for a while, but I found that if you point your AWS library at localstack:4566 then it’ll run it all against localstack correctly.
Hikikomori 5y ago
They also support local stepfunctions.

sverhagen 5y ago

It frustrates me how they always go for the biggest common denominator there, the lowest level shared by everyone, but what's hardly used by professional users: don't let me download a jar file, please give me Maven coordinates. Don't tell me how to click in the console to get an EC2 instance, please give me an example Terraform snippet. Don't just show me how AWS CLI returns a blob of JSON when I assume role, please tell me how to use aws-mfa properly. I understand and respect their hands off approach to these more opinionated tools. But ergo, I tend to skip over the Amazon properties when finding what I need in the search results! PS: Localstack is awesome! Also in combination with Testcontainers.

whummer 5y ago
+100 Another example to add to your list - the new AWS CLI v2: > don't force me to download a 30MB binary with embedded Python interpreter baked in, please just give us a pip package instead! ;)
- Sanguinaire 5y ago
  Or better yet, rewrite it in Go to avoid both the fat blob and python dependency management at the same time!
  - whummer 5y ago
    Touché!
  - chrsig 5y ago
    i mean, the go binary would still probably be a bit large, but having a statically linked binary to copy around would be welcome.
  - onetom 5y ago
    Or put the API definitions into some database (sqlite, datascript + nodejs or babashka or nbb), then you can decide what http library or mocked http solution would you like to use, eg https://github.com/oliyh/martian https://github.com/oliyh/martian Most of the code is just a thin veneer over that information...

zokier 5y ago

I general I do agree, but otoh I'm not sure if local appliances are the solution. I believe AWS' vision is that development happens against real cloud resources, which seems neat on paper but to that to really work provisioning resources needs to be super smooth to easily create developer sandboxes. CloudFormation definitely is not it.