6 ms·

Big +1 to “ Never give up on local testing”. The current code base I’m dealing with has many tests that require interaction with a dev environment on the cloud,

Big +1 to “ Never give up on local testing”. The current code base I’m dealing with has many tests that require interaction with a dev environment on the cloud, and occasionally these tests fail due to a timeout or some other thing not related to actually testing the code (and instead reveals that I forgot to refresh my MFA). Additionally the dependency thing can be huge; we ran into a weird bug for installing a particular dependency on our CI system so our test there keeps failing, but being able to run it locally let’s us know that the changes did not break our actual code tests.

anonymousDan 4y ago
How does this work in a serverless world? As far as I can tell local dev for most serverless environments that people actually care about is a joke.
- dijit 4y ago
  Usually there is some sort of local development framework for serveless functions. Most seem to use docker, here’s one for Google Cloud functions: https://cloud.google.com/functions/docs/running/overview https://cloud.google.com/functions/docs/running/overview If you use something like “cloud run” then it’s containers anyway.
- koblas 4y ago
  For golang, but would work in any language. What's really worked is the idea that your main loads configuration and injects all of the configured interfaces into your handler. You then can build all of the testing on mocks/stubs to test the behavior. If you access a database you access in through the interface which can mimic the appropriate behavior for your code in test vs production. If you need to you can do local integration testing of the db access layer.
- Hikikomori 4y ago
  With AWS SAM you can invoke functions locally, run api gateway and use step functions. There's also a 3rd party project to run various AWS services locally, can't remember the project name. There's also SAM sync that syncs changes you make quickly, so if you have your own dev copy in AWS you can quickly test changes.
  - tyingq 4y ago
    There's also a fair amount of lambdas where you can just invoke the lambda handler yourself without anything special other than a compatible version of python/node/etc. Variations of things like: node -e "console.log(require('./index').handler(require('./event.json')));"
- nine_k 4y ago
  Having used localstack [1], I can vouch that it's not a joke. [1]: https://github.com/localstack/localstack https://github.com/localstack/localstack
  - thrauat 4y ago
    thanks for saying that! people are pretty skeptical about localstack when they hear about it for the first time, and don't understand how we could ever emulate something remotely resembling AWS. tbh sometimes i'm baffled myself, it's pretty crazy what it can already do (i work there).
- methyl 4y ago
  For Cloudflare Workers, it's no longer a joke either: https://github.com/cloudflare/workerd https://github.com/cloudflare/workerd
- jasonhansel 4y ago
  Honestly, my answer is: don't use serverless, or at least have a non-serverless way of running your code locally, preferably outside of a container/VM. Otherwise you have no way of running or testing your code locally, much less attaching a debugger; IMHO this can slow down dev cycles so much that it cancels out the time saved by adopting a serverless architecture.
- viridian 4y ago
  Most people building stuff quickly in serverless seem to eschew the concept of local entirely, and it's something I've recently started doing too. I make a code change locally and it gets picked up by the deployment of my stack associated with my feature branch the second I save the file. It's an uncomfortable concept at first but I find it's helped me build things faster and has ultimately led to less rabbit hole chasing across multiple categories of issues.
- oceanplexian 4y ago
  Serverless deployments aren't painfully slow like they are with Kubernetes or other container orchestration platforms. At least with AWS lambda you can change and deploy your code in <10 seconds and have it taking traffic (In an infinite number of "environments"). So developing locally is kind of pointless.
  - FridgeSeal 4y ago
    > Serverless deployments aren't painfully slow like they are with Kubernetes What are you doing with your K8s cluster/containers that is making your deployments slow? My cluster at work pulls down containers faster than my local, and deployments are usually swapping within ~1 minute of being committed in git/the container build finishing…
recroad 4y ago
Then he contradicts it somewhat later by saying unit tests are the least important. I think he’s dead wrong there.
- nickm12 4y ago
  +1 unit tests are essential for software evolution like refactors, updating business logic, and updating dependencies. More generally, good testing requires a defense-in-depth strategy and unit testing, integration testing, and canary testing all have a role to play. All other things being equal, it's much better to catch a bug before you even push to the the source repo versus when it has one foot out the door to your customers.
- jasonhansel 4y ago
  He's not discouraging automated testing; he's just saying that we should prefer integration tests (using services running locally) to unit tests of individual functions. Look up "testing trophy" for more about this philosophy.
  - recroad 4y ago
    Totally with you on the value of integration tests. I'm familiar with the trophy and the triangle, but he's saying this: '..with unit tests notably coming in last place – i.e. “only if you have some time”.' The "only if you have time" part is what I disagree with. If we're doing TDD then I don't see how you can avoid writing unit tests, or even deprioritize them. It's also kind of weird to separate them like that, as Fowler discussed here: https://martinfowler.com/articles/2021-test-shapes.html https://martinfowler.com/articles/2021-test-shapes.html
    - pgwhalen 4y ago
      > The "only if you have time" part is what I disagree with. If we're doing TDD then I don't see how you can avoid writing unit tests, or even deprioritize them. Perhaps the author doesn't recommend TDD? They don't suggest anywhere that they do. It seems to me that the industry these days sees TDD as a tool for occasional use, not the dogma once proposed.

anonymousDan 4y ago

How does this work in a serverless world? As far as I can tell local dev for most serverless environments that people actually care about is a joke.

dijit 4y ago
Usually there is some sort of local development framework for serveless functions. Most seem to use docker, here’s one for Google Cloud functions: https://cloud.google.com/functions/docs/running/overview https://cloud.google.com/functions/docs/running/overview If you use something like “cloud run” then it’s containers anyway.
koblas 4y ago
For golang, but would work in any language. What's really worked is the idea that your main loads configuration and injects all of the configured interfaces into your handler. You then can build all of the testing on mocks/stubs to test the behavior. If you access a database you access in through the interface which can mimic the appropriate behavior for your code in test vs production. If you need to you can do local integration testing of the db access layer.
Hikikomori 4y ago
With AWS SAM you can invoke functions locally, run api gateway and use step functions. There's also a 3rd party project to run various AWS services locally, can't remember the project name. There's also SAM sync that syncs changes you make quickly, so if you have your own dev copy in AWS you can quickly test changes.
- tyingq 4y ago
  There's also a fair amount of lambdas where you can just invoke the lambda handler yourself without anything special other than a compatible version of python/node/etc. Variations of things like: node -e "console.log(require('./index').handler(require('./event.json')));"
nine_k 4y ago
Having used localstack [1], I can vouch that it's not a joke. [1]: https://github.com/localstack/localstack https://github.com/localstack/localstack
- thrauat 4y ago
  thanks for saying that! people are pretty skeptical about localstack when they hear about it for the first time, and don't understand how we could ever emulate something remotely resembling AWS. tbh sometimes i'm baffled myself, it's pretty crazy what it can already do (i work there).
methyl 4y ago
For Cloudflare Workers, it's no longer a joke either: https://github.com/cloudflare/workerd https://github.com/cloudflare/workerd
jasonhansel 4y ago
Honestly, my answer is: don't use serverless, or at least have a non-serverless way of running your code locally, preferably outside of a container/VM. Otherwise you have no way of running or testing your code locally, much less attaching a debugger; IMHO this can slow down dev cycles so much that it cancels out the time saved by adopting a serverless architecture.
viridian 4y ago
Most people building stuff quickly in serverless seem to eschew the concept of local entirely, and it's something I've recently started doing too. I make a code change locally and it gets picked up by the deployment of my stack associated with my feature branch the second I save the file. It's an uncomfortable concept at first but I find it's helped me build things faster and has ultimately led to less rabbit hole chasing across multiple categories of issues.
oceanplexian 4y ago
Serverless deployments aren't painfully slow like they are with Kubernetes or other container orchestration platforms. At least with AWS lambda you can change and deploy your code in <10 seconds and have it taking traffic (In an infinite number of "environments"). So developing locally is kind of pointless.
- FridgeSeal 4y ago
  > Serverless deployments aren't painfully slow like they are with Kubernetes What are you doing with your K8s cluster/containers that is making your deployments slow? My cluster at work pulls down containers faster than my local, and deployments are usually swapping within ~1 minute of being committed in git/the container build finishing…

recroad 4y ago

Then he contradicts it somewhat later by saying unit tests are the least important. I think he’s dead wrong there.

nickm12 4y ago
+1 unit tests are essential for software evolution like refactors, updating business logic, and updating dependencies. More generally, good testing requires a defense-in-depth strategy and unit testing, integration testing, and canary testing all have a role to play. All other things being equal, it's much better to catch a bug before you even push to the the source repo versus when it has one foot out the door to your customers.
jasonhansel 4y ago
He's not discouraging automated testing; he's just saying that we should prefer integration tests (using services running locally) to unit tests of individual functions. Look up "testing trophy" for more about this philosophy.
- recroad 4y ago
  Totally with you on the value of integration tests. I'm familiar with the trophy and the triangle, but he's saying this: '..with unit tests notably coming in last place – i.e. “only if you have some time”.' The "only if you have time" part is what I disagree with. If we're doing TDD then I don't see how you can avoid writing unit tests, or even deprioritize them. It's also kind of weird to separate them like that, as Fowler discussed here: https://martinfowler.com/articles/2021-test-shapes.html https://martinfowler.com/articles/2021-test-shapes.html
  - pgwhalen 4y ago
    > The "only if you have time" part is what I disagree with. If we're doing TDD then I don't see how you can avoid writing unit tests, or even deprioritize them. Perhaps the author doesn't recommend TDD? They don't suggest anywhere that they do. It seems to me that the industry these days sees TDD as a tool for occasional use, not the dogma once proposed.