4 ms·

At Cronitor we've just finished building out lambda infrastructure for a fan-out where I would otherwise use a worker application and supervisord. Some of this

At Cronitor we've just finished building out lambda infrastructure for a fan-out where I would otherwise use a worker application and supervisord. Some of this is lambda specific but I think it might be interesting nevertheless. Lambda has a lot of strengths, and it gives you solid primitives like immutable versions, version aliases/pointers and stateless functions. The problem I had was that it didn't give me any more than that. Here are just a few of the things we had figure out and then build ourselves: 1. The development workflow. How can I get a REPL workflow going that doesn't make me go crazy. 2. Deploys and Rollbacks. How can we safely deploy and rollback, especially in cases where you have the same lambda function in multiple regions and each region is at a different published version (because the version number is monotonically incrementing) 3. Permissions and Policies. The broad strokes are clear. But you want your S3 bucket or SNS topic to trigger lambda? Get ready to spend an hour trying to figure out what you've done wrong and what's missing from the vague directions. Hope you get that REPL flow solid first. In the end, we built several scripts (create_build, publish_version, promote_to_prod, etc) and we use these directly during development and from a Fabric-based deploy script. When I have time I plan to release this tooling open-source. If I had to do it all over again, I wouldn't. And I wouldn't use Cloud Functions. I would just use a t2.small instances with a simple worker.

ignoramous 11y ago
Valid points. Honest q: did you consider a framework like JAWS[0] for your installation? Don't any of those frameworks solve your problems out of the box? [0] https://github.com/serverless/serverless https://github.com/serverless/serverless
discodave 11y ago
What they need is Lambda + CodePipeline (or something like that). The ability model your whole application (Lambdas + tables + containers etc.) and have Alpha/Beta/Gamma/Prod with tests in between :)
viraptor 11y ago
My biggest issue is that (unless I'm missing something obvious), there's no way to add any configuration to the lambda. If I want different queues to do (slightly) different things, then I either need different code upload for each of them, or the parameter to be sent on the queue. I could not find any place where I can tell it - this is the code, this lambda uses it with N=5, that lambda uses it with N=42.
- glenjamin 11y ago
  I've spoken to an Amazon rep about this. The best approaches they were able to suggest were: * Lookup config from dynamo db when the lambda runs * Lookup config from S3 when the lambda runs * Bake config into the artifacts You can read the name of the alias you're currently running as from the context data, and use this as a key into some external system. Not ideal.
  - twagner 11y ago
    Suboptimal ;-). This is a common request, and we're looking at options for a config feature that's independent of the code payload. Tweet or email me detailed suggestions/asks for it: @timallenwagner or timwagne at amazon.com.
    - viraptor 11y ago
      Why not inject it to `context` parameter in some variable with known name? As long as the field is exposed to CF and in the panel, it should be all we need.
- encoderer 11y ago
  I struggled with this a bit at first. I moved all my config into my payload, which pleased me a lot, but only really works in the use case where Lambda is being triggered by a queue/topic message or by your direct invocation.
twagner 11y ago
This is great feedback encoderer. We're going to keep iterating fast to make deployment, packaging, and global (cross-region) updates even easier for AWS Lambda.

ignoramous 11y ago

Valid points. Honest q: did you consider a framework like JAWS[0] for your installation? Don't any of those frameworks solve your problems out of the box? [0] https://github.com/serverless/serverless https://github.com/serverless/serverless

discodave 11y ago

What they need is Lambda + CodePipeline (or something like that). The ability model your whole application (Lambdas + tables + containers etc.) and have Alpha/Beta/Gamma/Prod with tests in between :)

viraptor 11y ago

My biggest issue is that (unless I'm missing something obvious), there's no way to add any configuration to the lambda. If I want different queues to do (slightly) different things, then I either need different code upload for each of them, or the parameter to be sent on the queue. I could not find any place where I can tell it - this is the code, this lambda uses it with N=5, that lambda uses it with N=42.

glenjamin 11y ago
I've spoken to an Amazon rep about this. The best approaches they were able to suggest were: * Lookup config from dynamo db when the lambda runs * Lookup config from S3 when the lambda runs * Bake config into the artifacts You can read the name of the alias you're currently running as from the context data, and use this as a key into some external system. Not ideal.
- twagner 11y ago
  Suboptimal ;-). This is a common request, and we're looking at options for a config feature that's independent of the code payload. Tweet or email me detailed suggestions/asks for it: @timallenwagner or timwagne at amazon.com.
  - viraptor 11y ago
    Why not inject it to `context` parameter in some variable with known name? As long as the field is exposed to CF and in the panel, it should be all we need.
encoderer 11y ago
I struggled with this a bit at first. I moved all my config into my payload, which pleased me a lot, but only really works in the use case where Lambda is being triggered by a queue/topic message or by your direct invocation.

twagner 11y ago

This is great feedback encoderer. We're going to keep iterating fast to make deployment, packaging, and global (cross-region) updates even easier for AWS Lambda.