5 ms·

I made an image hosting tool on Lambda and S3, for internal corporate use. Staff can upload images to S3 via an SPA. The front end contacts the Lambda service t

I made an image hosting tool on Lambda and S3, for internal corporate use. Staff can upload images to S3 via an SPA. The front end contacts the Lambda service to request a pre-signed S3 upload URL, so the browser can upload directly to S3. It works really well. Observations: 1. Took too long to get something working. The common use case of hooking up a Lambda function to an HTTP endpoint is surprisingly fiddly and manual. 2. Very painful logging/monitoring. 3. The Node.js version of Lambda has a weird and ugly API that feels like it was designed by a comittee with little knowledge of Node.js idioms. 4. The Serverless framework produces a huge bundle unless you spend a lot of effort optimising it. It's also very slow to deploy incremental changes edit: – this is not only due to the large bundle size but also due to having to re-up the whole generated CloudFormation stack for most updates. 5. It was worth it in the end for making a useful little service that will exist forever with ultra-low running costs, but the developer experience could have been miles better, and I wouldn't want to have to work on that codebase again. --- Edit: here's the code: https://github.com/Financial-Times/ig-images-backend https://github.com/Financial-Times/ig-images-backend To address point 3 above, I wrote a wrapper function (in src/index.js) so I could write each HTTP Lambda endpoint as a straight async function that simply receives a single argument (the request event) and asynchronously returns the complete HTTP response. This wouldn't be good if you were returning a large response though; you'd probably be better streaming it.

2pointsomone 9y ago
Ditto.
weeefun 9y ago
Although I've just started using AWS Lambda and think it's a cool technology to use, I can understand your points. 2) Logging is indeed painful! You definitely need a separate tool/system for that. I have created a small CLI tool to view logs of multiple Lambdas which have been deployed using a CloudFormation template: https://github.com/seeebiii/lambdalogs https://github.com/seeebiii/lambdalogs This does not replace a good external system, but it can help for small searches in the logs. 4) Yes, this takes a lot. Though I'm not using the Serverless framework, deploying the code using CloudFormation takes me about 2:30 minutes (with a project using Java Lambdas), because CF is doing lots of checks in the background. I also wrote a tool for this to decrease the waiting time and just update the JS/Java code instead of the whole stack: https://github.com/seeebiii/lambda-updater https://github.com/seeebiii/lambda-updater Hope this helps you or someone else a bit!
- mcrider 9y ago
  For those using Serverless, being able to upload just the code (not a full Cloudformation stack) is built in, and will save you tons of time. Also, do what you can to limit/optimize your npm modules (if you're using node) since it has to be uploaded every time.
brango 9y ago
Chalice [1] makes hooking up lambda to HTTP endpoints easy, provided you don't mind using AWS API Gateway (and using python). I use it to handle a contact form on a static web site. It works really well. [1] https://github.com/awslabs/chalice https://github.com/awslabs/chalice
kesor 9y ago
The problem of logs is actually a problem of CloudWatch Logs being just not a very good service. A great way to solve that is to push all logs from CloudWatch Logs into an ElasticSearch cluster (using a Lambda function). AWS even has the code already done for you if you click the "subscribe" button in CWL. Then with Kibana/ElasticSearch the experience of inspecting and analysing logs is MUCH better.
- jamroom 9y ago
  This is what drives me nuts about AWS, but is genius on their part - they've got you to spend more $$ on ElasticSearch and Lambda to overcome the fact that one of their other products just doesn't work very well.
  - milesvp 9y ago
    I don't think this is intentional. My experience with AWS is that they aim to constantly improve products and create hosted services that are cheaper than if you frankensteined the same thing yourself. They don't always succeed, but I'm willing to give them the benefit of the doubt for how bad cloudwatch truly is, and just assume either they're blind to the pain because they know what not to do internally, or they're just really hamstrung in trying to modify that feature since so much relies on it. Hopefully this isn't just stockholm syndrome speaking though...
- adjohn 9y ago
  We built IOpipe[1] to address these issues by offering our own wrapper[2] that sends telemetry to our service. IOpipe aggregates metrics, and errors, and allows the creation of alerts with multiple rules per alert. [1] - https://iopipe.com https://iopipe.com [2] - https://github.com/iopipe/iopipe/ https://github.com/iopipe/iopipe/
  - munns 9y ago
    +1 to the folks at IOpipe. A really cool product that gives you very interesting visibility into your Lambda function executions!
z3t4 9y ago
how do you make sure it will run forever ? what about breaking changes and discontinued services ? can you move to another vendor without changes to your codebase ?
Prefinem 9y ago
1. I think a framework / tool will ensure this isn't an issue 2. Dear heavens yes. I ended up building a wrapper (similar to what you did to address 3) that handles logging, etc for any internal events. Everything else is a pass / fail check 3. Also had to build a wrapper. Context / Callback params are confusing 4. I wouldn't use Serverless unless you need it. Try something smaller. Apex is a nice, simple start. Shameless plug: I built a deployment tool because serverless wouldn't work for us and I wanted something in node (no binary like Apex - integrate into our build process) [1] [1]: https://github.com/Prefinem/lambdify https://github.com/Prefinem/lambdify
mikemann7121 9y ago
Hi, Most people failed to make money online because they don't have a proven system that works for them... Good news is, this system has helped more than 100 ordinary people, including newbies to make real money. You can be one of them too: http://rurl.us/v1GxM http://rurl.us/v1GxM The best part is... -You don't need to write or produce any unique content -You don't need to create any products to sell -You don't need to write up any sales letters -You don't need to provide any customer service or follow up support -You don't even need to learn about internet marketing! See how this can be done with real proof here: http://rurl.us/v1GxM http://rurl.us/v1GxM Don't be left behind.

2pointsomone 9y ago

Ditto.

weeefun 9y ago

Although I've just started using AWS Lambda and think it's a cool technology to use, I can understand your points. 2) Logging is indeed painful! You definitely need a separate tool/system for that. I have created a small CLI tool to view logs of multiple Lambdas which have been deployed using a CloudFormation template: https://github.com/seeebiii/lambdalogs https://github.com/seeebiii/lambdalogs This does not replace a good external system, but it can help for small searches in the logs. 4) Yes, this takes a lot. Though I'm not using the Serverless framework, deploying the code using CloudFormation takes me about 2:30 minutes (with a project using Java Lambdas), because CF is doing lots of checks in the background. I also wrote a tool for this to decrease the waiting time and just update the JS/Java code instead of the whole stack: https://github.com/seeebiii/lambda-updater https://github.com/seeebiii/lambda-updater Hope this helps you or someone else a bit!

mcrider 9y ago
For those using Serverless, being able to upload just the code (not a full Cloudformation stack) is built in, and will save you tons of time. Also, do what you can to limit/optimize your npm modules (if you're using node) since it has to be uploaded every time.

brango 9y ago

Chalice [1] makes hooking up lambda to HTTP endpoints easy, provided you don't mind using AWS API Gateway (and using python). I use it to handle a contact form on a static web site. It works really well. [1] https://github.com/awslabs/chalice https://github.com/awslabs/chalice

kesor 9y ago

The problem of logs is actually a problem of CloudWatch Logs being just not a very good service. A great way to solve that is to push all logs from CloudWatch Logs into an ElasticSearch cluster (using a Lambda function). AWS even has the code already done for you if you click the "subscribe" button in CWL. Then with Kibana/ElasticSearch the experience of inspecting and analysing logs is MUCH better.

jamroom 9y ago
This is what drives me nuts about AWS, but is genius on their part - they've got you to spend more $$ on ElasticSearch and Lambda to overcome the fact that one of their other products just doesn't work very well.
- milesvp 9y ago
  I don't think this is intentional. My experience with AWS is that they aim to constantly improve products and create hosted services that are cheaper than if you frankensteined the same thing yourself. They don't always succeed, but I'm willing to give them the benefit of the doubt for how bad cloudwatch truly is, and just assume either they're blind to the pain because they know what not to do internally, or they're just really hamstrung in trying to modify that feature since so much relies on it. Hopefully this isn't just stockholm syndrome speaking though...
adjohn 9y ago
We built IOpipe[1] to address these issues by offering our own wrapper[2] that sends telemetry to our service. IOpipe aggregates metrics, and errors, and allows the creation of alerts with multiple rules per alert. [1] - https://iopipe.com https://iopipe.com [2] - https://github.com/iopipe/iopipe/ https://github.com/iopipe/iopipe/
- munns 9y ago
  +1 to the folks at IOpipe. A really cool product that gives you very interesting visibility into your Lambda function executions!

z3t4 9y ago

how do you make sure it will run forever ? what about breaking changes and discontinued services ? can you move to another vendor without changes to your codebase ?

Prefinem 9y ago

1. I think a framework / tool will ensure this isn't an issue 2. Dear heavens yes. I ended up building a wrapper (similar to what you did to address 3) that handles logging, etc for any internal events. Everything else is a pass / fail check 3. Also had to build a wrapper. Context / Callback params are confusing 4. I wouldn't use Serverless unless you need it. Try something smaller. Apex is a nice, simple start. Shameless plug: I built a deployment tool because serverless wouldn't work for us and I wanted something in node (no binary like Apex - integrate into our build process) [1] [1]: https://github.com/Prefinem/lambdify https://github.com/Prefinem/lambdify

mikemann7121 9y ago

Hi, Most people failed to make money online because they don't have a proven system that works for them... Good news is, this system has helped more than 100 ordinary people, including newbies to make real money. You can be one of them too: http://rurl.us/v1GxM http://rurl.us/v1GxM The best part is... -You don't need to write or produce any unique content -You don't need to create any products to sell -You don't need to write up any sales letters -You don't need to provide any customer service or follow up support -You don't even need to learn about internet marketing! See how this can be done with real proof here: http://rurl.us/v1GxM http://rurl.us/v1GxM Don't be left behind.