4 ms·
Having just written a bunch of scripts to populate a Google Sheet with some KPI data for the business folks, I _completely_ agree. At first I wanted to put my
by jasonmp85 10y ago
Having just written a bunch of scripts to populate a Google Sheet with some KPI data for the business folks, I _completely_ agree.
At first I wanted to put my scripts in Lambda, put an API Gateway in front, and have them be callable by the Google Sheet on a schedule. This seemed "pure" because I didn't really _need_ independent resources to do the task. The sheet could just update itself using an on-demand server!
Then I realized there was a 30-second max execution time. My scripts, in serial, couldn't hit that. I'd need to rewrite them all to be parallel. Also, they were a bunch of bash/jq/curl I'd just thrown together. I'd have to write a Python wrapper to shell out and handle streams (ugh).
At this point, I gave up and fell back to the advice of an (ex-Heroku) coworker: why not just run them in a cron job and put the data in a Heroku DB and use Heroku Dataclips to update the Google Sheet.
Sure I have to "have" an EC2 instance and a Heroku PG box, but they're both so small they're either free or essentially so. After cloning my script repos, everything worked right away.
Sometimes it's better just to do the "inelegant" thing and move on.
- teej 10y agoEmbarrassingly, I completed practically the exact same project but stuck it through with Lambda/API Gateway and just have it pull the precomputed data from a database. I wish I had known about Heroku Dataclips earlier! The build/test/deploy cycle for a Google Sheets add-on is a total nightmare.
- CWuestefeld 10y agoThen I realized there was a 30-second max execution time. Perhaps you're talking about a limitation on your spreadsheet, but the max execution time for a Lambda function is configurable up to 300 seconds.
- jasonmp85 10y agoI believe it may have been a limitation of the API Gateway + Lambda thing. Just checked the docs again and it says there is an "Integration Timeout" as follows: > 30 seconds for all integration types, including Lambda, Lambda proxy, HTTP, HTTP proxy, and AWS integrations. So not Lambda alone, but in conjunction with the API Gateway.
- colechristensen 10y agoOn the other side of the coin, a few months ago I was annoyed by getting a redshift alert one too many times during maintenance that I spent a couple of hours on a Saturday writing a ~20 line Lambda project that took the notifications from Redshift and posted them in a Slack channel. I didn't have to worry about setting up automation for launching an ec2 instance, monitoring it, paying for it, or any of the complex trouble for what ended up being a single python function that cost abotu $0.10 per month to run.