6 ms·

AWS Lambda now supports up to 10 GB Ephemeral Storage

nunez 5y ago
this is fantastic for anyone wanting to run Chrome inside of Lambda (one of the biggest challenges is getting Chrome to fit)
- matt-attack 5y ago
  What's the use case? I see a sibling commenter mentioned puppeteer which I'm aware is essentially Chrome. Are you too referring to running puppeteer in Lambda? If so, what is the purpose?
  - mulhoon 5y ago
    Puppeteer-core can already fit in a lambda around 60mb and can be used for web scraping, taking screenshots, or performance testing.
    - atonse 5y ago
      How’s the overhead? Are you launching full chrome every lambda invocation?
      - mulhoon 5y ago
        In one lambda I can load chrome, visit a website, take a screenshot and save it to S3 in around 2s with around 280mb memory used. You could keep chrome open and reuse it if available, but you can't count on it with lambda being stateless. I use this Chromium binary https://www.npmjs.com/package/chrome-aws-lambda https://www.npmjs.com/package/chrome-aws-lambda
        vdm 5y ago
        > lambda being stateless /tmp is ephemeral in the sense that it dies with the Lambda instance, but it is also persistent between multiple serial invocations landing on that instance, even if the instance is frozen and thawed between some of those invocations. The process memory state on the instance (set up during the Init phase) is also available between invocations. https://docs.aws.amazon.com/lambda/latest/dg/runtimes-context.html https://docs.aws.amazon.com/lambda/latest/dg/runtimes-contex...
        zozbot234 5y ago
        You can't count on that freezing and thawing process, it's not guaranteed at all. The whole point of Lambda/FaaS in general is that each invocation might be truly ephemeral, depending on factors that are largely beyond any single user's control.
        nunez 5y ago
        this package requires using JS for your stack. your options are much slimmer for other languages.
    - nunez 5y ago
      Does that work outside of JS?
  - unixhero 5y ago
    Chrome headless
  - bl4ckm0r3 5y ago
    scalable pdf rendering
DevKoala 5y ago
I remember when I was breaking my head against the wall to fit puppeteer inside a lambda. Good times.
- otterley 5y ago
  What was the use case?
  - DevKoala 5y ago
    I was working for a programmatic advertising demand side platform. We had customers that submitted all sorts of ads to us and the lambda was in charge of rendering the ad and performing some validations on it. We accomplished it in the end using layers. It was in 2017, but it feels like a lifetime ago.
zozbot234 5y ago
If it's ephemeral to a function, it's not "storage". It's just core memory that may or may not be backed by physical "storage" media, and is organized as a tmpfs for the user's convenience.
- 10000truths 5y ago
  One man’s swap-backed tmpfs is another man’s page-cache-backed filesystem. /s
giorgioz 5y ago
This is also useful for deploying a big static website with AWS CDK which currently uses a Lambda and had the limit of 512mb (it was already possible to pass it by using an attached storage but this makes it easier).

this is fantastic for anyone wanting to run Chrome inside of Lambda (one of the biggest challenges is getting Chrome to fit)

matt-attack 5y ago
What's the use case? I see a sibling commenter mentioned puppeteer which I'm aware is essentially Chrome. Are you too referring to running puppeteer in Lambda? If so, what is the purpose?
- mulhoon 5y ago
  Puppeteer-core can already fit in a lambda around 60mb and can be used for web scraping, taking screenshots, or performance testing.
  - atonse 5y ago
    How’s the overhead? Are you launching full chrome every lambda invocation?
    - mulhoon 5y ago
      In one lambda I can load chrome, visit a website, take a screenshot and save it to S3 in around 2s with around 280mb memory used. You could keep chrome open and reuse it if available, but you can't count on it with lambda being stateless. I use this Chromium binary https://www.npmjs.com/package/chrome-aws-lambda https://www.npmjs.com/package/chrome-aws-lambda
      - vdm 5y ago
        > lambda being stateless /tmp is ephemeral in the sense that it dies with the Lambda instance, but it is also persistent between multiple serial invocations landing on that instance, even if the instance is frozen and thawed between some of those invocations. The process memory state on the instance (set up during the Init phase) is also available between invocations. https://docs.aws.amazon.com/lambda/latest/dg/runtimes-context.html https://docs.aws.amazon.com/lambda/latest/dg/runtimes-contex...
        zozbot234 5y ago
        You can't count on that freezing and thawing process, it's not guaranteed at all. The whole point of Lambda/FaaS in general is that each invocation might be truly ephemeral, depending on factors that are largely beyond any single user's control.
      - nunez 5y ago
        this package requires using JS for your stack. your options are much slimmer for other languages.
  - nunez 5y ago
    Does that work outside of JS?
- unixhero 5y ago
  Chrome headless
- bl4ckm0r3 5y ago
  scalable pdf rendering

DevKoala 5y ago

I remember when I was breaking my head against the wall to fit puppeteer inside a lambda. Good times.

otterley 5y ago
What was the use case?
- DevKoala 5y ago
  I was working for a programmatic advertising demand side platform. We had customers that submitted all sorts of ads to us and the lambda was in charge of rendering the ad and performing some validations on it. We accomplished it in the end using layers. It was in 2017, but it feels like a lifetime ago.

zozbot234 5y ago

If it's ephemeral to a function, it's not "storage". It's just core memory that may or may not be backed by physical "storage" media, and is organized as a tmpfs for the user's convenience.

10000truths 5y ago
One man’s swap-backed tmpfs is another man’s page-cache-backed filesystem. /s

giorgioz 5y ago

This is also useful for deploying a big static website with AWS CDK which currently uses a Lambda and had the limit of 512mb (it was already possible to pass it by using an attached storage but this makes it easier).