4 ms·

That's an interesting problem. We don't get many concurrent requests and if we do, well then it's not a huge deal. How many instances do you want running? You

That's an interesting problem. We don't get many concurrent requests and if we do, well then it's not a huge deal. How many instances do you want running? You could set up a separate keep alive path that sends another request to the lambda, with a variable on how deep into the keep alive request 'recursion' and break out if you deep enough. Does that make sense? Super weird and just off the top of my head. edit: this isn't a good solution either because if you have a lambda kicking off 4 other lambdas because you want 5 running, and someone makes a request well then you still haven't warmed up that 6th and your 5 lambdas are running the keep warm code...

evrydayhustling 8y ago
If I understand your suggestion right, it's to heartbeat concurrently to force more warm instances. We have played with that, but spikes are spikes - the most interesting ones defy expectations. As with many apps, the conditions that make us spike make performance more important, not less. Just found the same author as OP with a clever solution here: https://read.acloud.guru/cold-starting-lambdas-2c663055589e https://read.acloud.guru/cold-starting-lambdas-2c663055589e Having the app pre warm instances on a per-user basis is super cool -- for user-driven workloads like web servers. To make matters worse, we are serving an API that takes hits from third party streams -- so our concurrency is based on their client behavior, not something we can easily link to a session scope, like users. Tricky!
- Flozzin 8y ago
  Yes. That's what I mean. The per user basis does sound interesting. Sometimes though, you can't force a square peg in a round hole. I dislike server maintenance but docker is a decent alternative to lambdas if you can absorb the extra cost.
  - evrydayhustling 8y ago
    Agreed, I think that's the state of the art: if variable concurrency is important, manage your own spare capacity. But I expect AWS and other providers will some day let us pay for reserved capacity without managing it, and I can't wait.
    - jnwatson 8y ago
      Yeah it seems a like a lot of resources are wasted on useless pings.
      - Flozzin 8y ago
        It does. But if your endpoint is so inactive that it sits idle most of the time, having it on a server/ec2 instance means you are paying 24/7 for it to sit there not doing anything. You could argue that it's not that much different to pay to keep the lambda warm vs paying for a server to be idle half the time.

evrydayhustling 8y ago

If I understand your suggestion right, it's to heartbeat concurrently to force more warm instances. We have played with that, but spikes are spikes - the most interesting ones defy expectations. As with many apps, the conditions that make us spike make performance more important, not less. Just found the same author as OP with a clever solution here: https://read.acloud.guru/cold-starting-lambdas-2c663055589e https://read.acloud.guru/cold-starting-lambdas-2c663055589e Having the app pre warm instances on a per-user basis is super cool -- for user-driven workloads like web servers. To make matters worse, we are serving an API that takes hits from third party streams -- so our concurrency is based on their client behavior, not something we can easily link to a session scope, like users. Tricky!

Flozzin 8y ago
Yes. That's what I mean. The per user basis does sound interesting. Sometimes though, you can't force a square peg in a round hole. I dislike server maintenance but docker is a decent alternative to lambdas if you can absorb the extra cost.
- evrydayhustling 8y ago
  Agreed, I think that's the state of the art: if variable concurrency is important, manage your own spare capacity. But I expect AWS and other providers will some day let us pay for reserved capacity without managing it, and I can't wait.
  - jnwatson 8y ago
    Yeah it seems a like a lot of resources are wasted on useless pings.
    - Flozzin 8y ago
      It does. But if your endpoint is so inactive that it sits idle most of the time, having it on a server/ec2 instance means you are paying 24/7 for it to sit there not doing anything. You could argue that it's not that much different to pay to keep the lambda warm vs paying for a server to be idle half the time.