3 ms·

Interesting and thanks for the feedback! - Yes, either cloud functions (like Vercel's `/api` approach that we love) or docker containers (something like Google

by elitan 6y ago

Interesting and thanks for the feedback! - Yes, either cloud functions (like Vercel's `/api` approach that we love) or docker containers (something like Google Cloud Run) will be one three big features we'll be working on next. - Noted! - Not right now. But as soon as we start getting those requests from customers we'll prioritise it. - Noted! Again, thanks for the feedback!

nicoburns 6y ago
> Yes, either cloud functions (like Vercel's `/api` approach that we love) or docker containers (something like Google Cloud Run) will be one three big features we'll be working on next I'd love to use one of these serverless solutions (cloud run in particular looks great). But there is a key problem that they seem to share: cold starts. Our app doesn't see a lot of traffic, so we can happily run the entire thing on the equivalent of a small VM. If there was a massive traffic spike then I guess this would fall over, but we're an enterprise app so this is unliekly and we could fix that pretty quickly by provisioning more servers (in heroku this is as simple as dragging a slider). But even if we couldn't, falling over in abnormal conditions seems greatly preferable to there being a good chance of 3 seconds+ of latency being added to a normal everyday request. If you can solve this then you're on to a winner.
- elitan 6y ago
  Interesting. Are cold starts such a good problem? Cold starts with Node (Javascript) seems to be around 200ms. But the best thing would probably be to have it as an option. Scale down to 0 (cold starts) or 1 (no cold starts).
  - nicoburns 6y ago
    200ms would likely be fine (although not ideal), but I've seen more like to 3000ms with Firebase Cloud Functions, and have heard stories of similar with AWS Lambda. The fundamental problem seems to be the VM-or-container-per-request model. That means as soon as you get an extra concurrent user you have another cold start even if you already had one resource going. Whereas with a classic backend one server (even a small one) can handle thousands of concurrent requests and thus you have plenty of breathing room to start up another server before the current one gets completely bogged down.

by elitan 6y ago

nicoburns 6y ago

> Yes, either cloud functions (like Vercel's `/api` approach that we love) or docker containers (something like Google Cloud Run) will be one three big features we'll be working on next I'd love to use one of these serverless solutions (cloud run in particular looks great). But there is a key problem that they seem to share: cold starts. Our app doesn't see a lot of traffic, so we can happily run the entire thing on the equivalent of a small VM. If there was a massive traffic spike then I guess this would fall over, but we're an enterprise app so this is unliekly and we could fix that pretty quickly by provisioning more servers (in heroku this is as simple as dragging a slider). But even if we couldn't, falling over in abnormal conditions seems greatly preferable to there being a good chance of 3 seconds+ of latency being added to a normal everyday request. If you can solve this then you're on to a winner.

elitan 6y ago
Interesting. Are cold starts such a good problem? Cold starts with Node (Javascript) seems to be around 200ms. But the best thing would probably be to have it as an option. Scale down to 0 (cold starts) or 1 (no cold starts).
- nicoburns 6y ago
  200ms would likely be fine (although not ideal), but I've seen more like to 3000ms with Firebase Cloud Functions, and have heard stories of similar with AWS Lambda. The fundamental problem seems to be the VM-or-container-per-request model. That means as soon as you get an extra concurrent user you have another cold start even if you already had one resource going. Whereas with a classic backend one server (even a small one) can handle thousands of concurrent requests and thus you have plenty of breathing room to start up another server before the current one gets completely bogged down.