6 ms·

Yes, the open source Next.js doesn't include support for deploying middleware to the edge. If you run `next start` it runs middleware in the main server process

Yes, the open source Next.js doesn't include support for deploying middleware to the edge. If you run `next start` it runs middleware in the main server process. I work at Netlify and to support running Next middleware on edge functions we had to write our own open source integration for it, just as we did for deploying Next to lambda functions.

lagrange77 4y ago
As someone who has never used serverless functions before.. Netlify functions are based on AWS lambda functions, which spin up a container with say a node.js runtime environment everytime a request (or another event) comes in, then executes the user provided node.js function within the runtime. I can imagine that handy for something like cronjobs or api requests which happen occasionally. But using Netlify functions for things like Next.js SSR (or API routes) means, that for every single website request, a new container is setup, the function is run once, and the container is discarded. Am i right in this? Isn't that a huge overhead in contrast to a long running server process?
- mcintyre1994 4y ago
  Vercel uses AWS lambda under the hood as well, so it's probably comparable. AWS lambda is a bit smarter than what you said, it keeps containers around for some time able to respond to more requests. There's also concurrency, so if there's a lot of requests it'll keep a lot of containers running to deal with them, and then spin them down as the concurrent requests stop. The exact details are AWS secret sauce AFAIK, but Vercel claim in their docs that under 1% of requests get a cold start which is where a container needs to be setup to handle that request. https://vercel.com/support/articles/how-can-i-improve-serverless-function-lambda-cold-start-performance-on-vercel https://vercel.com/support/articles/how-can-i-improve-server...
  - cramforce 4y ago
    Disclaimer: Chief Architect @ Vercel This is a good summary. Edge Functions bring another set of trade-offs into the mix: - Startup-time so fast that if you hit a "cold start" it is still fast enough from a human perception (often <30ms attributable to startup) - Global by default - Cheaper on a per invocation - No limits on concurrent invocations BUT - Restricted API set, no node.js API support like Buffers or filesystem access - Smaller max binary size - Lower available CPU quota per request - Lower max RAM usage - Lower CPU-
    - ascorbic 4y ago
      Congrats on GA! I'd add to the pros that these edge runtime are a lot more standards-compliant than Node too. Users do get conufused about the CPU quota thing. They compare the 10ms or so that you get with edge functions, vs 10 seconds on lambda and think it's loads worse, without realising they're comparing apples and oranges. Both Cloudflare and Deno Deploy are limiting CPU time, whereas Lambdas are limited on wall clock time. I've had no trouble keeping long-running Deno Deploy functions running for far longer than a Lambda could run, e.g. when using Server-Sent Events or passing-through a large file download. As long as they're blocking on IO rather than CPU then you're fine. As long as you're not doing something like trying to process images, you're much more likely to hit a 10s lambda timeout than a 10ms CLoudflare or Deno limit.
- woadwarrior01 4y ago
  What you're describing is the cold start[1] request lifecycle for Lambda functions. But if a successive request comes soon thereafter, there's a faster warm start path, which is significantly quicker (~ <100ms). There's also a way to "provision"[2] resources to ensure that the function always stays warm, but that adds to the fixed cost, which is otherwise, near zero, if there are no requests. [1]: https://aws.amazon.com/blogs/compute/operating-lambda-performance-optimization-part-1/ https://aws.amazon.com/blogs/compute/operating-lambda-perfor... [2]: https://aws.amazon.com/blogs/aws/new-provisioned-concurrency-for-lambda-functions/ https://aws.amazon.com/blogs/aws/new-provisioned-concurrency...
- lagrange77 4y ago
  Okay, thank you guys. I think it's quite hard to estimate, weather using it is a good idea, ux/cost ratio wise.
- ascorbic 4y ago
  (Netlify engineer here) To echo what the others have said, lambdas are a bit smarter than that. They actually keep the node process running until it has been idle for around 5-15 minutes. We can make use of that ourselves by reusing the Next server too. We even cache data locally in the lambda tmpdir (though we do that more for Gatsby). The cold start issue though is one of the reasons that using isolates rather than Node-based functions is so great. Both Cloudflare Workers (used by Vercel) and Deno Deploy (used by Netlify) have extremely fast starts, even when cold. It's generally just a better fit for this model, which is why I'm so glad to see the approach becoming more popular and am genuinely please that Vercel's offering has gone GA. The more this model of edge computing spreads, the better. More framworks will add support, and it will be a virtuous cycle. The WinterCG work on creating a common standard for these runtimes is also a great project.
  - lagrange77 4y ago
    Read up on isolates. Really interesting stuff! It indeed seems to be a better and more natural fit for something like SSR route handlers, as far as i understand it. > More framworks will add support, and it will be a virtuous cycle. I'm in the early phase of developing a SSG in (TypeScript) Node currently. I think i'll study isolates and Deno a bit more, before going further. :)
    - itisit 4y ago
      V8 Isolates aren't exactly new (been around for about a decade); I'm sure the good folks at Netlify are up to speed on them. :) Containers were clearly the safer bet when FaaS options hit the scene 6-8 years ago. Now with WASM opening up possibilities, Cloudflare made the smart move to base their edge compute off isolates, given the significantly reduced overhead. Will be fun to see how the cloud competition responds!
      - ascorbic 4y ago
        I think they meant "I read up on isolates", rather than "You should read up on isolates"! I think you're exactly right though. Isolates are the way forward, and are gradually going to take more and more use cases from node-based functions.

lagrange77 4y ago

As someone who has never used serverless functions before.. Netlify functions are based on AWS lambda functions, which spin up a container with say a node.js runtime environment everytime a request (or another event) comes in, then executes the user provided node.js function within the runtime. I can imagine that handy for something like cronjobs or api requests which happen occasionally. But using Netlify functions for things like Next.js SSR (or API routes) means, that for every single website request, a new container is setup, the function is run once, and the container is discarded. Am i right in this? Isn't that a huge overhead in contrast to a long running server process?

mcintyre1994 4y ago
Vercel uses AWS lambda under the hood as well, so it's probably comparable. AWS lambda is a bit smarter than what you said, it keeps containers around for some time able to respond to more requests. There's also concurrency, so if there's a lot of requests it'll keep a lot of containers running to deal with them, and then spin them down as the concurrent requests stop. The exact details are AWS secret sauce AFAIK, but Vercel claim in their docs that under 1% of requests get a cold start which is where a container needs to be setup to handle that request. https://vercel.com/support/articles/how-can-i-improve-serverless-function-lambda-cold-start-performance-on-vercel https://vercel.com/support/articles/how-can-i-improve-server...
- cramforce 4y ago
  Disclaimer: Chief Architect @ Vercel This is a good summary. Edge Functions bring another set of trade-offs into the mix: - Startup-time so fast that if you hit a "cold start" it is still fast enough from a human perception (often <30ms attributable to startup) - Global by default - Cheaper on a per invocation - No limits on concurrent invocations BUT - Restricted API set, no node.js API support like Buffers or filesystem access - Smaller max binary size - Lower available CPU quota per request - Lower max RAM usage - Lower CPU-
  - ascorbic 4y ago
    Congrats on GA! I'd add to the pros that these edge runtime are a lot more standards-compliant than Node too. Users do get conufused about the CPU quota thing. They compare the 10ms or so that you get with edge functions, vs 10 seconds on lambda and think it's loads worse, without realising they're comparing apples and oranges. Both Cloudflare and Deno Deploy are limiting CPU time, whereas Lambdas are limited on wall clock time. I've had no trouble keeping long-running Deno Deploy functions running for far longer than a Lambda could run, e.g. when using Server-Sent Events or passing-through a large file download. As long as they're blocking on IO rather than CPU then you're fine. As long as you're not doing something like trying to process images, you're much more likely to hit a 10s lambda timeout than a 10ms CLoudflare or Deno limit.
woadwarrior01 4y ago
What you're describing is the cold start[1] request lifecycle for Lambda functions. But if a successive request comes soon thereafter, there's a faster warm start path, which is significantly quicker (~ <100ms). There's also a way to "provision"[2] resources to ensure that the function always stays warm, but that adds to the fixed cost, which is otherwise, near zero, if there are no requests. [1]: https://aws.amazon.com/blogs/compute/operating-lambda-performance-optimization-part-1/ https://aws.amazon.com/blogs/compute/operating-lambda-perfor... [2]: https://aws.amazon.com/blogs/aws/new-provisioned-concurrency-for-lambda-functions/ https://aws.amazon.com/blogs/aws/new-provisioned-concurrency...
lagrange77 4y ago
Okay, thank you guys. I think it's quite hard to estimate, weather using it is a good idea, ux/cost ratio wise.
ascorbic 4y ago
(Netlify engineer here) To echo what the others have said, lambdas are a bit smarter than that. They actually keep the node process running until it has been idle for around 5-15 minutes. We can make use of that ourselves by reusing the Next server too. We even cache data locally in the lambda tmpdir (though we do that more for Gatsby). The cold start issue though is one of the reasons that using isolates rather than Node-based functions is so great. Both Cloudflare Workers (used by Vercel) and Deno Deploy (used by Netlify) have extremely fast starts, even when cold. It's generally just a better fit for this model, which is why I'm so glad to see the approach becoming more popular and am genuinely please that Vercel's offering has gone GA. The more this model of edge computing spreads, the better. More framworks will add support, and it will be a virtuous cycle. The WinterCG work on creating a common standard for these runtimes is also a great project.
- lagrange77 4y ago
  Read up on isolates. Really interesting stuff! It indeed seems to be a better and more natural fit for something like SSR route handlers, as far as i understand it. > More framworks will add support, and it will be a virtuous cycle. I'm in the early phase of developing a SSG in (TypeScript) Node currently. I think i'll study isolates and Deno a bit more, before going further. :)
  - itisit 4y ago
    V8 Isolates aren't exactly new (been around for about a decade); I'm sure the good folks at Netlify are up to speed on them. :) Containers were clearly the safer bet when FaaS options hit the scene 6-8 years ago. Now with WASM opening up possibilities, Cloudflare made the smart move to base their edge compute off isolates, given the significantly reduced overhead. Will be fun to see how the cloud competition responds!
    - ascorbic 4y ago
      I think they meant "I read up on isolates", rather than "You should read up on isolates"! I think you're exactly right though. Isolates are the way forward, and are gradually going to take more and more use cases from node-based functions.