3 ms·

This might be a bit of #1 and bit of something else, is involving compiled languages. I see teams who normally develop in Java, will then attempt to port that e

This might be a bit of #1 and bit of something else, is involving compiled languages. I see teams who normally develop in Java, will then attempt to port that entire codebase into a Lambda function and hit massive cold start times. Their solution, instead of splitting up the codebase (we need the reusability... but it's too much work to turn these parts into libraries...) is to start playing with provisioned concurrency. I think there's a mindshift required with Lambdas, which isn't immediately obvious. Anyone else encounter this and choose the easy way out - recommending they use Node/Python instead?

chriswarbo 5y ago
I understand your point, but I disagree with the characterisation of 'compiled versus interpreted' (which I've seen elsewhere in blog posts too). Java, C#, etc. are also interpreted languages; they just-so-happen to have a separate compile-to-bytecode step (unlike e.g. Python, which compiles to bytecode when a file is first imported); and their interpreters just-so-happen to have very slow startup times. "Compiled languages" can have very fast startup times, if we avoid languages like Java (which sacrifice a lot of startup time in favour of steady-state throughput; which is obviously inappropriate for a Lambda). If we do compile down to a native (x86_64 Linux) binary, it requires a custom runtime (e.g. if using Haskell, Rust, etc.).
- brabel 5y ago
  > and their interpreters just-so-happen to have very slow startup times. I was surprised to find out that Common Lisp (SBCL) is so incredibly fast to start up that it can run faster than Rust even starting from source. And Lisp is the archetypical dynamic langauge! Made me question my entire career working with JVM-based languages.
  - lispm 5y ago
    Many Lisp systems like SBCL basically restore the main memory (heap) with all data, caches and native code. Then they call a start function. No need to load/link/compile/initialize code at start.
- Zababa 5y ago
  > if we avoid languages like Java (which sacrifice a lot of startup time in favour of steady-state throughput Could you expand on that? I understand that both the JVM and Node are JIT compilers, but I don't understand why the JVM usually takes longer to start and what are the trade-offs involved here compared to Node.
  - chriswarbo 5y ago
    The main differences are probably the compilers involved, which tend to be layered based on how often a routine gets called (starting interpreted, then using a fast-but-dumb JIT compiler, then a slow-but-smart JIT compiler). There are many JVMs, each operating slightly differently, but the HotSpot JIT seems to be the most widely used (e.g. https://cl4es.github.io/2019/11/20/OpenJDK-Startup-Update.html https://cl4es.github.io/2019/11/20/OpenJDK-Startup-Update.ht... ) Node runs on V8, which is described in more detail at https://v8.dev/blog/ignition-interpreter https://v8.dev/blog/ignition-interpreter Another big diffence is that Java does lots of special handling for classes, e.g. hooking up 'reflection' machinery, allowing dynamic 'class loaders', etc. which happens during an application's startup. In contrast, Javascript 'classes' are basically just functions (constructors); and Javascript functions are values just like anything else; hence there's less of an up-front cost to JS code. Python works in pretty much the same way from a language level (but its CPython interpreter works very differently to V8).
  - jollybean 5y ago
    This is not the type of answer you want, but it is the answer: Java/JS startup times are different largely for product reasons. V8 is highly optimized for fast startups because that's necessary in loading web pages. JVM startup time is generally not an issue anywhere but Lambdas, also, JVM apps run for longer and so they want to take advantage of performance data to optimize over time. If Oracle was desperate to make JVM start fast as a matter of life-or-death for an important revenue stream, then JVM would start quickly. What the trade-offs would be I don't know, but keep in mind there might not have to be any, at least materially. Products are made to do certain things and not others, if something was never a requirement, nobody ever cared to think about it hugely material terms, well it's unlikely to happen. FYI one simple tradeoff is that JVM might not compile code until it's run a bit and optimized whereas V8 might immediately compile some things meaning the later you get 'pretty fast right away' but miss out on the runtime characteristics that make it optimally fast, so in the former you pay a little bit of 'learning time' up front and then get better optimization.
nuerow 5y ago
> Anyone else encounter this and choose the easy way out - recommending they use Node/Python instead? I feel your comment fails to take into account the main reason teams stick with Java in AWS Lambdas: code and dependency reuse, and consequently turnaround time. Arguing that nodejs and Golang are the ideal runtime in problems involving peeling tasks out of a monolith and into AWS Lambdas is a complete waste of time. Your goal is to offload processing tasks out of your service asap without wasting time developing and testing code written from scratch. Thus, you just create a project for your lambda, pick up your Java code that works and is well tested, offload the battle-hardened code and it's delendencies to the lambda package, add a handler to the lambda, add a lambda client to the monolith, sprinkle integ tests, and you're done: you have a production service. You're running a long-lived background task which is invoked only from time to time, and you don't really care if it takes 1 or 2 minutes to run. Additionally, do you really want to force your team to manage two distinct tech stacks just because you want to offload a process out of a service? What will that cost you?
FreakLegion 5y ago
Provisioned concurrency isn't worth it in my experience. You can just use, say, a CloudWatch rule to ping your desired number of instances every 5-odd minutes and keep them warm. 288 pings per day for 100 instances at 100ms per instance/ping is 2,880 seconds of execution time, or 5,760 GB-seconds at 2GB RAM (1 full vCPU and a bit). At $0.0000166667 per GB-second, that's $35 a year. You could cut that in half with smarter pings than e.g. [1], but it's not worth the engineering cycles at this price. 1. Python version: https://github.com/robhowley/lambda-warmer-py https://github.com/robhowley/lambda-warmer-py
giorgioz 5y ago
We do use monolithic lambdas written in Node.js and do not have cold start times problems. The problem with cold starts is more related to Java than to monolithic lambdas.

chriswarbo 5y ago

I understand your point, but I disagree with the characterisation of 'compiled versus interpreted' (which I've seen elsewhere in blog posts too). Java, C#, etc. are also interpreted languages; they just-so-happen to have a separate compile-to-bytecode step (unlike e.g. Python, which compiles to bytecode when a file is first imported); and their interpreters just-so-happen to have very slow startup times. "Compiled languages" can have very fast startup times, if we avoid languages like Java (which sacrifice a lot of startup time in favour of steady-state throughput; which is obviously inappropriate for a Lambda). If we do compile down to a native (x86_64 Linux) binary, it requires a custom runtime (e.g. if using Haskell, Rust, etc.).

brabel 5y ago
> and their interpreters just-so-happen to have very slow startup times. I was surprised to find out that Common Lisp (SBCL) is so incredibly fast to start up that it can run faster than Rust even starting from source. And Lisp is the archetypical dynamic langauge! Made me question my entire career working with JVM-based languages.
- lispm 5y ago
  Many Lisp systems like SBCL basically restore the main memory (heap) with all data, caches and native code. Then they call a start function. No need to load/link/compile/initialize code at start.
Zababa 5y ago
> if we avoid languages like Java (which sacrifice a lot of startup time in favour of steady-state throughput Could you expand on that? I understand that both the JVM and Node are JIT compilers, but I don't understand why the JVM usually takes longer to start and what are the trade-offs involved here compared to Node.
- chriswarbo 5y ago
  The main differences are probably the compilers involved, which tend to be layered based on how often a routine gets called (starting interpreted, then using a fast-but-dumb JIT compiler, then a slow-but-smart JIT compiler). There are many JVMs, each operating slightly differently, but the HotSpot JIT seems to be the most widely used (e.g. https://cl4es.github.io/2019/11/20/OpenJDK-Startup-Update.html https://cl4es.github.io/2019/11/20/OpenJDK-Startup-Update.ht... ) Node runs on V8, which is described in more detail at https://v8.dev/blog/ignition-interpreter https://v8.dev/blog/ignition-interpreter Another big diffence is that Java does lots of special handling for classes, e.g. hooking up 'reflection' machinery, allowing dynamic 'class loaders', etc. which happens during an application's startup. In contrast, Javascript 'classes' are basically just functions (constructors); and Javascript functions are values just like anything else; hence there's less of an up-front cost to JS code. Python works in pretty much the same way from a language level (but its CPython interpreter works very differently to V8).
- jollybean 5y ago
  This is not the type of answer you want, but it is the answer: Java/JS startup times are different largely for product reasons. V8 is highly optimized for fast startups because that's necessary in loading web pages. JVM startup time is generally not an issue anywhere but Lambdas, also, JVM apps run for longer and so they want to take advantage of performance data to optimize over time. If Oracle was desperate to make JVM start fast as a matter of life-or-death for an important revenue stream, then JVM would start quickly. What the trade-offs would be I don't know, but keep in mind there might not have to be any, at least materially. Products are made to do certain things and not others, if something was never a requirement, nobody ever cared to think about it hugely material terms, well it's unlikely to happen. FYI one simple tradeoff is that JVM might not compile code until it's run a bit and optimized whereas V8 might immediately compile some things meaning the later you get 'pretty fast right away' but miss out on the runtime characteristics that make it optimally fast, so in the former you pay a little bit of 'learning time' up front and then get better optimization.

nuerow 5y ago

> Anyone else encounter this and choose the easy way out - recommending they use Node/Python instead? I feel your comment fails to take into account the main reason teams stick with Java in AWS Lambdas: code and dependency reuse, and consequently turnaround time. Arguing that nodejs and Golang are the ideal runtime in problems involving peeling tasks out of a monolith and into AWS Lambdas is a complete waste of time. Your goal is to offload processing tasks out of your service asap without wasting time developing and testing code written from scratch. Thus, you just create a project for your lambda, pick up your Java code that works and is well tested, offload the battle-hardened code and it's delendencies to the lambda package, add a handler to the lambda, add a lambda client to the monolith, sprinkle integ tests, and you're done: you have a production service. You're running a long-lived background task which is invoked only from time to time, and you don't really care if it takes 1 or 2 minutes to run. Additionally, do you really want to force your team to manage two distinct tech stacks just because you want to offload a process out of a service? What will that cost you?

FreakLegion 5y ago

Provisioned concurrency isn't worth it in my experience. You can just use, say, a CloudWatch rule to ping your desired number of instances every 5-odd minutes and keep them warm. 288 pings per day for 100 instances at 100ms per instance/ping is 2,880 seconds of execution time, or 5,760 GB-seconds at 2GB RAM (1 full vCPU and a bit). At $0.0000166667 per GB-second, that's $35 a year. You could cut that in half with smarter pings than e.g. [1], but it's not worth the engineering cycles at this price. 1. Python version: https://github.com/robhowley/lambda-warmer-py https://github.com/robhowley/lambda-warmer-py

giorgioz 5y ago

We do use monolithic lambdas written in Node.js and do not have cold start times problems. The problem with cold starts is more related to Java than to monolithic lambdas.