That depends on what you're trying to evaluate. If you want a language that can do CLI tools, you should absolutely include the startup time. If you're doing a

That depends on what you're trying to evaluate. If you want a language that can do CLI tools, you should absolutely include the startup time. If you're doing a long lived service, you should benchmark a warmed-up JVM. Different environment require different performance characteristics.