3 ms·

Even with warm disk caches, interpreted languages tend to have pretty slow startup times, because they spend a lot of CPU time doing the same stuff over and ove

Even with warm disk caches, interpreted languages tend to have pretty slow startup times, because they spend a lot of CPU time doing the same stuff over and over again. On my machine, the fastest of a dozen runs of just the interpreter is 66ms for Node.js (`time node -e ''`), and 14ms for CPython (`time python -c pass`). And my measurements a few years ago had overhead of about 1ms per module/file in Node.js, so if your source is spread over 100 files (which is honestly unrealistically low if you use any popular libraries and don’t bundle anything), your typical startup floor is now 200ms of CPU time, apart from disk time, on a good day. And some of those modules probably incur other costs, too. I have no figures on PHP, though if its execution model hasn’t changed from a decade ago most of this probably won’t affect it much because it effectively didn’t even persist modules. But I don’t know.

toast0 3y ago
As always, don't use languages that suck. On my system in hosting (FreeBSD 13.2-RELEASE, 2x Xeon L5640), time perl -e '' with perl5-5.34.3_3 reports 3ms. On the same system time php -r '' takes 16 ms with php82-8.2.11 and a handful of libraries installed. By my own criteria, php as a cgi sucks; but I did say if your program takes too long to startup, run it as a daemon, which I think includes linking it in like mod_php or mod_perl. On the topic of mod_php vs php_fcgi; with apache mod_php, an empty php file takes about 1.2 ms to serve, as measured by curl from when curl sends the GET request to when the content ends. I don't have a php fcgi server to try, but my recollection was something similar for minimum php output from lighttpd with php_fcgi on newer hardware.
- whartung 3y ago
  If you run an actual binary, the OS will not only cache the code pages, it will share them across processes. In the perl case, the actual interpreter is loaded, cached, and shared across the CGI invocations, while the actual script code will be cached from the disk, but each invocation will (likely) get their own copy of the code (i.e. they will duplicate it from cache when they read it in). Finally, of course, the runtime needs to start up again, compile/interpret the code, and, finally, execute it. A binary will have the fastest startup. I'm sure there may be things that could be done. I don't know if perl mmap'd the .pl file as a readonly buffer, if each invocation would share that file (probably), and thus prevent that initial duplication. Don't know if any of the interpreters do this. Odds are this gain is lost in the whole startup of the interpreter anyways. Also, using a binary, by a similar mechanic, lowers overall memory consumption. Have 100 connections all running the binary, they all share the code pages, and thus only "pay" for their individual data use. Share 100 connections with an interpreter, and they each pay their own cost for the entire program file plus any runtime data. Again, not so much a problem today with our ample memory resources, and mmap the source may remedy this, but this actually was an issue back in the day.
- chrismorgan 3y ago
  It may not be about any language implementation inferiority, but rather a question of what they’ve optimised for. Node.js is firmly designed for daemon operation, where having a high cost of instantiating a module, something that you only do at startup, is fine, if it means you later get better throughput because you’ve done JIT compilation or whatever. (It may also just be that Node.js is unnecessarily slow at these things because no one’s cared enough. But module loading being slow is reasonably well-known in the ecosystem at large so I doubt it’s particularly that.) PHP might have lower engine startup costs because of its history of CGI usage, and lower code loading costs because of its start-from-scratch-for-each-request model—but both are probably at a later runtime cost. Perl might have lower engine startup costs because it’s tended to be in a similar basket to PHP, plus more general scripting use, where you just want to get through things quickly: faster for one-offs, slower for repeated things. And my understanding of Perl is that it’s somewhat closer to just being an interpreter (which is excellent for latency but terrible for throughput) than are the other languages we’re discussing.
alright2565 3y ago
somewhat off-topic, but I want to share hyperfine[1], which is nearly purpose-built for measuring this kind of performance: $ hyperfine --shell=none "node -e ''" "python -c pass" "perl -e ''" "php -r ''" "bash -c ''" "echo" Benchmark 1: node -e '' Time (mean ± σ): 87.3 ms ± 8.0 ms [User: 69.8 ms, System: 18.4 ms] Range (min … max): 77.9 ms … 111.1 ms 31 runs Benchmark 2: python -c pass Time (mean ± σ): 24.4 ms ± 5.6 ms [User: 17.5 ms, System: 6.7 ms] Range (min … max): 15.8 ms … 31.3 ms 97 runs Benchmark 3: perl -e '' Time (mean ± σ): 2.8 ms ± 0.6 ms [User: 0.9 ms, System: 1.8 ms] Range (min … max): 1.7 ms … 3.9 ms 863 runs Benchmark 4: php -r '' Time (mean ± σ): 19.3 ms ± 4.2 ms [User: 8.9 ms, System: 10.0 ms] Range (min … max): 12.1 ms … 24.8 ms 241 runs Benchmark 5: bash -c '' Time (mean ± σ): 4.9 ms ± 0.8 ms [User: 2.8 ms, System: 1.8 ms] Range (min … max): 2.6 ms … 6.4 ms 552 runs Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. Benchmark 6: echo Time (mean ± σ): 1.2 ms ± 0.3 ms [User: 0.7 ms, System: 0.3 ms] Range (min … max): 0.8 ms … 2.1 ms 2237 runs Summary echo ran 2.41 ± 0.89 times faster than perl -e '' 4.17 ± 1.39 times faster than bash -c '' 16.37 ± 5.91 times faster than php -r '' 20.74 ± 7.64 times faster than python -c pass 74.21 ± 22.49 times faster than node -e '' (tested with node 21.5.0, python 3.11.6, perl 5.38.1, php 8.2.14, and bash 5.2.21 on Ryzen 2700X) [1]: https://github.com/sharkdp/hyperfine https://github.com/sharkdp/hyperfine
elric 3y ago
You don't have to use an interpreted language with CGI.
vidarh 3y ago
Interpreted languages often starts slow, yes. That is an issue with interpreted languages, not with CGI as a concept. If you trade low/no build time for high startup latency, by all means stay clear of CGI at all cost. I've built fast CGI based systems, but not recently, because I prefer the convenience of an interpreted language, so I am not arguing you should prefer CGI. Just that comparing a design pattern that is a consequence of not using CGI is a poor measure of CGI. If you were to seriously want to use CGIs, you wouldn't use runtimes that slow to start, or spread everything over hundreds of files, at least without a build step.

toast0 3y ago

As always, don't use languages that suck. On my system in hosting (FreeBSD 13.2-RELEASE, 2x Xeon L5640), time perl -e '' with perl5-5.34.3_3 reports 3ms. On the same system time php -r '' takes 16 ms with php82-8.2.11 and a handful of libraries installed. By my own criteria, php as a cgi sucks; but I did say if your program takes too long to startup, run it as a daemon, which I think includes linking it in like mod_php or mod_perl. On the topic of mod_php vs php_fcgi; with apache mod_php, an empty php file takes about 1.2 ms to serve, as measured by curl from when curl sends the GET request to when the content ends. I don't have a php fcgi server to try, but my recollection was something similar for minimum php output from lighttpd with php_fcgi on newer hardware.

whartung 3y ago
If you run an actual binary, the OS will not only cache the code pages, it will share them across processes. In the perl case, the actual interpreter is loaded, cached, and shared across the CGI invocations, while the actual script code will be cached from the disk, but each invocation will (likely) get their own copy of the code (i.e. they will duplicate it from cache when they read it in). Finally, of course, the runtime needs to start up again, compile/interpret the code, and, finally, execute it. A binary will have the fastest startup. I'm sure there may be things that could be done. I don't know if perl mmap'd the .pl file as a readonly buffer, if each invocation would share that file (probably), and thus prevent that initial duplication. Don't know if any of the interpreters do this. Odds are this gain is lost in the whole startup of the interpreter anyways. Also, using a binary, by a similar mechanic, lowers overall memory consumption. Have 100 connections all running the binary, they all share the code pages, and thus only "pay" for their individual data use. Share 100 connections with an interpreter, and they each pay their own cost for the entire program file plus any runtime data. Again, not so much a problem today with our ample memory resources, and mmap the source may remedy this, but this actually was an issue back in the day.
chrismorgan 3y ago
It may not be about any language implementation inferiority, but rather a question of what they’ve optimised for. Node.js is firmly designed for daemon operation, where having a high cost of instantiating a module, something that you only do at startup, is fine, if it means you later get better throughput because you’ve done JIT compilation or whatever. (It may also just be that Node.js is unnecessarily slow at these things because no one’s cared enough. But module loading being slow is reasonably well-known in the ecosystem at large so I doubt it’s particularly that.) PHP might have lower engine startup costs because of its history of CGI usage, and lower code loading costs because of its start-from-scratch-for-each-request model—but both are probably at a later runtime cost. Perl might have lower engine startup costs because it’s tended to be in a similar basket to PHP, plus more general scripting use, where you just want to get through things quickly: faster for one-offs, slower for repeated things. And my understanding of Perl is that it’s somewhat closer to just being an interpreter (which is excellent for latency but terrible for throughput) than are the other languages we’re discussing.

alright2565 3y ago

somewhat off-topic, but I want to share hyperfine[1], which is nearly purpose-built for measuring this kind of performance: $ hyperfine --shell=none "node -e ''" "python -c pass" "perl -e ''" "php -r ''" "bash -c ''" "echo" Benchmark 1: node -e '' Time (mean ± σ): 87.3 ms ± 8.0 ms [User: 69.8 ms, System: 18.4 ms] Range (min … max): 77.9 ms … 111.1 ms 31 runs Benchmark 2: python -c pass Time (mean ± σ): 24.4 ms ± 5.6 ms [User: 17.5 ms, System: 6.7 ms] Range (min … max): 15.8 ms … 31.3 ms 97 runs Benchmark 3: perl -e '' Time (mean ± σ): 2.8 ms ± 0.6 ms [User: 0.9 ms, System: 1.8 ms] Range (min … max): 1.7 ms … 3.9 ms 863 runs Benchmark 4: php -r '' Time (mean ± σ): 19.3 ms ± 4.2 ms [User: 8.9 ms, System: 10.0 ms] Range (min … max): 12.1 ms … 24.8 ms 241 runs Benchmark 5: bash -c '' Time (mean ± σ): 4.9 ms ± 0.8 ms [User: 2.8 ms, System: 1.8 ms] Range (min … max): 2.6 ms … 6.4 ms 552 runs Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. Benchmark 6: echo Time (mean ± σ): 1.2 ms ± 0.3 ms [User: 0.7 ms, System: 0.3 ms] Range (min … max): 0.8 ms … 2.1 ms 2237 runs Summary echo ran 2.41 ± 0.89 times faster than perl -e '' 4.17 ± 1.39 times faster than bash -c '' 16.37 ± 5.91 times faster than php -r '' 20.74 ± 7.64 times faster than python -c pass 74.21 ± 22.49 times faster than node -e '' (tested with node 21.5.0, python 3.11.6, perl 5.38.1, php 8.2.14, and bash 5.2.21 on Ryzen 2700X) [1]: https://github.com/sharkdp/hyperfine https://github.com/sharkdp/hyperfine

elric 3y ago

You don't have to use an interpreted language with CGI.

vidarh 3y ago

Interpreted languages often starts slow, yes. That is an issue with interpreted languages, not with CGI as a concept. If you trade low/no build time for high startup latency, by all means stay clear of CGI at all cost. I've built fast CGI based systems, but not recently, because I prefer the convenience of an interpreted language, so I am not arguing you should prefer CGI. Just that comparing a design pattern that is a consequence of not using CGI is a poor measure of CGI. If you were to seriously want to use CGIs, you wouldn't use runtimes that slow to start, or spread everything over hundreds of files, at least without a build step.