3 ms·

I'd like to understand a little more about the technical requirements here. 'Running with different credentials than the webserver' sure could've been solved wi

I'd like to understand a little more about the technical requirements here. 'Running with different credentials than the webserver' sure could've been solved with a fastcgi/wsgi setup and a decent way for expiration/monitoring of these processes (which the author admits failing to get his head around at the time of the implementation/writing of the blog post). As soon as you remove the startup time from the equation (by stopping to spawn a tool per request) the result is really just IO (writing a couple headers, serving a file) which _should_ perform quite similar in most applicable languages. While I'd like to see more (practical) Haskell content, this seems to be a lot of reinvention (see for example the mime-table commits). Whatever the language ended up being, was there no way to solve that issue more easily (and arguably, cleaner)?

chalst 15y ago
The code is pretty basic, and one can guess many of the requirements from it: http://andersk.mit.edu/gitweb/scripts-static-cat.git/blob_plain/refs/heads/master:/StaticCat.hs?js=1 http://andersk.mit.edu/gitweb/scripts-static-cat.git/blob_pl... The alternative to reinvention is usually borrowing other people's code, and it's often not clear that this will increase confidence in the correctness of the code. What I take from this post is a reminder that efficiency worries need not deter one from using languages like Haskell for this kind of task, despite the widespread belief that "CGI programs have to be small". The example would be that bit more compelling if the authors had used Quick Check to partly specify the code.
ezyang 15y ago
We understand the tuning parameters for these processes a bit more since this post was written a year and a half ago (We hit a magic phase shift for our other FastCGI processes, at which point our servers started falling over from 4000 users worth of FastCGIs. "Good thing we didn't listen to Geoffrey when he suggested we run an Apache for each user.") FastCGI static-cat should work (indeed, one of the original design intents was making this switch easy, if we decided to do it). But I disagree that language becomes irrelevant in this case: you still care about memory usage (since this will influence how many users you can support) and loading an entire interpreter can be pretty costly. I do agree that IO should be essentially comparable. As for the reinvention, all of these features are what you would normally find in Apache. But Apache is a large, complicated piece of software, and we don't want it to have access to arbitrary user files if it is compromised.
- darklajid 15y ago
  Thanks a lot. Of course you'll have done your homework, I just had the immediate 'Hmm.. Most of this exists already' reaction. Interpreter: Sure. I've no clue how many concurrent users you have (or is that the 4000 above?) - at some point that becomes a good point again. Anyway: Thanks a lot, appreciated the feedback directly from the source. :)
- zem 15y ago
  did you ever figure out why upx had such a large performance penalty?
deleted 15y ago
[deleted]
Craiggybear 15y ago
Same here. I must be missing something big here because none of this makes sense. Its reinvention for the sake of it.
sciurus 15y ago
There's more information at http://scripts.mit.edu/wiki/Technical_overview_of_scripts.mit.edu http://scripts.mit.edu/wiki/Technical_overview_of_scripts.mi...

chalst 15y ago

The code is pretty basic, and one can guess many of the requirements from it: http://andersk.mit.edu/gitweb/scripts-static-cat.git/blob_plain/refs/heads/master:/StaticCat.hs?js=1 http://andersk.mit.edu/gitweb/scripts-static-cat.git/blob_pl... The alternative to reinvention is usually borrowing other people's code, and it's often not clear that this will increase confidence in the correctness of the code. What I take from this post is a reminder that efficiency worries need not deter one from using languages like Haskell for this kind of task, despite the widespread belief that "CGI programs have to be small". The example would be that bit more compelling if the authors had used Quick Check to partly specify the code.

ezyang 15y ago

We understand the tuning parameters for these processes a bit more since this post was written a year and a half ago (We hit a magic phase shift for our other FastCGI processes, at which point our servers started falling over from 4000 users worth of FastCGIs. "Good thing we didn't listen to Geoffrey when he suggested we run an Apache for each user.") FastCGI static-cat should work (indeed, one of the original design intents was making this switch easy, if we decided to do it). But I disagree that language becomes irrelevant in this case: you still care about memory usage (since this will influence how many users you can support) and loading an entire interpreter can be pretty costly. I do agree that IO should be essentially comparable. As for the reinvention, all of these features are what you would normally find in Apache. But Apache is a large, complicated piece of software, and we don't want it to have access to arbitrary user files if it is compromised.

darklajid 15y ago
Thanks a lot. Of course you'll have done your homework, I just had the immediate 'Hmm.. Most of this exists already' reaction. Interpreter: Sure. I've no clue how many concurrent users you have (or is that the 4000 above?) - at some point that becomes a good point again. Anyway: Thanks a lot, appreciated the feedback directly from the source. :)
zem 15y ago
did you ever figure out why upx had such a large performance penalty?

deleted 15y ago

[deleted]

Craiggybear 15y ago

Same here. I must be missing something big here because none of this makes sense. Its reinvention for the sake of it.

sciurus 15y ago

There's more information at http://scripts.mit.edu/wiki/Technical_overview_of_scripts.mit.edu http://scripts.mit.edu/wiki/Technical_overview_of_scripts.mi...