13 ms·
The beauty of CGI and simple design
- 082349872349872 4y agoI once got a term sheet on the basis of a "website" which was a quick-n-dirty lash-up of CGI, cron, and email. Low-code can be done not only with new tech but also with old.
- eknkc 4y agoI would ocassionally see dynamic web pages with .exe extensions before 2000s. I assume Windows IIS was also CGI capable and people were building dynamic web handlers in whatever language, compiling down to an executable. It makes perfect sense but always felt out of place to me for some reason.
- HeckFeck 4y agoI recall eBay used to have ISAPI.dll in most of its request URLs. It bewildered my younger self who thought DLLs were only for a Windows machine, not a web server.
- mrweasel 4y agoA previous employer of mine build a CMS for newspapers, written in Delphi and ran on Windows under IIS, it too was had a .dll extension. I believe it was the same IIS API (ISAPI) that made it work. It was an insane product that had simply failed to move forwards. It's not that you can't do what you need to do using the ISAPI, but the ecosystem is just to small for things like the Delphi stuff and keeping up with expectations of customers becomes to much work.
- HeckFeck 4y agoIt's impressive that anything that built the correct DLL could be leveraged by it. But I assume ISAPI didn't have anything like CPAN or pip. It seems likely you'd have to write everything yourself - perhaps worthwhile if you were a very large corporation with the manpower. But something as simple as PHP would swamp it for the rest of us.
- RedShift1 4y agoIIS was and still is CGI capable, but starting a process on Windows is a very heavy weight thing (compared to starting a process on linux), so it doesn't scale well and a noticeable initial delay whenever it executes. To solve that MS made ISAPI, which runs inside the webserver similar to an Apache httpd module.
- Semaphor 4y agoUntil early 2009, we ran a PHP website under IIS via an ISAPI FastCGI module that executed PHP :D
- cassepipe 4y agoIn the CGI RFC, IIRC they say that what they call a script can totally be a binary. The fact that execve on linux can take a script with a #! shebang makes it it easy to implement, you don't have to care whehter you are exec'ing a binary or a script.
- torstenvl 4y agoI love CGI and still use it all the time. It's natural MVC too which is great. Someone should make a framework like Electron or Tauri or NeutralinoJS that works like CGI. Neutralino on is probably the closest but... could be simpler. Just push everything over a pipe.
- hnlmorg 4y ago> Someone should make a framework like Electron or Tauri or WebKit that works like CGI. Don't need to use TypeScript or Rust or whatever. Just push between backend and frontend over a file stream. I’d argue that already exists in the form of your $SHELL.
- deleted 4y ago[deleted]
- torstenvl 4y agoIf you know a shell that renders HTML and JS as a local graphical UI, I'd be interested to know what it is! Though I'm not sure requiring end users to install such a niche thing (assuming it exists) is a great distribution strategy.
- hnlmorg 4y agoYou can get graphical UIs in shells. Albeit they’re usually TUIs. But there’s nothing stopping someone from creating a more ”attractive” UI using sixel or even embedding web components via proprietary ANSI escape sequences. Personally though, I don’t think HTML and JavaScript is a particularly nice abstraction for writing UIs. But each to their own
- RedShift1 4y agoCan you explain the "natural MVC" part?
- transfire 4y agoGTK Server (https://www.gtk-server.org/intro.html https://www.gtk-server.org/intro.html)
- HeckFeck 4y agoRecently I built a CGI script using Perl for my own pleasure. It was quite easy indeed, just a case of importing Template Toolkit, GD and CGI.pm then printing the result to stdout. Since I'm only using CGI.pm to send the HTTP header it isn't really necessary. The result is here: https://www.thran.uk/cgi-win/fleg.pl https://www.thran.uk/cgi-win/fleg.pl It is a flag generator affectionately titled flegger.
- nicky0 4y agoYou can’t claim us we live here
- foldr 4y agoEmirates probably shouldn't have a big cross in the middle of their flag :D But this was fun. My favorite was the "Socialist Caliphate of Flugshire".
- ilyt 4y agoLast time I used Perl for anything web it was via https://mojolicious.org/ https://mojolicious.org/ It even does event-based and websockets
- BlueTemplar 4y agoI found CGI to combine quite well with bleeding edge JavaScript 7 : async / await (compared to JS6 promises that left me confused as a JavaScript newbie). I am more interested to know in which situations it's a bad idea to use CGI ?
- diamondap 4y agoIn the old days (1990s), it was a "bad idea" to use CGI for heavily trafficked sites because there was a lot of system overhead involved in spawning a new CGI process for every request, firing up an interpreter (usually Perl), parsing the code, then executing it. As one of the previous comments mentions, operating systems do a better job of caching now, so there's less overhead there. Fast CGI also gets rid of that problem by running your CGI script as a single persistent process. CGI may not be the best solution for really huge applications that have hundreds of routes or very complex business logic. That said, it's still incredibly useful. I use it on some personal sites, and it's nice to know that I can drop it into any new site without having to worry about dependencies and without having to do much configuration. It's simple and it works. Those are two big pluses in a world where things have gotten so complex.
- samwillis 4y agoCGI was abandoned due to performance problems, but it is surprising scalable on modern OSs, the kernel does a good job of caching the code so it's not really a full round trip to the file system and cold start on each request. Obviously you still have the overhead of an interpreter for many languages, but they also tend to do a fairly good job of caching their byte code. I absolutely wouldn't recommend it for new systems, but for a quick and dirty script it's fine. The minute you want to do url routing and have middleware, a persistent process makes everything much simpler and keeps everything in one place. I much prefer routing inside a framework than using Mod_Rewrite. Edit: This article covers what I was thinking of: "Myths About CGI Scalability" http://z505.com/cgi-bin/qkcont/qkcont.cgi?p=Myths%20About%20CGI%20Scalability http://z505.com/cgi-bin/qkcont/qkcont.cgi?p=Myths%20About%20... Previously discussed: https://news.ycombinator.com/item?id=32699395 https://news.ycombinator.com/item?id=32699395
- geenat 4y agomod_rewrite hasn't been necessary since nginx. 2023 - Use single binary caddy. It's like 2 lines of config total to remove the extension from your CGI url's.
- samwillis 4y agoAbsolutely, and Caddy is brilliant. My point really was when having url routing, such as ids/slugs in the url, it is much easier to implement in your application code than in the web server configuration. It keeps application code in one place.
- liveoneggs 4y agoWhat does nginx have to do with mod_rewrite?
- ptx 4y agonginx hasn't been necessary since mpm_event_module.
- ilyt 4y ago
- Jabbles 4y agohttps://en.wikipedia.org/wiki/Common_Gateway_Interface https://en.wikipedia.org/wiki/Common_Gateway_Interface
- deleted 4y ago[deleted]
- geenat 4y agoYup been thinking of this a ton lately. Combined with single binary caddy and sqlite (+wal) gets you down to copy + paste production-ready deployments.
- juanse 4y agoAre there any good and practical tutorials about this?
- okasaki 4y agoEh, as someone who started with CGI, I think using something like Flask is easier than fiddling around with CGI.
- smetj 4y agoYup agreed
- nightfly 4y agoYou can use flask with CGI too ;) I did this last month for a website that only needed to live for a few weeks so it could piggyback off an existing Apache setup and require less work to setup and run.
- NoboruWataya 4y agoIf you're familiar with Python, sure. A great thing about CGI is that you could use it with Bash if you were so inclined.
- ilyt 4y agoThat kind of shit gives sysadmins nightmares
- int_19h 4y agoWait till you hear about CGI in PL/SQL. https://docs.oracle.com/cd/A97335_02/apps.102/a90099/feature.htm https://docs.oracle.com/cd/A97335_02/apps.102/a90099/feature... I kinda wonder if anyone actually ever used that in production.
- roywashere 4y agoI find it kind of interesting that people frowned upon CGI because it was not efficient. And yet here we are using lambas on AWS like it’s the best invention ever
- SoapSeller 4y agoDoesn't lambdas are more efficient than CGI due to ability to keep the runtime env running (JIT cache and all)?
- saurik 4y agoYeah: AWS Lambda feels a lot more like the things that came after CGI, not CGI.
- speedgoose 4y agoYes it’s like FastCGI. But expensive.
- scotty79 4y agoFunny how the word "expensive" when referring to code execution might today mean actual money, not just memory or time.
- pjmlp 4y agoThat is why with experience one learns to ignore fashion trends until they eventually either become unavoidable, or fade away from conference talks.
- agumonkey 4y agoI wonder what's the best way: try to extract profit from the fads (cynic) or isolate..
- squarefoot 4y agoNo web dev here, but if memory serves, the main concerns about CGI were about safety, not performance or efficiency. If that's the case, I wonder if new more safe languages could make its use viable again.
- revskill 4y agoBecause of PHP. Noone wants to have joy coding in PHP. There's a big difference between get shit done and enjoy it.
- scotty79 4y agoI don't think I ever ran PHP through CGI. It was either mod_php in apache2 or FastCGI. And I started around PHP 4
- int_19h 4y agoI first saw PHP around that time, too, and you could see it set up on CGI on some hosting providers; I think it was usually older ones who had Perl originally and then just tacked PHP on to follow demand.
- marban 4y agoI've built anything from CGI guestbooks to webrings in the '90s but IMO the sweet-spot for solo projects in terms of time to market and scalability/stability is the humble combo of Flask & Caddy.
- cassepipe 4y agoI really like Haserl (https://haserl.sourceforge.net/ https://haserl.sourceforge.net/). It acts as a middleman for CGI scripts and and makes it easy to write them with a shell or lua while keeping the look a HTTP response with a html body.
- synergy20 4y agosame here,especially on embedded boards
- cassepipe 4y agoThought I'd post the description : """ Haserl is a small cgi wrapper that allows "PHP" style cgi programming, but uses a UNIX bash-like shell or Lua as the programming language. It is very small, so it can be used in embedded environments, or where something like PHP is too big. It combines three features into a small cgi engine: It parses POST and GET requests, placing form-elements as name=value pairs into the environment for the CGI script to use. This is somewhat like the uncgi wrapper. It opens a shell, and translates all text into printable statements. All text within <% ... %> constructs are passed verbatim to the shell. This is somewhat like writing PHP scripts. It can optionally be installed to drop its permissions to the owner of the script, giving it some of the security features of suexec or cgiwrapper. """
- cassepipe 4y agoIIRC from when I wrote a basic http server, the problem with CGI is that you have to store the output of the script in memory in order to insert the Content Length field value in the http respondse header. It seems like a rather innefficient operation, especially if repeated a million times on a server. Else you could just read from the your CGI process in a non blocking manner, writing back to the socket whenever output is available. Am I correct ?
- est 4y agoNextup: inetd was better than sidecars.
- mmcgaha 4y agoI wrote a web app that launched from inetd to control my cd player. It worked great but wasn't super practical since I tended to use it from the same computer that it ran from. Everything web was fun in the 90s though.
- grose 4y agoRecently I enjoyed some "retrofuturistic" development with WASM and CGI. Spin[1], a webserver written in Rust, can execute WASI[2] binaries that speak CGI. You can then deploy it to Fermyon Cloud or your own server and it "just works". It's a wonderful mix of old and new. I used it for PHP (Prolog Home Page): https://github.com/guregu/php https://github.com/guregu/php [1]: https://spin.fermyon.dev/ https://spin.fermyon.dev/ [2]: WASI is a POSIX-ish standard for WASM that gives you all the low level stuff like standard input and output. It includes all the bits and pieces needed for CGI to work.
- bmacho 4y agoIt interprets PHP-style commands, hence the name.
- IshKebab 4y agoKind of feels like the AssemblyScript guys were right - WASI is becoming a standard for WASM modules that foists Unix on Webassembly, including stuff that isn't really portable like symlinks. That makes sense if you want to compile existing software to WASM but that isn't the case here is it?
- grose 4y agoThis does use existing software. The onus behind this project was to test my port of Trealla Prolog (which was written by someone else) to WASM. Without a Unixy standard environment it would have been a lot harder. In theory, WASI should enable many CGI-like usages of pre-existing software, but in my experience the quirks between the WASI implementations push you towards using nonstandard things anyway. For example, trealla-js is mostly WASI but uses a couple extra imported functions to coordinate calling into JS. If I wanted to let PHP make outgoing requests I'd have to implement Spin's custom nonstandard components. I hope the WASM Components proposal makes this easier in the future. Personally, I might have preferred a more Plan 9-style approach that uses special files to do things like make outgoing connections, purely because it would be trivial to implement client-side as a simple printf vs. integrating wit-bindgen into the build system and all of that. However, I can see many benefits of the strongly typed component model as well. I don't have a strong opinion on the AssemblyScript controversy[1]. I agree that WASI is imperfect, especially in browsers, but I'm also glad that _something_ exists with not-horrible runtime support. WASI was pretty nice to use as someone who just wanted to get a C program running on WASM. [1] https://www.assemblyscript.org/standards-objections.html https://www.assemblyscript.org/standards-objections.html
- rgrieselhuber 4y agoThis will sound weird but I felt a sense to relief that there are other people still thinking about the web in this manner.
- monkeycantype 4y agoI do contract work for a company that has it's pretty customer facing site as a mix of angular/react/react native and .net core apis and the less pretty internal staff site which is vanilla html, js and c++ cgi apis, so I get to work in both on the same day. The internal c++ site has been running for ~25 years. The customer site seems to need to be re-written every time we add a new feature, because all the tech is out of support and there are breaking changes to libraries
- bombela 4y ago> Personal projects remain unfinished or abandoned because I’ve wanted to do things the “right” way. So I start building it the “right” way with complicated tools and processes, then raise my hands and think it’s all too much for what I want to do. It puts all sorts of things out of reach. So true it hurts.
- rgrieselhuber 4y agoThis brings to mind some of Paul Virilio’s writings, who discussed this idea of speed (to include not only velocity but also the sensory assault that comes with complexity and speed combined. This has been paraphrased as “speed is a continuation of fear by other means.” How many framework decisions are made in this mindset?
- riesenheim 4y agoThanks for this fascinating connection. "History progresses at the speed of its weapons systems." https://en.wikipedia.org/wiki/Paul_Virilio#War_of_movement https://en.wikipedia.org/wiki/Paul_Virilio#War_of_movement
- rgrieselhuber 4y agoVirilio was a very interesting writer.
- ZephyrBlu 4y agoYeah, the primary goal of any side project should be doing whatever maximizes the chance you continue working on it. The lesson from survivorship bias is to optimize for survival :).
- marginalia_nu 4y agoYeah, I like the analogy of a march, as a contrast to the sprints typically employed in commercial development. Like my search engine project is 2 years in now, and still not anywhere near "done". Yet it keeps getting better and more capable at a slow but steady pace.
- rwmj 4y agoAs someone who wrote a large, commercial social network for schools in Perl + CGI.pm in ~ 2000, here were the problems: * Perl was very memory inefficient so you could often only serve 4 users at the same time, given the limited RAM available (eg. 512M - 1G). Note that Perl's speed was not a problem. I suspect this one has probably gone away with modern servers with massive amounts of RAM, plus the cloud allows you to quickly scale up and down with demand. Nevertheless using a compiled language would be better. (I later wrote a vastly more efficient framework in C). * Every action on the page had to round-trip to the server, and always involved a database check (at minimum to map the user's cookie to a user ID). This is sort of solved now that Javascript is widely supported, although that brings its own issues along. Also memcached neatly solves the database access problem. Our website was developed a little bit before memcached. * It's very clumsy and time-consuming to write any non-trivial CRUD-style action using CGI. eg. Just having a table with update/delete buttons is going to involve writing paging code, update form, update submit form (repeat for every possible action). This is the kind of thing that Ruby on Rails solved quite nicely. * Organizational problems interfacing with the database. We had to negotiate with the DBA for every schema change, which is a problem when the DBA is a sociopath supported by management. Devops sort of solves this (but also I appreciated not being on call). * The general hassle of dealing with the web, like setting all the non-obvious HTTP headers to make it secure. I think modern frameworks just deal with this, although I've not used them very much. However if I was to go back and change anything, it wouldn't have been to change the technology. It would have been to extend the service first to university students and later to the whole world :-)
- daneel_w 4y ago"Perl was very memory inefficient so you could often only serve 4 users at the same time, given the limited RAM available (eg. 512M - 1G). Note that Perl's speed was not a problem. I suspect this one has probably gone away with modern servers with massive amounts of RAM. Nevertheless using a compiled language would be better. (I later wrote a vastly more efficient framework in C)." I cannot even imagine the strange complexities of this circa year 2000 web application if it could only fit 4 simultaneous page loads on 1 GB of RAM. It's baffling, in fact. Perl isn't really memory hungry these days. Was it really that bad 20 years ago? I would be surprised if the Perl 5 interpreter wasn't in fact much leaner back then. At least my immediate impression of this bullet point is that it wasn't a programming language problem, but a software design problem.
- reddec 4y agoI felt exactly the same as the author. That's why I created https://trusted-cgi.reddec.net/ https://trusted-cgi.reddec.net/ (currently on the way to v1 and under the heavy refactoring)
- habibur 4y agoCGI is slow [ spawns an OS process for every request ]. But FCGI is as fast as any other solution, like compiling the whole app in a single binary and running it as a HTTP server.
- sgbeal 4y ago> CGI is slow .... CGI is _computationally_ slow, but in practice it's faster than it needs to be for most sites. Some examples of CGI-powered sites i'm aware of include: - https://sqlite.org/forum https://sqlite.org/forum - https://sqlite.org/wasm https://sqlite.org/wasm (same domain, different sub-site) - https://fossil-scm.org/home https://fossil-scm.org/home - https://fossil-scm.org/forum https://fossil-scm.org/forum (same domain, separate sub-site) - https://fossil.wanderinghorse.net https://fossil.wanderinghorse.net All CGI, all the time, and _plenty_ performant for the purpose.
- yakubin 4y agoAdd to it https://git.kernel.org/ https://git.kernel.org/
- WJW 4y agoMany people confuse "X is slower than Y" with "X is slow in some absolute sense". Starting a new process takes well under a millisecond on most modern machines, so that would be completely lost in the latency of most web requests. The real benefits of persistent processes are in getting to reuse database connections and the like.
- ptx 4y ago> Starting a new process takes well under a millisecond on most modern machines But if that process is Python or Java it's going to be tens to hundreds of milliseconds before it's ready to serve the request.
- alaaalawi 4y agoYMMV, but thats why i love Forth. it compiles the full app (not interpret) in ms. some think its dead or only for the embedded, but for me and my projects its a deadly combination + ultra fun.
- hestefisk 4y agoI wrote Perl cgi scripts for a self-serve customer admin interface for an ISP in the early 2000s. As users grew, we started hitting the limits of what 1-2G ram could do. Thank goodness for mod_perl and HTML::Mason, which could REALLY speed up performance. I remain a huge fan of Perl these days.
- papito 4y agoStill remember reloading a page and seeing another customer's forms. mod_perl teaches you quickly about state.
- cutler 4y agoI used Perl's CGI::Application to transform an Excel spreadsheet of teachers, used for fielding telephone enquiries, into a PostgreSQL-backed web service used by 33 household brands. Everything ran on Apache CGI without a hitch for 15 years.
- sgbeal 4y agoIt might interest readers to know that the Fossil SCM's primary mode of operation is CGI, and has been since it began life in 2007: https://fossil-scm.org/home https://fossil-scm.org/home When i first discovered fossil, in December 2007, its ability to run as a CGI was one of its two "killer features" for me, and it's still right at the top of its list of killer SCM features for me. (Disclaimer: i'm a long-time fossil contributor, but its CGI support pre-dates my joining the project.)
- synergy20 4y agois there a good fastcgi implementation somewhere,the "official" one was not maintained since years ago. https://fastcgi-archives.github.io/ https://fastcgi-archives.github.io/
- habibur 4y agoI have followed this tutorial and wrote my own. https://www.mit.edu/~yandros/doc/specs/fcgi-spec.html https://www.mit.edu/~yandros/doc/specs/fcgi-spec.html Easier than integrating a 3rd party tool.
- neilv 4y agoFastCGI is tricky to implement, which can mean server bugs you don't ever want to have to debug. SCGI is much simpler: http://python.ca/scgi/protocol.txt http://python.ca/scgi/protocol.txt I started implementing a FastCGI client library for Racket, and (though I have experience getting into the bytes and bits of protocol implementation) it seemed harder than it needed to be. So I decided to try SCGI first, and SCGI ended up working great in production: https://docs.racket-lang.org/scgi/ https://docs.racket-lang.org/scgi/
- calltrak 4y ago[dead]
- lucidguppy 4y agoI'm having fun with django + htmx and going to deploy on elastic beanstalk. It hits that sweetspot of speed of development and responsiveness. You also have the benefit of staying in one language. Django has so much out of the box that you don't have to decide for yourself after the fact. So much accumulated knowledge of what works too. Sure go ahead rewrite it if it starts making money and you can afford development time/salary. But these older technologies are easier to fit inside one head and let you get enough speed to take off.
- hit8run 4y agoThis is also a very sane combination. HTMX is great and Django is also very impressive :) We need tools that work for us not us working to make them work.
- account-5 4y ago>We need tools that work for us not us working to make them work. Stealing this for work, couldn't agree more.
- hit8run 4y ago:) Go ahead. Spread the message.
- elwebmaster 4y agoWasn’t that what Microsoft used to give us back in the days of Visual Studio, .NET and especially LightSwitch? Amazing technology, more than 10 years later we don’t have anything coming close to it for developer productivity.
- uhtred 4y agoThe only side project I've managed to complete and put live in the last 10 years used Perl and CGI. That was last year. It was actually fun!
- ihateolives 4y agoLike an actual new project in Perl? How is Perl in 2023? I remember mucking around with it in early 2000 but it didn't click with me then.
- soapdog 4y agoI remember CGI very fondly. I'd still use it to this day in my personal projects if it was not for the fact that I'm running vanilla Caddy 2 that comes without support for it. Now, CGI and FastCGI are cool, but who else here built CGIs for running on Pre-OSX Macs? I remember AppleScript/AppleEvents based CGI. The Web Server would be a GUI program running on the Mac and the CGI would also be one. They'd communicate to one another sending AppleEvents. I once made good money with a system for running school exams with it. I miss those days a bit.
- kefka_p 4y agoI wrote a few CGI applications for Schotton’s MacHTTP and the WebStar server on MacOS 7. Back before the days of OpenTransport! Yes, I’m old.
- TekMol 4y agoOne of the strengths of building web apps in PHP is that you have this nice "Every pageload starts from scratch" system. Much easier to develop and reason about. It's a petty that in Python, long running threads are the norm. I hope the Python ecosystem can slowly convert to a CGI style approach.
- hprotagonist 4y agonothing prevents you from spawning a new worker in every endpoint. it’s a nice way to orchestrate long running workers actually: the main process exists exclusively to keep the app up/ talking to gunicorn/uvicorn and register PIDs of a worker process in a queue. but we’re all in on wsgi, i don’t see that changing.
- TekMol 4y agoThe way I understand you, the Python script responsible for an endpoint will keep running for multiple requests? I prefer it when it just runs for a single request.
- VWWHFSfQ 4y ago> No dependencies or libraries to install, upgrade, maintain, test, and secure. So you don't actually want this program to do nearly anything useful unless you write it all yourself. > Zero external configuration, other than telling your webserver to enable CGI on your file. So now you have to have a webserver external dependency and also configure it. Configuring Apache for CGI securely was no fun unless you were an expert webmaster. People really put their rose-colored glasses on when they reminisce about this stuff. But there's a reason why we don't use it anymore. We don't need to make up nonsense about why it was so great.
- MintPaw 4y ago> So you don't actually want this program to do nearly anything useful unless you write it all yourself. I think the idea is that there's no required dependencies. Unless I misunderstand, you can take on whatever dependencies you want from your chosen language.
- myth2018 4y agoWith all due respect, I think you missed some points there. > > No dependencies or libraries to install, upgrade, maintain, test, and secure. This is mentioned as opposed to what we have today with Node.js, Pip and so on. Old CGI programs surely had dependencies. But dependency management was dramatically simpler. Of course there are pros and cons in the comparison. But simplicity is the key reminiscence here. > But there's a reason why we don't use it anymore. However I wonder: are all those reasons still valid? To what extent were those reasons motivated by rational decisions instead of pure trend-following? We shall not fool ourselves: not every technical decision we make have grounds on reason. I suspect most of them are not.
- iamben 4y agoI started writing my dissertation project in Perl CGI back in like 2001 or whatever it was. I moved to PHP pretty quickly when I realised I could do the same things in 5 lines that took 70 in Perl. I'd been making websites for about 6 years at that point and PHP just absolutely changed my life. It was, arguably, exactly like this post talks about CGI - put some script in a file, FTP to your host and you're good to go. Web development was simple and exciting - you could get a long way very quickly. I was talking to a friend the other day who wants to learn some web dev for fun - he had no clue where to start because there's now SO much, so many 'best practice' posts, so much 'you must learn like this'. I know the world changes (and often for the better), but it was a magic time when you could get so far so quickly. As an aside, it's why I have huge amounts of respect for people like Pieter Levels and his 'just get it done' approach to building things.
- giovannibonetti 4y agoPerhaps we should start making tutorials that follow that idea from PHP: start with HTML, add an extension (e.g. .eex, .erb) and make it dynamic. This would lower the barrier for beginners. By the way, the best HTML tutorial I found so far is Scrimba. The idea of having interactive videos where you can pause and edit the code is amazing.
- iamben 4y agoYou're right. And I'm sure they exist - just hard to find in a sea of "you'll get nowhere without knowing React" (or whatever) tutorials. Equally, half the magic is ability to see it online quickly. Cheap, basically set up free web hosting and FTP made that very easy.
- endemic 4y agoI _really_ tried to learn Perl back in the 90s, but PHP was what “unlocked” server-side scripting for me.
- alerighi 4y agoTo me PHP is still a great tool to do these sort of stuff. Not to do a whole application, it was not what it was designed for, but for simple stuff it's great! And it's improved a lot PHP, nowadays it's not that bad language...
- mmcgaha 4y agoI built a bunch of "contact us" pages for folks over the years and always used python with the built in CGI library. The python folks thought so little of CGI that they ripped it out of the standard library. I wish I had just used Perl like god had intended CGI scripts to be written.
- pixelfarmer 4y agoI had a good laugh when reading this. On that note, I recently revived some nearly 20 year old perl CGI stuff. Had some minor syntax issues (language ambiguities which got fixed), but it was running just fine after having fixed these. There was also some C code with it (some tooling), needed header include fixing + function renaming, otherwise still worked. Small project all in all, not reliant on a lot of external things.
- anthonyhn 4y ago>The python folks thought so little of CGI that they ripped it out of the standard library. IIRC, the issue was that they didn't have a maintainer for the cgi library anymore, and that is why they removed it.
- samsquire 4y agoI appreciate this post, thank you for writing it. There is something elegant about PHP and CGI. Just put file somewhere and query it by browser. How would you fix the design of PHP and CGI to be scalable? I wrote a userspace scheduler that multiplexes lightweight threads onto kernel threads. It is a 1:M:N scheduler since there is a scheduler thread that preempts other thread's loops. I think I can write a runtime that uses files. I also have an epollserver. If I merge them together and write a HTTP router, then in theory you could have a modern server runner that can execute in-process with threads. I would need to think how to execute Ruby, Lua, Javascript code in a thread. Perhaps that's similar to NodeJs but it's a file runner rather than a server application. https://github.com/samsquire/preemptible-thread https://github.com/samsquire/preemptible-thread https://github.com/samsquire/epoll-server https://github.com/samsquire/epoll-server
- tyingq 4y ago>How would you fix the design of PHP and CGI to be scalable? A lot of sites get pretty far with just FastCGI. If that's not enough, then a user-space/mmap cache like Apcu. Then, if that's not enough, horizontal scaling and Redis or similar.
- tasn 4y agoI always felt that the best software should be designed like an onion. Very easy to get started (no need to peel too much to get to the outer layer), but there should always be the possibility to go deeper and peel more layers. Easy to get started is not great (as a general purpose tool) if there is no decent upgrade path. What replaced CGI scripts is slightly more complex, but also more naturally grows to other use-cases. In short: I personally don't miss CGI. :)
- VWWHFSfQ 4y agoCGI was also a security nightmare full of footguns. Not that modern software isn't, but CGI was especially bad. - Take untrusted input and pass it directly to shell scripts on your filesystem. What could go wrong? - Executables in cgi-bin were nearly always included in your document root on shared hosting servers. - File drops and web shells were rampant - A trivial misconfiguration (not setting +x?) could result in the web server just serving up the source code of your scripts directly to the client. There's your database credentials hard-coded right there at the top of your file now exposed for all to see. - You had to pay very close attention to resource limits on the server itself to prevent malicious things like fork-bombs - Apache was a nightmare to configure securely for CGI The list goes on. I don't miss it at all.
- kefka_p 4y agoYou didn’t have to hard-code database credentials into your app, though. That’s a design flaw, not a requirement of CGI. Who passes non-sanitized or otherwise untrusted input to executables, any more than is necessary? Why do you (seem to) believe modern approaches to dynamism are immune when the bulk of user input still comes from HTML forms? How do modern web applications avoid input from untrusted sources? Same applies to the directory layout of servers they were hosted on. That’s more a lack of server admin competence than a fault of CGI. Was far easier than configuring Tomcat to safely host Java web apps. I recall it being far easier to configure than WebObjects, PHP, Rails, etc. CGI programs were far from the only form of web application effected by local security implementations. In my experience common gateway interface applications consist of fewer files than something written against Spring, WebObjects, or 99% of server side web app frameworks. Fewer files means less system configuration and less attack surface, no? As to forking leading to DDOS, that was something that could happen to any forking server, which was pretty much all servers when CGI first came into existence. FastCGI helped address that issue.
- jstrieb 4y ago> Zero external configuration, other than telling your webserver to enable CGI on your file This is only true if you've done it before, and know what you're doing. In reality, it looks like a mess of `mod_cgi` configuration, trying different combinations of file permissions, finding the magic `cgi_bin` directory, finding the right obscure log files when there are inevitably errors, wrestling with CORS and other subtleties of HTTP headers, and other complexities that are only easy to navigate if you're already an experienced CGI user. That being said, I love the philosophy of using CGI for scripts. Instead of using CGI itself, though, I wrote a (single-file, statically-linked) web server called "QuickServ" to bring this philosophy into the twenty-first century. It has all of the upside of CGI, but is much easier to set up and run, especially for beginners. One of its benefits is that it automatically parses submitted HTTP forms, and converts the form fields to command line arguments. That means it's extremely easy to put existing CLIs on the web with minimal changes other than writing an HTML form front-end. If you like CGI, I will (shamelessly) ask that you check it out! https://github.com/jstrieb/quickserv https://github.com/jstrieb/quickserv https://news.ycombinator.com/item?id=29002694 https://news.ycombinator.com/item?id=29002694
- ilyt 4y agoI thought about that for few of the personal stuff but in the end I just opted to have a binary that's just http router + abstracted away way to handle requests. So when I need new feature I just add a module and deploy whole (that includes static files) blob. No need to create new service, add it to loadbalancer etc.
- iso1631 4y agoa2enmod cgi; service apache2 restart Then in your config file AddHandler cgi-script .cgi Options +ExecCGI
- superkuh 4y agoYou don't even need CGI. Just write your programs to tail the webserver access log file and generate the appropriate pages/changes. :) I'm not making fun. I actually do this to implement my comment system.
- ilyt 4y agoAs someone that was sysadmin back in the days where cgi-bin was norm I can only say one thing about cgi-bni: projectile vomits profusely > No dependencies or libraries to install, upgrade, maintain, test, and secure. Except entire webserver. Just install <your language's preferred web server lib> if you want simple. > Works with any programming language ever that can read text and print text. You won't use it with those that can't anyway unless for mental excercise > Zero external configuration, other than telling your webserver to enable CGI on your file. Except entire webserver. Just install <your language's preferred web server lib> if you want simple. Seriously, what kind of nostalgia someone needs to have to have any positive thoughts over cgi-bin?
- didip 4y agoJust use your favorite programming language’s http server library. The development time will be faster and the code will be more cohesive.
- ytjohn 4y agoI've been doing a good bit of research and planning for self-hosted Functions as a Service (also marketed as Serverless). Now, right off the bat, setting up one of the frameworks in kubernetes to accomplish this is going to be more complicated than setting up a cgi-bin capable system. So I wouldn't quite recommend self hosting it for a personal project. But if someone else set it up, or you're using one of the cloud ones, FaaS sort of reinvents the cgi-bin mindset. Here is a talk by one of the Fission.io devs showing the similarities. https://www.youtube.com/watch?v=X-XV6vvwhuo https://www.youtube.com/watch?v=X-XV6vvwhuo Not a plug for fission, not affiliated. I mainly like how they integrate with Keda for event/message queue setups.
- qbasic_forever 4y agoIn 2023 I don't think CGI is as easy as it was back in the day. The major proxies/servers most people use like nginx only support FCGI which isn't directly compatible with CGI--you need to run helpers like fcgiwrap or other new layers of complexity. The only major servers that still support plain old CGI are apache and lighttpd. IMHO properly configuring apache is a mess. I think what people really want when they pine for the days of CGI is simple file-based routing for dynamic content. Look around and you can see new frameworks are coming back around to this... or consider plain old PHP.
- phkahler 4y agoIs there a way to maintain a sort of process pool so CGI doesn't have to start a new process with each request? It's also important that a CGI crash doesn't kill the web server.
- kristov 4y agoI think FCGI is what you want.
- Animats 4y agoI still use FCGI. FCGI works a lot like CGI; it loads programs when a request comes in, and the programs deal with the request. But it's much higher performance, because it doesn't have to freshly load the target program each time. FCGI is an orchestration system. It starts up more worker processes if there are more requests, up to some limit it figures out by itself. If there are few requests, some of the worker processes get an end of file notification and exit. If a worker process crashes, it loads a fresh copy. Little or no human attention required. One of my systems has been up for 1788 days now. FCGI works fine with Go programs, Python programs, etc. If you use Go, you can get performance reasonably close to what the hardware can do. Until you get to the point that you're renting additional servers as load increases, you don't need more than that for most applications. You can run FCGI on some cheap shared hosting systems. Why pay more?
- lma21 4y agofor Go, do you rely on `net/http/fcgi` to run it? Do you have any example i can have a look at?
- Animats 4y ago> for Go, do you rely on `net/http/fcgi` to run it? Yes. > Do you have any example I can have a look at? https://github.com/John-Nagle/vehiclelogserver/blob/master/vehiclelogserver.go https://github.com/John-Nagle/vehiclelogserver/blob/master/v... There's not much machinery required. package main import ( "database/sql" "net/http" "net/http/fcgi" ) // // Called by FCGI for each request // func (sv FastCGIServer) ServeHTTP(w http.ResponseWriter, req *http.Request) { body := make([]byte, 5000) // buffer for body, which should not be too big if req.Body != nil { len, _ := req.Body.Read(body) // body of HTTP request bodycontent := body[0:len] // take correct part of buffer Handlerequest(sv, w, bodycontent, req) // handle request } } // Run FCGI server func main() { sv := new(FastCGIServer) fcgi.Serve(nil, sv) } // Handlerequest -- handle one request from a client func Handlerequest(sv FastCGIServer, w http.ResponseWriter, bodycontent []byte, req *http.Request) { // GENERATE REPLY CONTENT HERE w.WriteHeader(statuscode) // internal server error w.Write(content) // report error as text ***TEMP*** } That will run on low-end hosting at Dreamhost. So there you are, in a modern language, with hard-compiled code with good performance, and reasonably good fault tolerance. Next step up would be a load balancer and redundant databases.
- rvense 4y agoI developed a need recently to gather some answers to a few simple questions from people all around the internet. Maybe a few hundred at most. I don't want to make a Google survey or a Doodle or any of that. It's not for me, not off the clock anyway. I built a simple HTML form to start with. Just typed it out like it was 1997 again (and then added a little Javascript so it remembers that you've submitted the form to soft-block people from submitting more than once; and some CSS to make it look a little nicer and work on phones and such). Then I thought I had to build a small backend with a database... I'd originally planned to do a simple CGI thing that stuck the form in SQLite or whatever, but then, when I was doing the frontend, I saw that the web server that I use to serve the file actually puts the entire query string in the access log on disk. So that's it. That's the backend. I can just grep the log file when I want the answers.
- deleted 4y ago[deleted]
- derekzhouzhen 4y agoFor CGI you need a interpreted language that start up fast. Last I checked Perl5 is still the king. Writing CGI scripts with node.js is theoretically possible but you are in for a lot of pain.
- rascul 4y agoIt doesn't have to be an interpreted language. As long as you can read environment variables and write to stdout then it can be any language.
- dizhn 4y agoPHP's original creator was doing this with C when he decided to write a simpler alternative I believe.
- derekzhouzhen 4y agoYou can do CGI in C, but isn't it defeating the biggest advantage which is simplicity?
- rascul 4y agoI don't necessarily find that adding a compiler step defeats the simplicity, if simplicity is the goal. It might be easier to use a language with better string handling than C, though.
- int_19h 4y agoIt is, since C has horribly deficient string handling facilities in its stdlib. But a compiled language is not necessarily C. And, well, simplicity can be relative: if you already know language X, using it will likely be simpler for you than some other better language that you would have to learn first. The beauty of CGI is that its simplicity makes it a language-agnostic API, letting you pick the simplest language for yourself.
- ihateolives 4y agoWe had CGI program written in C in production in 1999 or thereabouts. Worked fine. The problem was, that there was only one person who could understand it and update it. It was ditched at some point in favour of PHP I think.
- jeffbee 4y agoNot mentioned: testing CGIs is extremely simple because it is just stdio.
- ananonymoususer 4y agoI had to click and read the article to see whether CGI in this context was "Computer Generated Imagery" or "Common Gateway Interface". I've always been a fan of simple CGI interfaces. More middle-ware always means more bugs.
- freyfogle 4y agoYou will be pleased to know CGI.pm v4.55 was just released last week https://metacpan.org/dist/CGI/view/lib/CGI.pod https://metacpan.org/dist/CGI/view/lib/CGI.pod It's still there, you can still use it. Works great.
- chazeon 4y agoIf all the tools you need is in the shell, and you don't need to maintain a state, CGI is extremely simple and powerful, all you need to do is to write a shell script. I found this useful when I create a HTTP service to check a remote service's status over SSH.
- 1vuio0pswjnm7 4y agoI got a 404 when trying to access the referenced blog post about CGI. https://web.archive.org/web/20210822121843if_/https://halestrom.net/darksleep/blog/046_cgi/ https://web.archive.org/web/20210822121843if_/https://halest... https://webcache.googleusercontent.com/search?q=cache:https://halestrom.net/darksleep/blog/046_cgi/ https://webcache.googleusercontent.com/search?q=cache:https:... "If I'm ever in the situation again of helping new people learn web technology then I'm going to get or convert them to use CGI right off the bat. It's easier to teach, it's easier to understand, easier to get working on most webservers and isn't locked in to any particular language or framework. The only downside of CGI that I know about is the fact it starts a new process to handle each user request. Yes that's a problem in big sites handling hundreds or thousands of visitors per second. But by the time a new student gets to running a big site they will have already encountered many, many other scalability issues in their code and backend/storage. Let alone teaching them database and security concepts. There's a reason we have quotes like "premature optimisation is the root of all evil". I don't think students new to webdev should be started on anything other than CGI. They can use any language they want. They can actually understand what they're doing. And they're not hitting any artificial barriers or limits set by frameworks or libraries." OpenWRT still uses CGI https://openwrt.org/docs/guide-user/luci/luci.on.lighttpd https://openwrt.org/docs/guide-user/luci/luci.on.lighttpd
- nine_k 4y agoCGI is fine for unfrequently accessed pages on a machine that has some other purpose. Say, a stats / monitoring page on a file server, or something similar. [FastCGI] keeps most of the CGI's simplicity of the interface, while keeping the worker processes running, instead of starting a fresh copy every time. It's very widely used to this day. [FastCGI]: https://en.wikipedia.org/wiki/FastCGI https://en.wikipedia.org/wiki/FastCGI
- tannhaeuser 4y agoCGI can be used for high performance sites/routes when used with HTTP caching. Though CGI leaves response time on the table by taking arguments via the environment such that you can't start/pool CGI processes before requests are arriving. FastCGI needs multiplexed transport of HTTP traffic in to and out of a single process, changing code complexity drastically and taxing memory management. It's far from keeping CGI's simplicity.
- sedatk 4y agoI wrote Eksi Sozluk, the most popular Turkish social platform to date, back in 1999 using Delphi as CGI executables[1]. Delphi was the tool that I knew best, and CGI worked for me at the beginning. I almost immediately started having problems with EXE files being constantly locked, making updates impossible (as it was Windows based), which required restarting the server for any kind of update. CGI executables also caused problems when I switched to an Alpha AXP RISC server which emulated x86, bringing the performance to a crawl. That made me switch to classic ASP, and about 10 years later, to ASP.NET MVC with routing, unit tests, abstractions, jQuery, all the shiny things at the time. Web didn't stop there of course; there came SPA's, React, Vue and whatnot. Now, seeing the yearning for CGI in 2023 feels funny. Have we come full circle? :) ___ [1] https://github.com/ssg/sozluk-cgi https://github.com/ssg/sozluk-cgi