7 ms·
Rethinking of CGI as a selfhosted lambda server
- _bxg1 6y agoa) This site is very hard to navigate; on mobile the text jumps around whenever you touch and it's usually cut off past the edge of the screen b) I couldn't figure out what this is. At first I thought it was computer graphics rendering (what "CGI" usually means), but after skimming several paragraphs I don't think that's what it is? But I'm still not sure. c) Clicking the github link tries to download a .zip, which beyond making it harder to figure out what this is, was mildly distressing. Edit: re: the downvotes, presumably the author would want to know that their site is broken on mobile and that the content is inaccessible to the average reader who might be curious to learn about it, in ways that could be easily remedied.
- Animats 6y agoIt's about the Lambda product of Amazon Web Services. No relation to Church's lambda calculus or lambda constructs in programming. The article does not make that clear at all. (One of the privileges of being a really big company is that you can name your products using common terms and take over that term. Like, "Windows". Even though Sun used that first.)
- _bxg1 6y agoI did actually follow the "Lambda" portion of it, for what it's worth
- bdcravens 6y agoI think your downvotes are for your comment about CGI; while the term is ambiguous, in the context of serving the web it refers to Common Gateway Interface, the original method for serving anything other than static files. https://en.wikipedia.org/wiki/Common_Gateway_Interface https://en.wikipedia.org/wiki/Common_Gateway_Interface For many HNers, this is an understood term.
- _bxg1 6y agoI don't think it's unreasonable to ask that acronyms in general be expanded towards the top, especially when there's a commonly-known overload in the same general subject area. Personally, I've been working in the field for six years and studied in it for six years before that, and I'm pretty sure I've never heard that term. Simply writing out the full phrase "Common Gateway Interface" would a) tell me it's not what I thought, and b) give me something to google so I can learn about the subject and/or decide if I'm interested in continuing to read.
- bdcravens 6y agoThere's some advantage to abbreviations/acronyms (since titles have a length limit), but in general, I agree. I propose the first place we start is to agree that "crypto" doesn't always mean "cryptocurrency" :-)
- catalogia 6y agoIf you don't already know what 'CGI' means in this context, I'm skeptical that "Common Gateway Interface" would actually resolve much confusion.
- _bxg1 6y agoIt would tell me that it doesn't have to do with graphics, and that it probably has to do with networking. That's a whole lot more than is made clear as it is.
- fouc 6y agoNice idea, bring back the old Common Gateway Interface for web development.
- chii 6y agothe cycle of technology. What is old will be new again.
- reddec 6y agoExactly! Just adapted for a new reality. And I found it quite exciting to adapt old proven technology to a new stack
- josephcsible 6y agoWon't happen for anything serious, because a process per request doesn't scale. FastCGI is probably as close as anything modern will ever get.
- reddec 6y agoThe point is to use low-end machines for a number of logic that 99% will do nothing. As example: webhooks. In this scenario there is no need for high performance/low latency but there is a high demand of low resource usage. However there's a way to scale it horizontally using shared storage. But it should out of scope. Btw I am author of it and I will be happy to answer on any questions. Thanks for your interest!
- zwetan 6y agomoving from CGi to FastCGI is pretty easy
- aidenn0 6y agoThe problem with fastCGI is that it's only very slightly more complex to write a web server compared to a fcgi server
- 6y ago
- kalium-xyz 6y agoWe have gone full circle c:
- waheoo 6y agoIm still trying to understand the difference between a .php file and a lamda "serverless" application.
- CGamesPlay 6y agoWell a php file (that isn't a library, so one that produces output) is one of the things that you could write to be a lambda function. But not every lambda function can be expressed as a php file. So I could download some video filtering lambda functions and they can be written in anything, then I can link them against my application, which I write in PHP. The language/platform/dependency freedom is what takes it from literal CGI script to lambda function.
- naniwaduni 6y agoIt took a moment of thinking to decide which of "literal CGI script" or "lambda function" you were attributing "language/platform/dependency freedom" to. I would've attributed them to CGI.
- CGamesPlay 6y agoAh, that's fair. I was mostly attempting to highlight that a php script is a subset of lambda functions (from a conceptual viewpoint). Of course it's also a subset of CGI scripts as well (in a much more literal viewpoint).
- buster 6y agoThis does not sound right. The platform independence and freedom is the big win for lambda where it is basically the perfect vendor lock-in? With serverless you give up a lot of independence.
- tuukkah 6y agoIs this compatible with AWS Lambdas on some level? Could it be? What about FastCGI/WSGI/... compatibility? I'm thinking it could be really nice for hobby projects and for developing on a laptop.
- reddec 6y agoAny script/application that can parse STDIN and write something to STDOUT is capable for the project. Hobby projects or projects with low number of requests are ideally fits. And it could be run on very low-end devices: raspberr pi, cheapest VPS, etc..
- fiddlerwoaroof 6y agoOpenfaas.com enables cgi-style applications to be run on k8s
- reddec 6y agoHeavy, very heavy. Even with k3s. I tried it first. Even plain k3s will run slowly on 1GB machine (a target audience)
- fiddlerwoaroof 6y agoYeah, I was thinking about this as a migration path for scaling-out CGI scripts
- silasb 6y agoI've been hacking on some go code to put together a set of services to run lambda functions locally. It's a hodgepodge of different services. https://github.com/silasb/lambda-engine https://github.com/silasb/lambda-engine https://github.com/silasb/lambda-scheduler https://github.com/silasb/lambda-scheduler https://github.com/silasb/deno-aws-lambda-example https://github.com/silasb/deno-aws-lambda-example Since this is a POC it's probably not worth looking at or trying to use as the instructions are in bad shape.
- floatingatoll 6y agoOP, your site is unusable on mobile. I wasn’t able to read about your project as scrolling causes it to force half the text and diagrams out of frame.
- adnanh 6y agoI did something very similar for handling webhooks (but can also be used for anything else), check it out at https://github.com/adnanh/webhook https://github.com/adnanh/webhook People have used it in various segments, from actual deployments on push, to home automation, someone even wrote a guide on how to control the Xiaomi Robot Vacuum using the Amazon Dash button :-)
- reddec 6y agoGreat project! However, I think our projects similar but not the same. Looks like we can exchange some ideas from each-other
- adnanh 6y agoYeah, my guess is that they share the same concept - expose triggers via HTTP(s) to run user commands. The webhook handling put on some additional requirements on my project: to go a little bit beyond piping the input to the script :-)
- reddec 6y agoRight. Trusted-cgi also adds security checks (with various checks), "actions", scheduler and web UI =) I would like to suggest to stop measure benefits of both projects and just "steal" them from each other, making end-user experience better for both projects ;-)
- michael-ax 6y agoWhen I published webhub @ href.com in 1995 we called these 'custom runners'. Looks like you'd have approved of that concept :)
- reddec 6y agoThanks for tip: several ideas from there look very promising
- est 6y agoinetd was like what, eBPF hacks and sidecars?
- rossmohax 6y agonginx unit (https://unit.nginx.org/configuration/#process-management https://unit.nginx.org/configuration/#process-management) does something very similar. with `spare = 0` config option it will start app process lazily and shutdown when idle. There is support for Ruby, Python, PHP, Java, Node, Go runtimes.
- reddec 6y agoInteresting. Could be useful for high-load, however for quick setup feels like a little bit complicated. Are any known UI, or scheduler settings? Following is a general response for similar questions. I pretty sure it's possible to do the same thing as trusted-cgi by combining several tools: cron + nginx/nginx-unit + some deployment + stats + whatever. It could be faster, more flexible, but... I am not sure that I will personally want to setup it each time for new boxes. Or write auto-install scripts. I would like to write logic, not make environment. Benefits trusted-cgi are: low resource usage (very low), easy setup, web UI and focus on including several features in one package. It's possible to find better solutions for each parts, but I would like hear product that providing complete solution instead of providing "bricks". I am a big fan UNIX way, but sometimes we need an assembled thing, instead of another piece of constructor.
- xorcist 6y agoYou can also use socket activation in uWSGI for this, together with a reasonable value for die-on-idle. What's nice about it is that applications run unmodified. uWSGI really is the swiss army toolchain for web operations.
- HelloNurse 6y agoHaving first met Apache when Perl CGI scripts were bleeding edge fancy "web app" technology, I hoped to read a satirical article, but it's actually an earnest suggestion to go full circle. It does less than proper CGI, due to giving up HTTP (particularly response headers, status codes and content types) in order to process constrained JSON, and it does it in a more complicated way (UI? Scheduler? Git repositories?), but it's still better than other options.
- reddec 6y agoTo be fair I am not trying to say that CGI is a bleeding edge technology) I met a problem: low-end devices that should host a big number of small pieces of logic. To solve the problem I looked back to the history and found CGI and adapted it to the specific case. As someone said before here in comments: it's just reinvented "good old technology", but I would like to say that I can't find anything bad in it. Old technology means a number of documentations, it means easy to explain. Ofcourse there are number of issues caused the technology became outdated, but in some cases (defined in target audience) it acceptable. Even sometimes better than newer solutions. More of that, if you will look at the modern technology (serverless functions/lambda functions) you probably will find the same ideas but wrapped into modern solutions (containers and cloud).
- exabrial 6y ago> I also tried self-hosted solutions based on k3s but it too heavy for 1GB server (yep, it is, don’t believe in marketing). ow.
- readams 6y agoLosing the ability to stream the request and response is a big downgrade from normal CGI. I'm struggling to see how this improves at all on it.
- Matthias247 6y agoWhy would you lose it? As far as I understand bodies are still streamed through stdin/stdout. Are you concerned about the headers? You should have all of those anyway before dispatching.
- Supermancho 6y ago...also added the constraint of non-deterministic micro-timeouts. This is some hackery, which is fun, but not something I would want to run.
- gavinray 6y agoIsn't this just OpenFaaS? OpenFaaS has a single-binary distributable that doesn't require Docker/k8s they released called faasd: https://github.com/openfaas/faasd https://github.com/openfaas/faasd