6 ms·
A simple web server written in Awk
- NonEUCitizen 3y ago"requires socat"
- cf100clunk 3y agoOr cd to the html directory and run this: python -m http.server 8080 which does not require anything else like socat.
- prokopton 3y agoI think this is more of a “ain’t this interesting?” sort of thing.
- cf100clunk 3y agoNot a nawk on awk (pun intended) or socat, just a reply to the parent's comment.
- Brian_K_White 3y agoThis is functional and uninteresting.
- cf100clunk 3y agoFor those who were drawn to ''awk'' in the title it is probably uninteresting. For those who were drawn to ''simple web server'', it is likely of some interest.
- Brian_K_White 3y agoThere are 1000 simple web servers. That part was never interesting to anyone interesting themselves. If it had been "process some records with awk" that too would have been uninteresting even though it said awk.
- sodapopcan 3y ago"requires python"
- CSSer 3y agoWhich in a way is somehow worse :P
- hackideiomat 3y agoin many ways
- cf100clunk 3y agoYes, for some that would be a show-stopper. It is a nice one-line web service string, but should probably be run thusly: nohup python -m http.server 8080 2>/dev/null &
- pmarreck 3y agoNothing is interesting about overrated Python though
- cf100clunk 3y agoUbiquitous, omnipresent, consistently rated in the top 2 or 3 most used computer languages. You don't like it, fine.
- Brian_K_White 3y agoApparently not fine. Are you about to cry?
- deleted 3y ago[deleted]
- keepamovin 3y agoFor tcp. There's another one which uses GAwk's internal tcp stack, but I wasn't sure if that would work in normal awk, and I'm a purist like that so you got this one.
- pmarreck 3y agoI’d like to see both, please
- nine_k 3y agoSocat is a trivial stream converter, like a pipe fitting with different thread sizes on two ends. The interesting part is the logic and the actual.string processing.
- ISL 3y agoMy first interesting question was, "How do I open a port in awk?!"
- bewuethr 3y agoGNU Awk has an entire separate manual for TCP/IP internetworking: https://www.gnu.org/software/gawk/manual/gawkinet/ https://www.gnu.org/software/gawk/manual/gawkinet/
- eichin 3y agoTrue, but you can go the other way and do it with only socat: socat -v tcp-listen:8888,reuseaddr,fork exec:'cat RESPONSE' is a (Very Static) entire web server, if RESPONSE has 2 header lines, a blank, and an html body. (A little too late in the morning for me to do the golfing needed to eliminate the "cat" and do the cr-nl from whatever shell you're already running, though.)
- eichin 3y ago(scrolls down further in the socat manpage) Ohhh, under "EXAMPLES": you can combine a crlf option to TCP-LISTEN, SYSTEM instead of EXEC, and "echo -e", and there's actually a slightly useful http server already done :-)
- scrumlord 3y ago[dead]
- metadat 3y agoIt's funny how closely this resembles nginx.conf.
- dosk 3y ago23 years earlier https://marek.terminus.sk/prog/hawkh.tgz https://marek.terminus.sk/prog/hawkh.tgz homepage https://marek.terminus.sk/prog/hawkh.shtml https://marek.terminus.sk/prog/hawkh.shtml
- metadat 3y agoMarek's achieves it without socat, using only cat and sh built-ins. Badass OG.
- arpa 3y agoa small nitpick - it uses inetd, which actually handles the network stuff allowing you to work on stdin/stdout. At least that's what i remember from 2003(?) when i wrote an identd for conntracked connections in bash.
- calvinmorrison 3y agothen on top of that you can run werc a web framework written entirely in rc, save for a few of the filters which i think are awk files (like the markdown2html converter)
- deleted 3y ago[deleted]
- mdale 3y agoAwwwwk such a cute web server :)
- iamcreasy 3y agoInteresting, but what is a TCP wrapper?
- kstrauser 3y agoSomething that handles the TCP connections and passes them to the wrapped program’s stdin and stdout. Back in the day, lots of network daemons were written as command line tools and executed via inetd.
- deleted 3y ago[deleted]
- kqr 3y agoIs there a modern solution for this that isn't some variant of FastCGI? I know it's inadvisable but sometimes I would like to write a proof of concept that doesn't require me to code a reliable, long-running process.
- ncruces 3y agoCompile a CGI program in any language to WASI, then use https://github.com/deislabs/wagi https://github.com/deislabs/wagi to run it.
- kstrauser 3y agoinetd still exists. I’d probably just use that. Edit: There's a CGI plugin for Caddy that looks easy to set up: https://github.com/aksdb/caddy-cgi https://github.com/aksdb/caddy-cgi
- dmux 3y agoIf you enjoy stretching what you can get done with limited environments, I recently discovered BusyBox includes not only an AWK implementation but an HTTP server[0] that supports CGI as well. I spent a weekend setting up a really simple web-app using recutils [1] as the database and BusyBox AWK as the programming language. If you enjoy bricolage and you feel like wasting a weekend, I would recommend giving it a try! [0] https://openwrt.org/docs/guide-user/services/webserver/http.httpd https://openwrt.org/docs/guide-user/services/webserver/http.... [1] https://www.gnu.org/software/recutils/ https://www.gnu.org/software/recutils/
- deleted 3y ago[deleted]
- Aachen 3y agoThat looks so featureful (thinking of attack surface here), you might as well install a regular web server at that point. What's the benefit of using busybox for this? Edit: ah is this file size? Or does this refer to RAM? Either way, sounds like that's the benefit > BB httpd can be compiled with only basic features like CGI and ETag and will have only 8Kb
- Crestwave 3y agoBusyBox provides a complete set of POSIX utilities and more in a single minimal binary; there are lots of embedded systems with pretty much just BusyBox installed, although I'm not sure how common it is to compile it with httpd. GP linked an OpenWRT article, so I'm guessing (the site seems to be down) it's included in that OS.
- chasil 3y agoI have found that busybox awk: 1. Does not implement the GNU networking extensions. 2. Does not implement the GNU array sorting extensions. 3. Does implement the mktime() and strftime extensions. I see that there is a call to stat() in the code; I didn't know that GNU added this.
- 3y ago
- notorandit 3y agoYou can achieve the same in bash. With no so at.
- cf100clunk 3y agoCould you post a bash string or script that does this?
- LispSporks22 3y agoI thought you could listen() accept() in modern awks.
- tyingq 3y agoSort of. Gawk supports magic file names like "/inet/tcp/8080/0/0" to be a tcp client or server, but you don't get control of accept()...it's all mooshed together. So you can make a slow, single threaded webserver with it. Which is too bad, because the usually shipped gawk extensions also give you fork. If they had separated listen() and accept(), you could actually make a reasonable webserver. See my other comment for an example of what you're limited to.
- Yannik_Sc 3y agoIf I got this term correctly, these days it would rather be a "Serverless webserver in awk". Also sounds more catchy.
- hackideiomat 3y ago> gsub(/\/\.\./, "/", request_filename) # avoid directory trasversal Hmmmm http://localhost:8888/..../..../..../..../..../..../.../..../etc/passwd http://localhost:8888/..../..../..../..../..../..../.../.......
- tmsbrg 3y agodamn, you beat me to it. Was gonna write: http://localhost:8888/..../..../..../..../..../..../etc/hostname http://localhost:8888/..../..../..../..../..../..../etc/host... mypc These regex substitutions are so easy to bypass :)
- hackideiomat 3y agoAlways fun :D
- x-complexity 3y agoAll this tells me is that preventing directory traversals can only be done by checking absolute file paths are within a bounded range, and nothing else.
- nonameiguess 3y agoRunning the server as a service account that can only read its own directories, running it in a chroot, running it in a mount and pid namespace, using SELinux to further restrict what files it can read even in principle. Of course, if you're trying to go superminimal anyway, it's not that big a deal to create a server that doesn't even have sensitive data on it. You can make init simply mount a root filesystem that only has busybox and whatever files you want to serve and starts up the httpd process and nothing else. Turn Linux into a unikernel basically. If you compile busybox yourself, you're also able to remove all the subcommands you don't actually need.
- pmarreck 3y agochange that to: > gsub(/\/\.\.+\/?/, "/", request_filename) # avoid directory traversal source: am "regex expert" >..< (and know how to spell)
- tyingq 3y agoAh, I had posted a golfed silly gawk webserver here some time ago. It uses gawk's built in tcp. https://news.ycombinator.com/item?id=22085459 https://news.ycombinator.com/item?id=22085459 Pretty printed: https://gist.github.com/tyingq/4e568425e2e68e6390f3105e588787c6 https://gist.github.com/tyingq/4e568425e2e68e6390f3105e58878...
- tripflag 3y agoneat! i was aware of bash's built-in tcp client at /dev/tcp, but gawk being able to function as a tcp server is way cooler. Thanks for that bit of knowledge :>
- dzove855 3y agoActually, bash has now a loadable builtin to listen on tcp socket. I wrote a script and the patch. https://github.com/dzove855/Bash-web-server https://github.com/dzove855/Bash-web-server The patch is now included in bash 5.2
- Aachen 3y agoWhat does the loop and sleep 1 do? Is that to respawn upon crashes (why'd it crash?) or does it exit socat after handling a request? I recently made a netcat webserver returning only one static response just to have a tiny info page for my new email server, that needs a while true loop but no sleep. I then benchmarked the performance and was very surprised to find that a slow VPS manages [spoiler answer] https://lgms.nl/p/cau/?b64&bY%2FBTsQwDER%2FZbivViDxAxw58Q1pOtkEWrvEzu7y97gFceJkjT3zbL9d2fH8iM7PQXPDFtqYVWa0gi8dqGldo9kct%2BYVFB2Xirw0itsDXsK5aie8JsHT%2F6gj%2BT7MocLf7BmvwZ7B%2B8bsnJFCylzGguSYAnGCNcncVws5G1zRh8QmYkpWsahuJ%2BTK%2FGEo2o8J782ce05LMEtbjtscPrrYYTH2%2FW8t5bRrgW3pFrNwdwZX6DkCW9dMs5%2F03wVTi4ei7qBNuyNdUpPzNw%3D%3D https://lgms.nl/p/cau/?b64&bY%2FBTsQwDER%2FZbivViDxAxw58Q1pO... Performance graph: https://snipboard.io/Vtn0MO.jpg https://snipboard.io/Vtn0MO.jpg
- tyingq 3y agoIt's a one-shot awk script. Handle one request and exit.
- Aachen 3y agoHuh okay, then why the sleep?! That kills performance but why'd you do that
- tyingq 3y agoNo idea, not my script. Socat does support this kind of syntax, which would seem better suited in this case. socat -v -v TCP-LISTEN:8080,reuseaddr,fork SYSTEM:/some/script It also has a settable max-children so you can put some bounds on it. I did post another comment where I shared my gawk webserver, but I'm not using socat.
- Crestwave 3y agoThe sleep is probably to prevent endless revival of the script when you try to cancel it with Ctrl-C. But yeah, it's not ideal considering you can't make more than one request within a second.
- drzaiusx11 3y agoFor those who like simple shell tools taken to extremes, this sed script implements "Super Mario Level 1 for the NES" https://github.com/chebykinn/sedmario https://github.com/chebykinn/sedmario
- wigster 3y agocrazy. lol
- thricegr8 3y agoI know this seems the antithesis, but I've always wondered if you could fold and mangle cURL to actually act as a web server?
- tyingq 3y agoI don't think so. Poking around the source code, the only thing exposed in the cli client around listen() and accept() is related to FTP, because of the way FTP works. It does mean, though, that libcurl has functions like Curl_conn_tcp_listen_set() and Curl_conn_tcp_accepted_set() that could be used for what you're describing. It's just that they are only used for FTP now.
- hi41 3y agoI love when people do projects with awk! I received my awk 2nd edition book just yesterday! I am so excited about reading it!
- abecedarius 3y agoCool news -- I liked the first edition a lot back in the 90s.
- pmarreck 3y agoI've heard of Awk for years and (probably like many) only used it for single-line snippets for the vast majority of that time. Imagine my surprise when I just decided to look into it one day (after finding slightly more complex Awk scripts that did a lot with very little code which piqued my curiosity) and finding this very nice line-oriented DSL that has aged SHOCKINGLY well given how old it is. I just wish it had interrupt handling of some sort without running a custom fork/patched version
- debarshri 3y agoIt must be a very awk-ward service to deploy.