8 ms·
Turn any application that uses stdin/stdout into a WebSocket server
- xxchan 13y agoAmazing how the old is new again. Welcome back, CGI! I hope everyone's aware that this is just a toy and should never be used to do any real work, because most command line tools were never written to be exposed to the internet at large.
- baq 13y agocaveat: real work != internet. cgi is still very useful for those super small internal one-shot scripts.
- danieldk 13y agoWell, for some folks it works fine on the internet as well: http://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg02065.html http://www.mail-archive.com/fossil-users@lists.fossil-scm.or... tl;dr: sqlite.org and fossil-scm.org run using a simple HTTP server via inetd and some parts of the website are CGI scripts. In 2010 they served over a quarter million requests per day on a tiny VPS with only 3% CPU load.
- weland 13y agoShhh! You're on Hackernews man...
- girvo 13y agoSo I've been curious about this; Nimrod has some SCGI libraries and the like built in, but I didn't know whether that would be good enough for real world usage without building a concurrent application server into my App itself. How does it handle multiple requests at once without impacting performance badly? I know that's a noob question, I struggled to find good information on SCGI/CGI for today's use cases, so I sort of wrote it off as too hard basket...
- mikeash 13y agoI run my entire web site on CGI scripts invoked by Apache. I really don't get the hate for it. It's extremely convenient and works great.
- ajbetteridge 13y agoSame here. And to get the flame bait machine started, we even do it in Perl!
- _ak 13y agoCGI (when used with suexec) is still the tool of choice for your usual low-coat shared webhosting provider, and it's still good enough if all your customers want to do is deploy Wordpress, phpBB and maybe some custom PHP and Perl CGI scripts.
- lhaussknecht 13y agoTake a look at how the Leap Motion works. You connect the Leap Motion to your usb port, a demon/service interacts with the Leap Motion and opens a websocket server on localhost so that websites can interact with the hardware device without browser plugins.
- X4 13y agoThanks lhaussknecht That's also what I thought! I don't understand the negativity here! "Oh, this is a toy. Oh, this doesn't meet my Enterprise needs. Oh, this isn't EAL7¹ Certified, I can't run my Atombomb defense system with it." Dear Ladies and Gentlemen, you have to admit that this is ueber useful for so many scenarios, that are out of the box, you're thinking in. I can now hack a firebase clone, just for fun and my prototypes don't require the setup of dozens or a hundred npm packages. Besides the coolness of NodeJS, there are still some use-cases for hackers with this. -- 1. http://en.wikipedia.org/wiki/Evaluation_Assurance_Level#EAL7:_Formally_Verified_Design_and_Tested http://en.wikipedia.org/wiki/Evaluation_Assurance_Level#EAL7...
- fit2rule 13y agoWhats so toy-like about this? Its no toy but rather a useful method of tying differing software systems together .. I can think of many productive uses for this.
- nostrademons 13y agoProblem is that it requires that the app handle all the security aspects of talking to the Internet, because STDIN/STDOUT is an unrestricted protocol. Normally when you write a webapp you have a well-tested parser to validate the HTTP request for you, and you have safe APIs to access parameters, and your templating language auto-escapes dangerous characters for you, and the response goes through middleware to prevent XSRF attacks, and the whole thing is written in a safe language where there's no possibility of buffer overruns. With this, you have to handle all of that in your application binary. You can certainly do this. However, if you are asking why you need to do this, you are almost certainly not ready to do this. I agree that this is a pretty useful method of tying software systems together. However, one of the the systems you're tying together is the public Internet, where all sorts of danger lurks. Writing secure parsers is hard. I've written an HTML parser [1], one of the most well-tested ones out there (I spent the better part of a year doing nothing but testing & debugging, and ran literally billions of documents through it), and Google's security team still found 2 bugs in it with about a day's work. [1] https://github.com/google/gumbo-parser https://github.com/google/gumbo-parser
- acqq 13y agonostrademons's comment is for me the most important comment here in this discussion. Thanks nostrademons! Just as an illustration, from the linked page: "Security. Gumbo was initially designed for a product that worked with trusted input files only. We're working to harden this and make sure that it behaves as expected even on malicious input, but for now, Gumbo should only be run on trusted input or within a sandbox."
- bct 13y agoThat's an issue with this implementation, not the idea of hooking up stdin/stdout to websockets.
- asb 13y agoThe first thing that came to my mind was djb's tcpserver which as far as I can see does much the same thing http://cr.yp.to/ucspi-tcp/tcpserver.html http://cr.yp.to/ucspi-tcp/tcpserver.html
- gwu78 13y agotcpserver does not pipe input. It sets certain environmental variables, closes fd's 0 and 1 and runs another command line program. It has no quivalent to QUERY_STRING. Very different from CGI or this Websocket contraption. Much safer. But the parent says that "most command line tools were never written to be exposed to the internet at large", and tcpserver is a command line tool that runs other command line tools, so is tcpserver OK to use for "real work" by his standards? Maybe we should use a GUI tool "written to be exposed to the internet at large"?
- pmelendez 13y agoCGI won't maintain an open connection though
- sciurus 13y ago27 years since inetd came into being, and now we've made it web-scale. ;-) http://en.wikipedia.org/wiki/Inetd http://en.wikipedia.org/wiki/Inetd I actually don't mean to be snarky, this program acknowledges its heritage and is a cool hack!
- phaed 13y agoThis is beautiful. I can think of a dozen use cases for this right now for one of my pet projects.
- schappim 13y agoCall me unimaginative. What uses are you thinking?
- phaed 13y agoI run a popular Minecraft server, we have tons of scripts between the mc server, forum, admin interfaces, and all the other supporting stuff we run. This makes it so easy to create new functionality that interfaces with our various scripts. I'm hacking away as we speak.
- schrodinger 13y agoanyone getting an HTTPS warning? It's telling me that github.com cert was signed by an untrusted issuer...
- tlrobinson 13y agoNot on Chrome.
- andyjohnson0 13y agoOk for me. Latest Chrome 31.0.1650.63 on Windows.
- mappu 13y agoChrome tells me it's valid (signed by DigiCert) with a sha1 thumbprint of d7 12 e9 69 65 dc f2 36 c8 74 c7 03 7d c0 b2 24 a9 3b d2 33. Check your system time?
- dutchbrit 13y agoCheck your date/time
- tlrobinson 13y agoBasically the same thing in Node.coffee, just because: { Server } = require 'ws' { spawn } = require 'child_process' command = process.argv[2] args = process.argv[3..] wss = new Server port: 8080 wss.on 'connection', (ws) -> ps = spawn command, args ps.stdout.on 'data', (data) -> ws.send data.toString() ws.on 'message', (data) -> ps.stdin.write data.toString() ws.on 'close', -> ps.kill() ps.on 'close', -> ws.close() (Needs a bit more error handling)
- thomasahle 13y agoA nice example on Node. Only I don't quote get the ',' syntax. Is it a scoping thing?
- jbrooksuk 13y agoIt's a Coffee syntax thing. ws.on 'close', -> ps.kill() That compiles to: ws.on('close', function() { return ps.kill(); });
- thu 13y agoI have never done any Coffee but it looks like arguments are separated by commas, and, in this case, the second argument is a function of no argument (so the left hand side of the arrow is empty). Edit: it was meant to be a reply to the parent.
- kaoD 13y agoNice! I love Node for these kind of asynchronous handling of data, it fits perfectly. Keep in mind you can go down the stream route. Instead of: ps.stdout.on 'data', (data) -> ws.send data.toString() ws.on 'message', (data) -> ps.stdin.write data.toString() Just do: ps.stdout.pipe ws ws.pipe ps.stdin I started using Node before Streams and I often forget piping too! I still have a question for Node.js experts... would the 'close' events had to be handled then? Streams handle and propagate the 'end' event just fine, but should the underlying resource be closed? I guess killing `ps` is mandatory since ending `stdout` won't kill the process (or will it?), but is closing `ws` still mandatory too?
- minikomi 13y agoAwesome.. I use the same kind of thing to monitor adb output sometimes: https://github.com/minikomi/pipesock https://github.com/minikomi/pipesock also in go. Edit: Doesn't do any receiving, only pipes to a socket what it gets.
- babby 13y agoThat image made me chuckle. Had to confirm that it was just sourced from google images; but still. :)
- hepek 13y agoisn't this reimplementing netcat -e command? http://linux.die.net/man/1/ncat http://linux.die.net/man/1/ncat
- Sanddancer 13y agoSeems more like an inetd to me. Give a command and run it again and again.
- joewalnes 13y agoIt's similar to netcat -e and inetd, except it also handles the WebSocket protocol (negotiating the handshake and message framing), so your apps don't need to.
- sgt 13y agoIf you're on Ubuntu (and maybe other distros as well), be sure to install the nc-traditional package. Otherwise you don't get the nifty features such as -e
- bct 13y agoBasically, but it also handles some details specific to the websockets protocol.
- gren 13y agowebsocketd --port=8080 bash Then in the browser console: ws.send("ls") I just ported bash to the web :)
- mrspeaker 13y agoHa ha! That's so cool. Hey, off topic... what's your IP? ;)
- huhtenberg 13y agoDon't know his, but I've got the best one 127.123.234.345!
- awestroke 13y agows.send('cat $HOME/.ssh/{id_rsa,id_rsa.pub,known_hosts}'); ws.send('history'); ws.send('echo "' + pubkey + '" >> $HOME/.ssh/authorized_hosts');
- jamescun 13y agoCombine that with Term.js[1] and you could have a fully functioning terminal in the browser. [1] https://github.com/chjj/term.js https://github.com/chjj/term.js
- glomph 13y agoThere is already https://code.google.com/p/shellinabox/ https://code.google.com/p/shellinabox/
- X4 13y agoThe possibilities are endless, just add a websocket proxy as a front-server and you can allow dynamic and even secure control of an entire infrastructure (given that you add auth layer). Here's something you could use for that: https://news.ycombinator.com/item?id=6880487 https://news.ycombinator.com/item?id=6880487 Or just setup haproxy with it.
- 13y ago
- babby 13y agoFor some stupid reason I hadn't considered that logging stdin/out/err to a web interface for my node.js web apps via websockets. To think, I made a browser-side web IRC interface, and didn't consider this. It would be so useful for my clients, who don't know how, or find it too archaic to ssh in. To just open up the admin area and see what's going on, provided they're indeed full-permission admins. Then, to actually send input from said interface, that could make ssh'ing into the server something one need not do often beyond initially setting up the app. Thanks for the idea. Assuming we're not running as root, and the admin side of things is secure, am I not considering any critical pitfalls of this approach? Also, any frameworky cmsy thingers that already do this? Is this new, am I a unique snowflake?
- X4 13y agoyup new, but use a restricted jailed and chrooted shell. Ask your #sysadmin of choice =) You can even build a REST like API wrapper for this. That's what I was going to do.
- babby 13y agoYeah, I thought the same thing concerning the REST api, after mulling it over a bit. Being tied directly to an app could be a bit awkward in practice (Bit of a problem when either can take either's process down).
- guard-of-terra 13y agoIs this just xinetd for the ignorant?
- X4 13y agono. you can still chain xinetd.
- joewalnes 13y agoAuthor here. If you use this on top of programs like bash, well ermm, you get what you deserve ;). Here's an example of how I used websocketd to create a little dashboard for monitoring Linux CPU/memory/IO stats. It basically uses websocketd to stream the output of vmstat to a web-page that plots the numbers: https://github.com/joewalnes/web-vmstats https://github.com/joewalnes/web-vmstats Other useful examples: tailing log files, executing long running job and monitoring output, or interactive querying of datasets that require a long running 'cursor'. This is not for everyone or everything. Remember that like CGI, a process is forked for each connection so it's not the kind of thing if you want to handle a million concurrent connections on a single server. However for dashboards, admin tools, quick mashups, visualizations, etc - it's a pretty handy tool.
- tlrobinson 13y agoYou should probably include a warning anyway. When I was starting out programming I would do stuff like exec-ing shell scripts from PHP, oblivious to the consequences.
- CSDude 13y agoI have a project where small time running tasks requires user input, this would be very nice to have. What would be nice to have is attaching the stdouts of procesess via file descriptors. But I think it could be done easily.
- eddywebs 13y agoIf the shells script takes input parameters can we pass that and eventually turn it into some kind of web service ?
- nitrogen 13y agoBy integrating anything shell-based with anything web-based you encounter a plethora of potential exploits and security holes. That said, if you need to run a script that requires command line parameters, and all you have is standard in, then write a wrapper script that reads N lines of text from stdin first, then parses those lines into the parameters you require. You have to be very conscious of input escaping and validation, though; otherwise your box will be owned in no time.
- joewalnes 13y agoI don't recommend any kind of input parsing in a shell script. You're better of having websocketd fork something like Python, Ruby, Perl, PHP, etc. Like CGI, websocketd will pass additional URL path and query parameters as environment variables. So you could access something like ws://somehost/?a=b and you'd access the QUERY_STRING environment variable in your script (which contains "a=b"). Virtually every scripting language already has a library for parsing CGI environment variables so you can do that. Basically, websocketd tries to emulate CGI as much as possible, but provide WebSocket streams instead of HTTP responses.
- ykumar6 13y agoHere is a live version http://runnable.com/UqkIJXqriJwGAATm/basic-count-example-for-shell-bash-and-websocketd# http://runnable.com/UqkIJXqriJwGAATm/basic-count-example-for...