7 ms·

Awkward – A Node.js-based terminal emulator

76 points by _iostreamer_ 10y ago

amelius 10y ago
This is great! I'll just let this evolve a bit, and wait for broader community support, and then I'll happily replace bash by this shell!
- _iostreamer_ 10y ago
  I would love if that happened :D
nailer 10y ago
This is good, but the data structures - eg arrays of arrays - don't really match the underlying data. Imagine what 'ip addr' or 'ifconfig' would look like - they output paragraphs rather than lines, so scraping lines wouldn't produce good output. It'd be better - and FAR more work - to make an object pipeline based powershell equivalent for Nix, based on JSON. You could write cmdlets in any language that outputs JSON and do better at this project's goal of avoiding scraping. ps node | select pid instead of: ps().find(function(line){ return line[3] === 'node' })[0] Since the fields have keys, you avoid magic numbers and the code's easier to read. (you could of course alias those to make 'pidof node' like Linux distros do) The end result would be much better than Powershell, since it would use JSON and not be tied to .net languages.
- _iostreamer_ 10y ago
  That sounds so good and better. Thank you, I will work on it.
  - ramses0 10y ago
    Consider: wrapPs( "ps -ef" ) => [ {pid: 1234, name: "foo.sh" }, ... ] wrapLs( "ls -al" ) => [ ... ] (not the syntax but the idea). You want to allow interested individuals to write the wrappers "forcefully" against uncooperative maintainers (separate people, separate ideas, separate motivations, separate release cycle, etc). Eventually, some enlightened shell-command owners might add "ls -al --json" or "ps -ef --json", "git stat --json" which removes the need for a wrapper script, but the wrapper script allows innovation peripheral to the core without affecting the core until finally due to overwhelming support the core is extended with something "good" and agreed upon via consensus usage. For the short term, perhaps if "ps" and "ps.wrap" exist in your path, your awkward shell can inject the "* .wrap" automatically around the given command in a way that doesn't require a lot of typing, either automatically, or via shortened syntax. (ps -ef).map(...) !(ps -ef).map(...) {{ ps -ef }}.map(...) I don't know but eventually you'd want to get to a point where you're bridging between the interactive use case (ps -ef printing a nice text table) and the pipeline use case (ps -ef --json letting you easily access different fields), maybe {{ ps -ef }} either calls * .wrap if it exists or if it doesn't exist calls it with --json (this assumes the command supports --json directly) Regardless, this idea has some legs and I look forward to seeing how much traction it gets.
devnonymous 10y ago
Neat hack! I like the idea. Initially when I read the title I thought it would be yet another 'make my shell behave more like my preferred language' type of project. Those while fun to code and play with aren't really useful in the long run. This though might just be useful.
- _iostreamer_ 10y ago
  Thank you!
itaysk 10y ago
Like. But PowerShell still rules. Wish it was oss and x-plat
- _iostreamer_ 10y ago
  I am gonna try PowerShell now. Never used it.
  - feiss 10y ago
    It's really nice, althought you have to like it's syntax.. Google some cheatsheets, will be helpful Its Object Oriented approach it's what many try to emulate in mac/linux. This new hype of improving the CLIs is exciting!
- nikolay 10y ago
  Microsoft seems to be working on porting PowerShell to Linux already [0] and there's a partial port using Mono [1]! [0]: https://windowsserver.uservoice.com/forums/301869-powershell/suggestions/11689284-powershell-core-for-linux https://windowsserver.uservoice.com/forums/301869-powershell... [1]: https://pash-project.github.io/ https://pash-project.github.io/
  - oblio 10y ago
    Oh, I see that pash was revived. It had been abandoned since 2009 (I think).
agentultra 10y ago
Uh... ps | awk 'NR>1{ print $1, $4}' I'll give that the syntax of these shell programs take some work to understand but they're hard to beat for the purpose they serve. If you're parsing plenty of JSON these days I find 'jq' indispensable. It's basically awk/sed for JSON. I would just be sure to study those programs as they took quite a bit of effort to make them fast. And there are plenty of alternatives with extended syntax and features. Best of luck.
- _iostreamer_ 10y ago
  I understand and agree. You could see this thing as me riding the JS hype train.
- erikrothoff 10y ago
  I'm sorry but that seems to be some of the least intuitive code I have ever seen. What are the practical applications of knowing AWK over knowing JS? (Besides the shell voodoo). I'm not trying to start a language war, I just want to understand why learning a completely new set of grammars with a very limited domain would be worth the effort.
  - brianshaler 10y ago
    I think of it as a poorly designed DSL. Still, a poorly designed DSL, once memorized, is way more practical on the command line than typing out javascript function calls, even with the fat arrow syntax. Maybe it's a declarative vs imperative kind of thing. I hate having to reference manpages or use trial and error every time I need to sed/awk, but I don't know if wrapping commands in JS functions and results in Array.prototype is the best solution.
  - deuill 10y ago
    Awk is actually very intuitive once you learn the basics (should take about an hour). Awk is essentially a line-oriented pattern matching language, that is, a collection of patterns -> actions that are applied to every line of input. Syntax is very similar to C, but feels much more lightweight. To decompose the example above: NR > 1 { print $1, $4 } There are two parts to this, the pattern (the part outside the brackets), and the action (the part inside the brackets). Every line parsed is split into fields and stored in $1..$NR, where NR is the number of fields. The entire line is also available in the $0 variable. The default separator is the space character, though that can be changed. So, knowing the above semantics, the meaning of the above example should be clear: If the number of fields for this line is larger than 1, print the first and fourth field of the line. It's a very powerful paradigm, and you can do crazy stuff with Awk. Examples are an x86 assembler [0] and a SASS-style CSS preprocessor [1] (plugging myself there a bit). Unix coreutils are very powerful once you're familiar with them. [0]: http://doc.cat-v.org/henry_spencer/amazing_awk_assembler/ http://doc.cat-v.org/henry_spencer/amazing_awk_assembler/ [1]: https://github.com/deuill/fawkss https://github.com/deuill/fawkss
nhemsley 10y ago
Hey, This is very cool. It does remind me of powershell in it's aims & inspiration. But agree with nailer that using json (or javascript data structures) would hopefully end up with a better, more cohesive shell in awkward. I no longer use Powershell (I got sick of the microsoft ecosystem) I'm not sure what format Powershell uses to pipe structured data around, but it seemed to be some kind of ad-hoc 'whatever-they-came-up-with-at-the-time' mish mash, instead of using say, JSON, and leveraging the power of arrays, dictionaries et al. In fairness json was just starting when I was using Powershell, so it wouldnt have been on their radar. One question, How are you turning output from ls into arrays? Or is ls just a javascript function that behaves like ls? Oh, also, what strategies do you envision for wrapping arbitrary commands, or creating an ecosystem of wrappers? Perhaps some kind of plugin system? I am currently also getting excited by hyperterm, and wonder if there is some kind of fusion there with your project (that is getting _alot_ of traction). Cheers for the cool software!
- _iostreamer_ 10y ago
  Thank you! Regarding how I structure the command output, this is the module which does the job https://github.com/iostreamer-X/Awkward/blob/master/terminal/structure.coffee https://github.com/iostreamer-X/Awkward/blob/master/terminal... It splits on basis of '\n' and removes empty chars, then finally pushes the cleaned output in an array. Regarding, the plugin system. Well, honestly I haven't thought about it but I do like the sound of it. Thanks for sharing the idea. I am yet to try hyperterm but from what I have seen, it looks sooo awesome, and I think things could get 'awkward' there :p
- nhemsley 10y ago
  So Ive actually been looking through the source for awkward, and it led me down the rabbit hole to shelljs, and cash. shelljs actually reimpliments alot of shell commands in javascript, eventually mutating the output of the functions into flat strings here: https://github.com/shelljs/shelljs/blob/master/src/common.js#L93 https://github.com/shelljs/shelljs/blob/master/src/common.js... The point being that the outputs of each of these functions is available (technically, it may require refactoring to get at the output pre-stringified) without having to parse formatted text. Alot of work has been done here.
lhnz 10y ago
I wonder how long it will take before something like this gets implemented as a `Hyperterm` [0] plugin. I think it could be quite useful to combine `npm` packages with shell commands and `map`, `filter`, etc, in your terminal. The only thing I am unsure about are the underlying representations of the shell output. Arrays of arrays are not very semantic, so the code that parses them would not be very pleasant. Given a shape like: [ [ 'PID', 'TTY', 'TIME', 'CMD' ], [ '13750', 'pts/14', '00:00:00', 'bash' ], [ '25193', 'pts/14', '00:00:03', 'node' ], [ '25283', 'pts/14', '00:00:00', 'sh' ], [ '25284', 'pts/14', '00:00:00', 'ps' ] ] I would probably like to be able to apply some kind of transforms by default so that I can work with: [ { pid: '13750', tty: 'pts/14', time: '00:00:00', cmd: 'bash' }, { pid: '25193', tty: 'pts/14', time: '00:00:03', cmd: 'node' }, { pid: '25283', tty: 'pts/14', time: '00:00:00', cmd: 'sh' }, { pid: '25284', tty: 'pts/14', time: '00:00:00', cmd: 'ps' } ] Though if this was the case you would want some way of applying the right transform depending on the arguments passed into the command. I guess the bad thing is that there is currently presumably no way of using other shell features like pipes. Also, if you're going to start outputting different shaped data, then I think you should consider hooking this all up to `flowtype` [1] and creating type signatures to provide DX [2] [0] https://github.com/zeit/hyperterm https://github.com/zeit/hyperterm [1] https://flowtype.org/ https://flowtype.org/ [2] https://news.ycombinator.com/item?id=12129026 https://news.ycombinator.com/item?id=12129026
nikolay 10y ago
Why do people start new projects in CoffeeScript?!
- Klathmon 10y ago
  Because it's a language just like any other? It's not like it's deprecated or anything.
  - nikolay 10y ago
    It's irrelevant given TypeScript and ES2015+.
    - Klathmon 10y ago
      I'd say it's far from irrelevant. The fact that it uses whitespace instead of braces is a big plus for people used to that, and generally you write less coffeescript compared to TS or ES. Plus it has things like switch expressions, list comprehensions, chained comparisons, the safe nav operator, and a ton of other little features that either don't show up in TypeScript or ES. And it's very simple (no configuration needed), and compiles to very readable javascript which means converting to plain JS is very easy (hell, there's even a purpose built tool called decaffeinate which will translate it directly to ES2015) You might not like the language, I don't really like it all that much anymore either, but it's not irrelevant, and for a small personal project like this it seems like a great choice which is easy to switch away from if needed.
      - nikolay 10y ago
        Well, its declining popularity makes it irrelevant.
- nameless912 10y ago
  For the same reason people don't write every program in binary using a magnetized needle straight to their hard disk. People like abstractions, and this guy happened to like CoffeeScript as his abstraction du jour.
sebastianconcpt 10y ago
Making all in a terminal an object is very interesting!! I'm curious about applicability though. What can you do with this that you wouldn't normally do in a traditional term?
nikolay 10y ago
I tend to like ShellJS a lot recently. And yesterday I found this: https://news.ycombinator.com/item?id=12125660 https://news.ycombinator.com/item?id=12125660
__b__ 10y ago
The following comment could apply to hundreds of similar submissions to HN in recent years. Many people -- many of them young people I suspect -- have invested a lot of time learning Javascript. It has become a very popular language. But it remains to be seen whether Javascript will be as long lasting as the UNIX shell and standard utilties. Historically speaking, computer languages have been known to fall into and out of popular usage. For people who invested a lot of time learning the shell and ubiquitous utilties such as AWK, it appears the investment has paid off. I'm not too worried about the terminal disappearing any time soon. How long until the next submission that aims to abtract away the need to learn how to use a UNIX terminal -- directly. There are probably hundreds more on Github alone. What if we conslidated them all in one place: 1001 attempts to abstract away the need to learn UNIX. By no means am I suggesting these attempts have not been successful. What I'm suggesting is that the need for them will not abate. It could be that UNIX terminals, and programs like AWK, are not just a fad.
- sdegutis 10y ago
  They're going to continue to pop up, just like JavaScript frameworks will. Because they're not popping up due to any necessity, but because someone (1) learned JavaScript, (2) learned some shell commands, (3) had the epiphany that they could abstract shell commands away using JavaScript, and (4) isn't aware of the previous thousand attempts by others to do the same thing.
  - Klathmon 10y ago
    That's incredibly dismissive. AWK isn't perfect (nothing is). So that means that there are still improvements that can be made to it. One of the improvements to AWK (or something awk-like) would be to make it have a more familiar syntax to those who don't already know awk. I'll admit, the only thing I use awk for currently is to pull out columns of the output to pass on to the next tool. I know it can do more, but I don't have time to really learn and understand the langauge. While I don't like many things about this project, I really like some of the ideas that it's suggesting. I'd love a "awk but with javascript" where you can use a subset of javascript to work like awk where you can pipe in data and have it output "regular" lines that can be passed to the next tool on the command line. letting me use familiar javascript functions and syntax to map/reduce/filter over lines of input and output plaintext would be amazing. This doesn't quite do that in the way I want, but it's a cool thing nonetheless.
mholmes680 10y ago
FWIW, I've been using Groovy for this purpose to encapsulate processes in *Nix and Windows. Not perfect, not lightweight, but happens to get the job done.
- vorg 10y ago
  Apache Groovy's perfect for these kind of tasks where it doesn't need to be lightweight. Just don't use it for building actual applications -- build them in a decent typed language like Java, Scala, or Kotlin, then use Groovy as a multi-OS version of Bash to automate builds, tests, and runs.
Klathmon 10y ago
how can you apply flags to the command? For example, I use `ps -ef` a lot, how would I do that with this system?
- _iostreamer_ 10y ago
  This seems to work, (ps -ef)().map((a)=>{return [a[0],a[7]]}).forEach((a)=>{console.log(a)})
  - Klathmon 10y ago
    Ah, put the whole command in parens! It might make sense to put a note in the README about that, unless it's already there and I missed it somehow...
    - _iostreamer_ 10y ago
      I am sorry, it slipped from my mind. I will add it now.
      - dc2 10y ago
        Just a note, _iostreamer_, you can use `command.parse` in Vorpal to pre-append parens to every command before it's evaluated (your project is written in Vorpal). BTW, love this idea. - the Vorpal guy
nameless912 10y ago
Normally I have a severe allergic reaction to anyone trying to do anything serious with Javascript, but this looks like a legitimately good idea and a decent innovation on how shells work. I'm beginning to think that it isn't Javascript that I hate, it's the way people use it (i.e. to build bloated, slow, terrible webapps). But this is actually pretty cool. JS is shockingly decent at text processing so repurposing it as a shell language kinda makes sense.
- Klathmon 10y ago
  It's a bit verbose for a shell language (all the async stuff tends to just get in the way in my experience using it like that), but it's great in the sense that everything can be treated as "text" and the functional bits work well when parsing over lines.
HelloImDumb 10y ago
49 comments and no one points out that this is in no way a terminal emulator? It's a (rather rudimentary) shell.
spdustin 10y ago
I'm imagining a world where coreutils and built-ins all respect an environment variable, say STDOUT_JSON=1, and would output JSON. That would be a great start. Especially if the JSON-Schema could be output with a new built-in, manj. How Herculean of a task is this?
- amelius 10y ago
  Not a good idea. Imagine that you invoke some program A that in turn invokes a coreutils program B. Imagine that A has been written before today (in other words, it is not aware of the STDOUT_JSON technique). Since A and B inherit the STDOUT_JSON environment variable, A will receive JSON. But it is not expecting this, so it will probably crash.
  - spdustin 10y ago
    Thanks for that answer, that lead me to learn and understand more about standard streams and file descriptors than I ever cared to before! That's a good thing. :) So then, what about this approach: since the de-facto standard streams are STDIN, STDOUT and STDERR using file descriptors 0, 1 and 2, respectively, are there three more reserved numbers (I couldn't find any in my google-glance) that could be used for, say, JSONIN, JSONOUT and JSONERR? I think the heart of the matter is: unless it becomes part of the POSIX standard, is there any hope on agreeing on an informal "UNIX philosophy" sort of approach that would allow coreutils and shell built-ins to accept and pass structured responses in any kind of object form?
AstroJetson 10y ago
Sorry I'm late to this, but I'm a big AWK fan, I use it daily and have done so for years. For sysadmin things its great for doing data manipulation in shell scripts. I get that Awkward is a shell, and I can see the use. But it seems very clunky to use. This AWK script was posted in this thread ps | awk 'NR>1{ print $1, $4}' It takes the output of the PS command, tosses the header away and then prints the value of the first and fourth fields out. This data can be piped to other programs for additional processing. The NR>1 idiom is pretty well known after you look at a few example scripts that use it. To replicate the scrip that the Awkward video uses, it's just ps | awk '{ print $1, $4}' It's shorter than the Awkward example. I think it's simple, for every line you get from PS print the first set of text and the fourth set of text. Not sure what Awkward would need to do to skip the header. @erikrothoff makes a point about whats the value in knowing AWK over JS. I'd like to think that it gives you a powerful portable tool that can be used as either a stand alone application or as part of a shell script. If you are a Windows user, then it's not going to fit as well, but if you are a Unix/Linux/OS X user, it's a worthy tool to add to your arsenal. The syntax may be "some of the least intuitive code" but investing about 20 minutes can fix that. There are lots of tutorials that can give you a quick leg up. There are also a few "one line AWK scripts" collections that are good samples. One of the most powerful things about AWK is the ability to use regular expressions to see if you want to process the record. The ability to go through a file, pluck out the record(s) and process them is very nice. ps | awk '/astro/ {print $1, $4}' will give me any user that has astro in it. Not sure how that would work in Awkward. The use of regular expressions like this is what makes AWK a go to language for text processing. Not here to add to the AWK vs Javascript debate, but wanted to add AWK is a powerful tool, having it in your toolbox is a good thing.

76 points by _iostreamer_ 10y ago

This is great! I'll just let this evolve a bit, and wait for broader community support, and then I'll happily replace bash by this shell!

_iostreamer_ 10y ago
I would love if that happened :D

nailer 10y ago

This is good, but the data structures - eg arrays of arrays - don't really match the underlying data. Imagine what 'ip addr' or 'ifconfig' would look like - they output paragraphs rather than lines, so scraping lines wouldn't produce good output. It'd be better - and FAR more work - to make an object pipeline based powershell equivalent for Nix, based on JSON. You could write cmdlets in any language that outputs JSON and do better at this project's goal of avoiding scraping. ps node | select pid instead of: ps().find(function(line){ return line[3] === 'node' })[0] Since the fields have keys, you avoid magic numbers and the code's easier to read. (you could of course alias those to make 'pidof node' like Linux distros do) The end result would be much better than Powershell, since it would use JSON and not be tied to .net languages.

_iostreamer_ 10y ago
That sounds so good and better. Thank you, I will work on it.
- ramses0 10y ago
  Consider: wrapPs( "ps -ef" ) => [ {pid: 1234, name: "foo.sh" }, ... ] wrapLs( "ls -al" ) => [ ... ] (not the syntax but the idea). You want to allow interested individuals to write the wrappers "forcefully" against uncooperative maintainers (separate people, separate ideas, separate motivations, separate release cycle, etc). Eventually, some enlightened shell-command owners might add "ls -al --json" or "ps -ef --json", "git stat --json" which removes the need for a wrapper script, but the wrapper script allows innovation peripheral to the core without affecting the core until finally due to overwhelming support the core is extended with something "good" and agreed upon via consensus usage. For the short term, perhaps if "ps" and "ps.wrap" exist in your path, your awkward shell can inject the "* .wrap" automatically around the given command in a way that doesn't require a lot of typing, either automatically, or via shortened syntax. (ps -ef).map(...) !(ps -ef).map(...) {{ ps -ef }}.map(...) I don't know but eventually you'd want to get to a point where you're bridging between the interactive use case (ps -ef printing a nice text table) and the pipeline use case (ps -ef --json letting you easily access different fields), maybe {{ ps -ef }} either calls * .wrap if it exists or if it doesn't exist calls it with --json (this assumes the command supports --json directly) Regardless, this idea has some legs and I look forward to seeing how much traction it gets.

devnonymous 10y ago

Neat hack! I like the idea. Initially when I read the title I thought it would be yet another 'make my shell behave more like my preferred language' type of project. Those while fun to code and play with aren't really useful in the long run. This though might just be useful.

_iostreamer_ 10y ago
Thank you!

itaysk 10y ago

Like. But PowerShell still rules. Wish it was oss and x-plat

_iostreamer_ 10y ago
I am gonna try PowerShell now. Never used it.
- feiss 10y ago
  It's really nice, althought you have to like it's syntax.. Google some cheatsheets, will be helpful Its Object Oriented approach it's what many try to emulate in mac/linux. This new hype of improving the CLIs is exciting!
nikolay 10y ago
Microsoft seems to be working on porting PowerShell to Linux already [0] and there's a partial port using Mono [1]! [0]: https://windowsserver.uservoice.com/forums/301869-powershell/suggestions/11689284-powershell-core-for-linux https://windowsserver.uservoice.com/forums/301869-powershell... [1]: https://pash-project.github.io/ https://pash-project.github.io/
- oblio 10y ago
  Oh, I see that pash was revived. It had been abandoned since 2009 (I think).

agentultra 10y ago

Uh... ps | awk 'NR>1{ print $1, $4}' I'll give that the syntax of these shell programs take some work to understand but they're hard to beat for the purpose they serve. If you're parsing plenty of JSON these days I find 'jq' indispensable. It's basically awk/sed for JSON. I would just be sure to study those programs as they took quite a bit of effort to make them fast. And there are plenty of alternatives with extended syntax and features. Best of luck.

_iostreamer_ 10y ago
I understand and agree. You could see this thing as me riding the JS hype train.
erikrothoff 10y ago
I'm sorry but that seems to be some of the least intuitive code I have ever seen. What are the practical applications of knowing AWK over knowing JS? (Besides the shell voodoo). I'm not trying to start a language war, I just want to understand why learning a completely new set of grammars with a very limited domain would be worth the effort.
- brianshaler 10y ago
  I think of it as a poorly designed DSL. Still, a poorly designed DSL, once memorized, is way more practical on the command line than typing out javascript function calls, even with the fat arrow syntax. Maybe it's a declarative vs imperative kind of thing. I hate having to reference manpages or use trial and error every time I need to sed/awk, but I don't know if wrapping commands in JS functions and results in Array.prototype is the best solution.
- deuill 10y ago
  Awk is actually very intuitive once you learn the basics (should take about an hour). Awk is essentially a line-oriented pattern matching language, that is, a collection of patterns -> actions that are applied to every line of input. Syntax is very similar to C, but feels much more lightweight. To decompose the example above: NR > 1 { print $1, $4 } There are two parts to this, the pattern (the part outside the brackets), and the action (the part inside the brackets). Every line parsed is split into fields and stored in $1..$NR, where NR is the number of fields. The entire line is also available in the $0 variable. The default separator is the space character, though that can be changed. So, knowing the above semantics, the meaning of the above example should be clear: If the number of fields for this line is larger than 1, print the first and fourth field of the line. It's a very powerful paradigm, and you can do crazy stuff with Awk. Examples are an x86 assembler [0] and a SASS-style CSS preprocessor [1] (plugging myself there a bit). Unix coreutils are very powerful once you're familiar with them. [0]: http://doc.cat-v.org/henry_spencer/amazing_awk_assembler/ http://doc.cat-v.org/henry_spencer/amazing_awk_assembler/ [1]: https://github.com/deuill/fawkss https://github.com/deuill/fawkss

nhemsley 10y ago

Hey, This is very cool. It does remind me of powershell in it's aims & inspiration. But agree with nailer that using json (or javascript data structures) would hopefully end up with a better, more cohesive shell in awkward. I no longer use Powershell (I got sick of the microsoft ecosystem) I'm not sure what format Powershell uses to pipe structured data around, but it seemed to be some kind of ad-hoc 'whatever-they-came-up-with-at-the-time' mish mash, instead of using say, JSON, and leveraging the power of arrays, dictionaries et al. In fairness json was just starting when I was using Powershell, so it wouldnt have been on their radar. One question, How are you turning output from ls into arrays? Or is ls just a javascript function that behaves like ls? Oh, also, what strategies do you envision for wrapping arbitrary commands, or creating an ecosystem of wrappers? Perhaps some kind of plugin system? I am currently also getting excited by hyperterm, and wonder if there is some kind of fusion there with your project (that is getting _alot_ of traction). Cheers for the cool software!

_iostreamer_ 10y ago
Thank you! Regarding how I structure the command output, this is the module which does the job https://github.com/iostreamer-X/Awkward/blob/master/terminal/structure.coffee https://github.com/iostreamer-X/Awkward/blob/master/terminal... It splits on basis of '\n' and removes empty chars, then finally pushes the cleaned output in an array. Regarding, the plugin system. Well, honestly I haven't thought about it but I do like the sound of it. Thanks for sharing the idea. I am yet to try hyperterm but from what I have seen, it looks sooo awesome, and I think things could get 'awkward' there :p
nhemsley 10y ago
So Ive actually been looking through the source for awkward, and it led me down the rabbit hole to shelljs, and cash. shelljs actually reimpliments alot of shell commands in javascript, eventually mutating the output of the functions into flat strings here: https://github.com/shelljs/shelljs/blob/master/src/common.js#L93 https://github.com/shelljs/shelljs/blob/master/src/common.js... The point being that the outputs of each of these functions is available (technically, it may require refactoring to get at the output pre-stringified) without having to parse formatted text. Alot of work has been done here.

lhnz 10y ago

I wonder how long it will take before something like this gets implemented as a `Hyperterm` [0] plugin. I think it could be quite useful to combine `npm` packages with shell commands and `map`, `filter`, etc, in your terminal. The only thing I am unsure about are the underlying representations of the shell output. Arrays of arrays are not very semantic, so the code that parses them would not be very pleasant. Given a shape like: [ [ 'PID', 'TTY', 'TIME', 'CMD' ], [ '13750', 'pts/14', '00:00:00', 'bash' ], [ '25193', 'pts/14', '00:00:03', 'node' ], [ '25283', 'pts/14', '00:00:00', 'sh' ], [ '25284', 'pts/14', '00:00:00', 'ps' ] ] I would probably like to be able to apply some kind of transforms by default so that I can work with: [ { pid: '13750', tty: 'pts/14', time: '00:00:00', cmd: 'bash' }, { pid: '25193', tty: 'pts/14', time: '00:00:03', cmd: 'node' }, { pid: '25283', tty: 'pts/14', time: '00:00:00', cmd: 'sh' }, { pid: '25284', tty: 'pts/14', time: '00:00:00', cmd: 'ps' } ] Though if this was the case you would want some way of applying the right transform depending on the arguments passed into the command. I guess the bad thing is that there is currently presumably no way of using other shell features like pipes. Also, if you're going to start outputting different shaped data, then I think you should consider hooking this all up to `flowtype` [1] and creating type signatures to provide DX [2] [0] https://github.com/zeit/hyperterm https://github.com/zeit/hyperterm [1] https://flowtype.org/ https://flowtype.org/ [2] https://news.ycombinator.com/item?id=12129026 https://news.ycombinator.com/item?id=12129026

nikolay 10y ago

Why do people start new projects in CoffeeScript?!

Klathmon 10y ago
Because it's a language just like any other? It's not like it's deprecated or anything.
- nikolay 10y ago
  It's irrelevant given TypeScript and ES2015+.
  - Klathmon 10y ago
    I'd say it's far from irrelevant. The fact that it uses whitespace instead of braces is a big plus for people used to that, and generally you write less coffeescript compared to TS or ES. Plus it has things like switch expressions, list comprehensions, chained comparisons, the safe nav operator, and a ton of other little features that either don't show up in TypeScript or ES. And it's very simple (no configuration needed), and compiles to very readable javascript which means converting to plain JS is very easy (hell, there's even a purpose built tool called decaffeinate which will translate it directly to ES2015) You might not like the language, I don't really like it all that much anymore either, but it's not irrelevant, and for a small personal project like this it seems like a great choice which is easy to switch away from if needed.
    - nikolay 10y ago
      Well, its declining popularity makes it irrelevant.
nameless912 10y ago
For the same reason people don't write every program in binary using a magnetized needle straight to their hard disk. People like abstractions, and this guy happened to like CoffeeScript as his abstraction du jour.

sebastianconcpt 10y ago

Making all in a terminal an object is very interesting!! I'm curious about applicability though. What can you do with this that you wouldn't normally do in a traditional term?

nikolay 10y ago

I tend to like ShellJS a lot recently. And yesterday I found this: https://news.ycombinator.com/item?id=12125660 https://news.ycombinator.com/item?id=12125660

__b__ 10y ago

The following comment could apply to hundreds of similar submissions to HN in recent years. Many people -- many of them young people I suspect -- have invested a lot of time learning Javascript. It has become a very popular language. But it remains to be seen whether Javascript will be as long lasting as the UNIX shell and standard utilties. Historically speaking, computer languages have been known to fall into and out of popular usage. For people who invested a lot of time learning the shell and ubiquitous utilties such as AWK, it appears the investment has paid off. I'm not too worried about the terminal disappearing any time soon. How long until the next submission that aims to abtract away the need to learn how to use a UNIX terminal -- directly. There are probably hundreds more on Github alone. What if we conslidated them all in one place: 1001 attempts to abstract away the need to learn UNIX. By no means am I suggesting these attempts have not been successful. What I'm suggesting is that the need for them will not abate. It could be that UNIX terminals, and programs like AWK, are not just a fad.

sdegutis 10y ago
They're going to continue to pop up, just like JavaScript frameworks will. Because they're not popping up due to any necessity, but because someone (1) learned JavaScript, (2) learned some shell commands, (3) had the epiphany that they could abstract shell commands away using JavaScript, and (4) isn't aware of the previous thousand attempts by others to do the same thing.
- Klathmon 10y ago
  That's incredibly dismissive. AWK isn't perfect (nothing is). So that means that there are still improvements that can be made to it. One of the improvements to AWK (or something awk-like) would be to make it have a more familiar syntax to those who don't already know awk. I'll admit, the only thing I use awk for currently is to pull out columns of the output to pass on to the next tool. I know it can do more, but I don't have time to really learn and understand the langauge. While I don't like many things about this project, I really like some of the ideas that it's suggesting. I'd love a "awk but with javascript" where you can use a subset of javascript to work like awk where you can pipe in data and have it output "regular" lines that can be passed to the next tool on the command line. letting me use familiar javascript functions and syntax to map/reduce/filter over lines of input and output plaintext would be amazing. This doesn't quite do that in the way I want, but it's a cool thing nonetheless.

mholmes680 10y ago

FWIW, I've been using Groovy for this purpose to encapsulate processes in *Nix and Windows. Not perfect, not lightweight, but happens to get the job done.

vorg 10y ago
Apache Groovy's perfect for these kind of tasks where it doesn't need to be lightweight. Just don't use it for building actual applications -- build them in a decent typed language like Java, Scala, or Kotlin, then use Groovy as a multi-OS version of Bash to automate builds, tests, and runs.

Klathmon 10y ago

how can you apply flags to the command? For example, I use `ps -ef` a lot, how would I do that with this system?

_iostreamer_ 10y ago
This seems to work, (ps -ef)().map((a)=>{return [a[0],a[7]]}).forEach((a)=>{console.log(a)})
- Klathmon 10y ago
  Ah, put the whole command in parens! It might make sense to put a note in the README about that, unless it's already there and I missed it somehow...
  - _iostreamer_ 10y ago
    I am sorry, it slipped from my mind. I will add it now.
    - dc2 10y ago
      Just a note, _iostreamer_, you can use `command.parse` in Vorpal to pre-append parens to every command before it's evaluated (your project is written in Vorpal). BTW, love this idea. - the Vorpal guy

nameless912 10y ago

Normally I have a severe allergic reaction to anyone trying to do anything serious with Javascript, but this looks like a legitimately good idea and a decent innovation on how shells work. I'm beginning to think that it isn't Javascript that I hate, it's the way people use it (i.e. to build bloated, slow, terrible webapps). But this is actually pretty cool. JS is shockingly decent at text processing so repurposing it as a shell language kinda makes sense.

Klathmon 10y ago
It's a bit verbose for a shell language (all the async stuff tends to just get in the way in my experience using it like that), but it's great in the sense that everything can be treated as "text" and the functional bits work well when parsing over lines.

HelloImDumb 10y ago

49 comments and no one points out that this is in no way a terminal emulator? It's a (rather rudimentary) shell.

spdustin 10y ago

I'm imagining a world where coreutils and built-ins all respect an environment variable, say STDOUT_JSON=1, and would output JSON. That would be a great start. Especially if the JSON-Schema could be output with a new built-in, manj. How Herculean of a task is this?

amelius 10y ago
Not a good idea. Imagine that you invoke some program A that in turn invokes a coreutils program B. Imagine that A has been written before today (in other words, it is not aware of the STDOUT_JSON technique). Since A and B inherit the STDOUT_JSON environment variable, A will receive JSON. But it is not expecting this, so it will probably crash.
- spdustin 10y ago
  Thanks for that answer, that lead me to learn and understand more about standard streams and file descriptors than I ever cared to before! That's a good thing. :) So then, what about this approach: since the de-facto standard streams are STDIN, STDOUT and STDERR using file descriptors 0, 1 and 2, respectively, are there three more reserved numbers (I couldn't find any in my google-glance) that could be used for, say, JSONIN, JSONOUT and JSONERR? I think the heart of the matter is: unless it becomes part of the POSIX standard, is there any hope on agreeing on an informal "UNIX philosophy" sort of approach that would allow coreutils and shell built-ins to accept and pass structured responses in any kind of object form?

AstroJetson 10y ago

Sorry I'm late to this, but I'm a big AWK fan, I use it daily and have done so for years. For sysadmin things its great for doing data manipulation in shell scripts. I get that Awkward is a shell, and I can see the use. But it seems very clunky to use. This AWK script was posted in this thread ps | awk 'NR>1{ print $1, $4}' It takes the output of the PS command, tosses the header away and then prints the value of the first and fourth fields out. This data can be piped to other programs for additional processing. The NR>1 idiom is pretty well known after you look at a few example scripts that use it. To replicate the scrip that the Awkward video uses, it's just ps | awk '{ print $1, $4}' It's shorter than the Awkward example. I think it's simple, for every line you get from PS print the first set of text and the fourth set of text. Not sure what Awkward would need to do to skip the header. @erikrothoff makes a point about whats the value in knowing AWK over JS. I'd like to think that it gives you a powerful portable tool that can be used as either a stand alone application or as part of a shell script. If you are a Windows user, then it's not going to fit as well, but if you are a Unix/Linux/OS X user, it's a worthy tool to add to your arsenal. The syntax may be "some of the least intuitive code" but investing about 20 minutes can fix that. There are lots of tutorials that can give you a quick leg up. There are also a few "one line AWK scripts" collections that are good samples. One of the most powerful things about AWK is the ability to use regular expressions to see if you want to process the record. The ability to go through a file, pluck out the record(s) and process them is very nice. ps | awk '/astro/ {print $1, $4}' will give me any user that has astro in it. Not sure how that would work in Awkward. The use of regular expressions like this is what makes AWK a go to language for text processing. Not here to add to the AWK vs Javascript debate, but wanted to add AWK is a powerful tool, having it in your toolbox is a good thing.