6 ms·

FuncShell – A Haskell-based alternative to awk

141 points by _iostreamer_ 9y ago

tyingq 9y ago
Similar idea, but uses sqlite to make an awk like utility with sql syntax: https://github.com/dbohdan/sqawk https://github.com/dbohdan/sqawk
- deleted 9y ago
  [deleted]
- deleted 9y ago
  [deleted]
- jaytaylor 9y ago
  And just yesterday, people were asking [0] what Tcl is good for! This is a great example. [0] https://news.ycombinator.com/item?id=14229266 https://news.ycombinator.com/item?id=14229266
- ams6110 9y ago
  There's also q, which I've happily used for all sorts of data munging tasks: https://harelba.github.io/q/ https://harelba.github.io/q/ One example from that page: ps -ef | q -H "SELECT UID,COUNT(*) cnt FROM - GROUP BY UID ORDER BY cnt DESC LIMIT 3"
rocqua 9y ago
I'm amazed by the powerline-command prompt. Shiny things seem to draw my attention a bit too much.
- c0achmcguirk 9y ago
  Haha, me too! I want to read an article on getting my zsh prompt to look like that.
  - baldfat 9y ago
    Windows - https://amreldib.com/blog/CustomizeWindowsCmderPrompt/ https://amreldib.com/blog/CustomizeWindowsCmderPrompt/ zsh - https://gist.githubusercontent.com/agnoster/3712874/raw/c3107c06c04fb42b0ca27b0a81b15854819969c6/agnoster.zsh-theme https://gist.githubusercontent.com/agnoster/3712874/raw/c310... powerline theme on github - https://github.com/banga/powerline-shell https://github.com/banga/powerline-shell
heinrichhartman 9y ago
Similar idea, but uses LUA as inline language: https://github.com/HeinrichHartmann/luawk https://github.com/HeinrichHartmann/luawk ``` > printf abc | luawk -p '_:gsub("a","A")' Abc ```
- srean 9y ago
  An awk with a luajit backend would be quite interesting.
codewithcheese 9y ago
I like how you made this version use pipes instead of being a shell. Do you have any plans to create a new JS version of Awkward using pipes?
- _iostreamer_ 9y ago
  I understand that this caters to a very specific audience which knows haskell. Hence I plan to support JS as the inline language. I don't want to create a new JS version because I want to learn more of Haskell :p
ilurkedhere 9y ago
In typical usage, do you need the type hints much? I saw ":: Float" in there for the df example, for instance.
- _iostreamer_ 9y ago
  Not really, in the example of df, I was filtering on the basis of size, which is not int usually but a decimal value, hence I had to provide the type hint.
baldfat 9y ago
I always have mixed feelings on awk alternatives. I find awk to be one of the most underused tools in scripting. Awk has saved me numerous of times. Now replacing latex with a modern syntax would be awesome!!!
- RodericDay 9y ago
  I tried to explain to a historian with an open source textbook initiative the basics of latex. As soon as I opened the demo page that had looked nice enough, with him by my side, my heart sank. It's just so far away from being usable by non-technical users. It's too bad, really.
  - baldfat 9y ago
    A good theme and sharelatex.com can go a long way to get someone to use it. I usually only get Math Majors to invest in it. I did almost all my Master Theology papers in it. Just difficult to use different languages. I usually have to use one or all of these Greek, Hebrew, Aramaic, Latin, Danish or German in my papers. Latex didn't make that easy. Actually nothing does.
    - RodericDay 9y ago
      I used sharelatex. the notation full of \, {, }, etc. is just too confusing for people.
      - Jach 9y ago
        What isn't too confusing then? Markdown and HTML are just riffs, it's all based around the idea of opening/closing tokens and special tag names that do different things. I think if there is in fact a technical barrier, then it's the underlying fact that your representation of a document has a separation of content and design/layout. If people can't deal with that, then they can only use WYSIWYG (and not the hacks of rendering to the side while you still type in TeX), and there's no point in trying to update LaTeX's notation to fit people who never will get that underlying separation.
jaytaylor 9y ago
Do any of you know how they got such a beautiful looking shell [0]? I'd love to know if there's a clean way to achieve such a nice result. [0] https://github.com/iostreamer-X/FuncShell/raw/master/fsh.gif https://github.com/iostreamer-X/FuncShell/raw/master/fsh.gif EDIT As an aside, FuncShell looks like a beautiful demonstration of how powerful Haskell can be. The source code is terse and clean. The only oddity I noticed isthe rather large 68MB binary file committed into master in the repository [1]. Why do people do this? Especially on GitHub where there's support for cutting releases with a binary for each targeted platform... [1] https://github.com/iostreamer-X/FuncShell/blob/master/fsh https://github.com/iostreamer-X/FuncShell/blob/master/fsh
- cpa 9y ago
  Try zsh + oh-my-zsh https://github.com/robbyrussell/oh-my-zsh https://github.com/robbyrussell/oh-my-zsh I think the gif you linked is the vanilla oh-my-zsh theme.
- suprfnk 9y ago
  That's a pretty standard zsh with oh-my-zsh prompt: https://github.com/robbyrussell/oh-my-zsh https://github.com/robbyrussell/oh-my-zsh Scroll down a bit and you'll see it. Though I personally prefer zsh with prezto, because oh-my-zsh feels too bloated and heavy: https://github.com/sorin-ionescu/prezto https://github.com/sorin-ionescu/prezto
  - jorvi 9y ago
    Prezto is abandoned. If you absolutely need a framework go for Zim, although I like the simplicity of Zplug and adding plugins or themes as I need them.
    - ujal 9y ago
      The repo doesn't seem abondoned to me.
      - jorvi 9y ago
omaranto 9y ago
Similar tools that use Python as the query language: - The Pyed Piper: https://code.google.com/archive/p/pyp/ https://code.google.com/archive/p/pyp/ - Puffin: https://github.com/kespindler/puffin https://github.com/kespindler/puffin
rnhmjoj 9y ago
What happened to hell[1]? It was apparently destroyed by this commit[2]. [1]: https://www.reddit.com/r/programming/comments/1r7oci/hell_a_haskell_shell/ https://www.reddit.com/r/programming/comments/1r7oci/hell_a_... [2]: https://github.com/chrisdone/hell/commit/f34259e5e92f6edf464b65d9735965b0533b470b https://github.com/chrisdone/hell/commit/f34259e5e92f6edf464...
gregwebs 9y ago
Nice! How do I bring new function into scope though? This looks very similar to hawk, which has documentation on how to add your custom code [1]. Also, there are some nice CLI tools for JSON [2] or CSV [3] specific data. [1] https://github.com/gelisam/hawk [2] https://stedolan.github.io/jq/manual/ [3] https://github.com/BurntSushi/xsv/blob/master/README.md
- codygman 9y ago
  I think hawk should steal the idea of aliasing "|]" personally ;)

141 points by _iostreamer_ 9y ago

Similar idea, but uses sqlite to make an awk like utility with sql syntax: https://github.com/dbohdan/sqawk https://github.com/dbohdan/sqawk

deleted 9y ago
[deleted]
deleted 9y ago
[deleted]
jaytaylor 9y ago
And just yesterday, people were asking [0] what Tcl is good for! This is a great example. [0] https://news.ycombinator.com/item?id=14229266 https://news.ycombinator.com/item?id=14229266
ams6110 9y ago
There's also q, which I've happily used for all sorts of data munging tasks: https://harelba.github.io/q/ https://harelba.github.io/q/ One example from that page: ps -ef | q -H "SELECT UID,COUNT(*) cnt FROM - GROUP BY UID ORDER BY cnt DESC LIMIT 3"

rocqua 9y ago

I'm amazed by the powerline-command prompt. Shiny things seem to draw my attention a bit too much.

c0achmcguirk 9y ago
Haha, me too! I want to read an article on getting my zsh prompt to look like that.
- baldfat 9y ago
  Windows - https://amreldib.com/blog/CustomizeWindowsCmderPrompt/ https://amreldib.com/blog/CustomizeWindowsCmderPrompt/ zsh - https://gist.githubusercontent.com/agnoster/3712874/raw/c3107c06c04fb42b0ca27b0a81b15854819969c6/agnoster.zsh-theme https://gist.githubusercontent.com/agnoster/3712874/raw/c310... powerline theme on github - https://github.com/banga/powerline-shell https://github.com/banga/powerline-shell

heinrichhartman 9y ago

Similar idea, but uses LUA as inline language: https://github.com/HeinrichHartmann/luawk https://github.com/HeinrichHartmann/luawk ``` > printf abc | luawk -p '_:gsub("a","A")' Abc ```

srean 9y ago
An awk with a luajit backend would be quite interesting.

codewithcheese 9y ago

I like how you made this version use pipes instead of being a shell. Do you have any plans to create a new JS version of Awkward using pipes?

_iostreamer_ 9y ago
I understand that this caters to a very specific audience which knows haskell. Hence I plan to support JS as the inline language. I don't want to create a new JS version because I want to learn more of Haskell :p

ilurkedhere 9y ago

In typical usage, do you need the type hints much? I saw ":: Float" in there for the df example, for instance.

_iostreamer_ 9y ago
Not really, in the example of df, I was filtering on the basis of size, which is not int usually but a decimal value, hence I had to provide the type hint.

baldfat 9y ago

I always have mixed feelings on awk alternatives. I find awk to be one of the most underused tools in scripting. Awk has saved me numerous of times. Now replacing latex with a modern syntax would be awesome!!!

RodericDay 9y ago
I tried to explain to a historian with an open source textbook initiative the basics of latex. As soon as I opened the demo page that had looked nice enough, with him by my side, my heart sank. It's just so far away from being usable by non-technical users. It's too bad, really.
- baldfat 9y ago
  A good theme and sharelatex.com can go a long way to get someone to use it. I usually only get Math Majors to invest in it. I did almost all my Master Theology papers in it. Just difficult to use different languages. I usually have to use one or all of these Greek, Hebrew, Aramaic, Latin, Danish or German in my papers. Latex didn't make that easy. Actually nothing does.
  - RodericDay 9y ago
    I used sharelatex. the notation full of \, {, }, etc. is just too confusing for people.
    - Jach 9y ago
      What isn't too confusing then? Markdown and HTML are just riffs, it's all based around the idea of opening/closing tokens and special tag names that do different things. I think if there is in fact a technical barrier, then it's the underlying fact that your representation of a document has a separation of content and design/layout. If people can't deal with that, then they can only use WYSIWYG (and not the hacks of rendering to the side while you still type in TeX), and there's no point in trying to update LaTeX's notation to fit people who never will get that underlying separation.

jaytaylor 9y ago

Do any of you know how they got such a beautiful looking shell [0]? I'd love to know if there's a clean way to achieve such a nice result. [0] https://github.com/iostreamer-X/FuncShell/raw/master/fsh.gif https://github.com/iostreamer-X/FuncShell/raw/master/fsh.gif EDIT As an aside, FuncShell looks like a beautiful demonstration of how powerful Haskell can be. The source code is terse and clean. The only oddity I noticed isthe rather large 68MB binary file committed into master in the repository [1]. Why do people do this? Especially on GitHub where there's support for cutting releases with a binary for each targeted platform... [1] https://github.com/iostreamer-X/FuncShell/blob/master/fsh https://github.com/iostreamer-X/FuncShell/blob/master/fsh

cpa 9y ago
Try zsh + oh-my-zsh https://github.com/robbyrussell/oh-my-zsh https://github.com/robbyrussell/oh-my-zsh I think the gif you linked is the vanilla oh-my-zsh theme.
suprfnk 9y ago
That's a pretty standard zsh with oh-my-zsh prompt: https://github.com/robbyrussell/oh-my-zsh https://github.com/robbyrussell/oh-my-zsh Scroll down a bit and you'll see it. Though I personally prefer zsh with prezto, because oh-my-zsh feels too bloated and heavy: https://github.com/sorin-ionescu/prezto https://github.com/sorin-ionescu/prezto
- jorvi 9y ago
  Prezto is abandoned. If you absolutely need a framework go for Zim, although I like the simplicity of Zplug and adding plugins or themes as I need them.
  - ujal 9y ago
    The repo doesn't seem abondoned to me.
    - jorvi 9y ago

omaranto 9y ago

Similar tools that use Python as the query language: - The Pyed Piper: https://code.google.com/archive/p/pyp/ https://code.google.com/archive/p/pyp/ - Puffin: https://github.com/kespindler/puffin https://github.com/kespindler/puffin

rnhmjoj 9y ago

What happened to hell[1]? It was apparently destroyed by this commit[2]. [1]: https://www.reddit.com/r/programming/comments/1r7oci/hell_a_haskell_shell/ https://www.reddit.com/r/programming/comments/1r7oci/hell_a_... [2]: https://github.com/chrisdone/hell/commit/f34259e5e92f6edf464b65d9735965b0533b470b https://github.com/chrisdone/hell/commit/f34259e5e92f6edf464...

gregwebs 9y ago

Nice! How do I bring new function into scope though? This looks very similar to hawk, which has documentation on how to add your custom code [1]. Also, there are some nice CLI tools for JSON [2] or CSV [3] specific data. [1] https://github.com/gelisam/hawk [2] https://stedolan.github.io/jq/manual/ [3] https://github.com/BurntSushi/xsv/blob/master/README.md

codygman 9y ago
I think hawk should steal the idea of aliasing "|]" personally ;)