4 ms·
This would have been great 10-20 years ago, or even at the coining of Unix pipes. By today's standards, however, the syntax feels clunky and dated. I'd like to
by esafak 1y ago
This would have been great 10-20 years ago, or even at the coining of Unix pipes. By today's standards, however, the syntax feels clunky and dated. I'd like to see contemporary shells like nushell and elvish copy these ideas, with attribution of course, in a more modern way. That is the best way I can see to honor this stagnant project: https://github.com/dspinellis/dgsh https://github.com/dspinellis/dgsh
- hnlmorg 1y agoMurex has had this capability for years. (https://github.com/lmorg/murex https://github.com/lmorg/murex) I’m on my phone at the moment and cooking so cannot type any examples, but if I get time, I’ll throw together some comparisons later tonight
- esafak 1y agoI could not find any mention of DAGs or directed acyclic graphs in the documentation.
- hnlmorg 1y agoYeah it’s not technically DAG since it uses iteration, but then dgsh will use iteration under the hood too. However Murex does support CSP-style concurrency. So while there’s no syntax sugar for writing graphs, you can very easily create adhoc pipes and pass them around instead of using stdout / stderr. So it wouldn’t actually take much to refine that with some DAG-friendly syntax. In fact maybe that can be my next project…
- DSpinellis 1y agoI'm curious: what do you mean by "dgsh will use iteration under the hood too"? Dgsh does several things under the hood, but I wouldn't characterize any of them as iteration.
- hnlmorg 1y agoYes you’re right. My apologies. I was glancing at the examples while cooking, specifically the git example (https://www2.dmst.aueb.gr/dds/sw/dgsh/#commit-stats https://www2.dmst.aueb.gr/dds/sw/dgsh/#commit-stats) thinking that it was iterating over the lines output from git, but clearly that’s not even how bash would work. That will teach me for commenting without giving something my full attention first doh! Looking properly at this, I can see no iteration is needed. Which actually makes the Murex implementation even easier because Murex already has tee pipes just like dgsh. It’s just not (yet) particularly well documented.
- DSpinellis 1y agoAdmiring your multi-tasking!
- hnlmorg 1y agoHaha thank you but it’s really not that good otherwise I’d have grokked the iteration thing before making a fool out of myself!
- em-bee 1y agowould you be able to share or point to some examples? i am curious.
- hnlmorg 1y agoSorry, it’s been a very busy couple of days so not had 5 minutes to test anything. Stay tuned though, What I’m going to do is write a blog post about it. It’s an interesting enough topic to deserve one
- em-bee 1y agoawesome. i am looking forward to that. that blog on the murex website could use some attention anyways :-)
- zokier 1y agoWell, the project started 12 years ago (as sgsh), so that fits into your 10-20 years ago window :)
- DSpinellis 1y agoI went through two iterations before adopting the current syntax. Truth is neither me nor Doug McIlroy, the inventor of Unix pipes, who kindly and generously provided feedback during dgsh's development, had something better to propose. What syntax would you propose?
- esafak 1y agoGreetings, Diomidis. I would suggest a familiar notation like "[a, b] -> c" in a dedicated dag block: dag text_stats { tee -> [ split_words, count_chars ] # word-based frequencies split_words -> tee_words tee_words -> ngram2 -> save_digram tee_words -> ngram3 -> save_trigram tee_words -> ranked_frequency -> save_words # character-based frequencies count_chars -> add_percentage chars_to_lines -> ranked_frequency -> add_percentage -> save_chars } run text_stats < input.txt https://www2.dmst.aueb.gr/dds/sw/dgsh/#text-properties https://www2.dmst.aueb.gr/dds/sw/dgsh/#text-properties or dag commit_graph { git_log -> filter_recent -> sort -n -> [ uniq_committers, sort_by_email ] uniq_committers -> [ last_commit, first_commit, committer_positions ] [ last_commit, first_commit ] -> cat -> tr '\n' ' ' -> days_between [ committer_positions, sort_by_email ] -> join_by_email -> sort -k2n -> [ make_bitmap_header, plot_per_day ] [ uniq_committers, days_between ] -> emit_dims -> plot_per_day make_bitmap_header -> cat plot_per_day -> morphconv -> [ to_png_large, to_png_small ] } run commit_graph https://www2.dmst.aueb.gr/dds/sw/dgsh/#committer-plot https://www2.dmst.aueb.gr/dds/sw/dgsh/#committer-plot The translations above are computer-assisted and may contain mistakes, but you get the idea.
- DSpinellis 1y agoThank you for the suggestion. This would mean that you'd also then create some mapping from each name (like git_log) to its implementation, right?
- esafak 1y agoYes, using shell functions: git_log() { git log --pretty=tformat:'%at %ae' } Separating function definitions allows you to run, test, and re-use them.
- o11c 1y agoFrankly, I find that anything more than some preparatory `exec {my_fd}< <(commands ...)` is an unmaintainable mess, so bash is plenty for any program that should be implemented in bash.
- DSpinellis 1y agoManually playing around with fds is definitely unmaintainable. My hope is that a clean syntax can help create maintainable complex pipelines.