7 ms·

Show HN: Xc – A Markdown Defined Task Runner

113 points by joerdav 4y ago

snake_case 4y ago
Nice to see other markdown-based task runners! I'm the creator of mask which is another alternative, written in Rust. Our approaches slightly differ. It looks like xc parses the README.md file for commands while mask looks for a maskfile.md by default, though you can provide a --maskfile arg to specify any markdown file that follows the expected format. https://github.com/jacobdeichert/mask https://github.com/jacobdeichert/mask
- teknopaul 4y ago
  Why not a simple well commented shell script? Most code editor will syntax highlight it a lot better.
  - boxed 4y ago
    In the docs for iommi we've tried both the documentation-centric and the code-centric approaches. We started with the documentation-centric approach because we had a lot of documentation and wanted to check if the code examples in them worked. So I wrote a bad parser that extracted the code from rST and put them into python files that we then ran. This worked... ok-ish. Then we switched to the code-centric approach that we use today where we write tests that we generate the documentation from. There are specially marked documentation strings plus some tags for parts of the tests that shouldn't be included in the generated documentation, and some other cool little features like saving down the output of test requests to their own html files and then embedding them with iframes instead of using screenshots. I am a programmer so I like the code-centric approach more, but they are pretty similar. The big thing is that you need to control the file format/parser yourself.
  - snake_case 4y ago
    Before mask, I used a custom bash command runner which relied on a directory structure to implement the command and subcommand tree. This was pretty simple and nice to use. So if it’s working for you, there’s no reason to look for alternatives. Mask takes advantage of the markdown structure in a few ways. Headings define top-level commands and subheadings represent nested subcommands, which makes it extremely easy to structure a command tree. Also, mask checks the code block lang code (ruby, python, js, fish, etc…) and executes the script using that runtime as long as you have it installed. There’s other features, but those two are great examples why markdown works well as a command definition format.
  - airtonix 4y ago
    [dead]
  - e12e 4y ago
    > Most code editor will syntax highlight it a lot better. Really? I thought most now understood markdown code blocks with language tag?
- MuffinFlavored 4y ago
  Let me ask a silly question. Use case, I'm on Mac OS X (aka not Linux, not Windows). I have an Apple M1 processor (aka not x86_64). I have 8GB of RAM (silly mistake on my end, I know). Therefore, Docker + virtual machine based solutions are too expensive memory wise. However, I also like to think in terms of "let me separate this functionality into say... a Kubernetes workload like a pod" There's no good containerization solution for Mac OS. Can something like `mask` be used to achieve basically a "poor man's k8s" for a long running service? Basically, run these 5 or 6 services in parallel, let me be able to see their logs individually. If not, I completely understand. I just can't tell if there is an actual need for a solution like this, if it already exists and I just can't find it, etc.
  - spmurrayzzz 4y ago
    Have you tried using a process manager like supervisord[1] or pm2[2]? The former has a pretty clean declarative interface that you could view as a replacement for something like a docker-compose file or a k8s pod config. You don't get any containerization/isolation benefits obviously, but it sounds like you've already accepted that. [1] http://supervisord.org/ http://supervisord.org/ [2] https://github.com/Unitech/pm2 https://github.com/Unitech/pm2
    - snake_case 4y ago
      And another possibly lighter alternative is something like concurrently: https://github.com/open-cli-tools/concurrently https://github.com/open-cli-tools/concurrently
    - qbasic_forever 4y ago
      Foreman is a nice, easy to use local process manager. It's what heroku and such use with a simple Procfile for defining multiple processes, but it would work fine for running multiple developer tools, testing services/dependencies, etc. There are multiple implementations of it, heres the canonical one: https://github.com/ddollar/foreman https://github.com/ddollar/foreman And heres a go version that's easier to install and use: https://github.com/ddollar/forego https://github.com/ddollar/forego
alwaysbeconsing 4y ago
Your doc links are slightly broken. On the page https://xcfile.dev/task-syntax/ https://xcfile.dev/task-syntax/ the links don't resolve, although the ones in the sidebar do. The former are missing the trailing slash that the sidebar links contain: https://xcfile.dev/task-syntax/task-name https://xcfile.dev/task-syntax/task-name vs. https://xcfile.dev/task-syntax/task-name/ https://xcfile.dev/task-syntax/task-name/
- joerdav 4y ago
  Thanks for pointing that out, I recently did a rewrite of the docs. Seems to be one that works locally but not on remote. Thanks!
CharlieDigital 4y ago
This actually reminds me a lot of Gauge from Thoughtworks: https://github.com/getgauge/gauge https://github.com/getgauge/gauge It's typically paired with Taiko for test automation, but generally speaking it's a markdown to logical instruction engine. I dig it, but also worth taking a look at what the Thoughtworks team has done especially around the VS Code tooling and language server work that they did to bring intellisense into their Markdown templates.
- qbasic_forever 4y ago
  Wow I had not heard of gauge until now but it looks really cool! Seems like a nice more general behavior driven development tool. Very cool!
sigmonsays 4y ago
It's interesting to see emacs features reinvented this is a trival thing to achieve in emacs or org mode or babel in a number of ways. Why is copy and paste hard to do though?
- OJFord 4y ago
  This comment appears in any submission, doesn't it? Doesn't matter what it is, it's probably a trivial emacs feature. :') (It probably genuinely is in the infamous Dropbox thread somewhere - 'not only that but it's also built in to emacs'!)
  - mLuby 4y ago
    Which suggests the wrapper is the blocker. (I have no horse in the emacs vim holy war; this argument could apply to macOS just as easily.)
    - beepbooptheory 4y ago
      Sometimes maybe, I can see that here. But a lot of the times its people pushing their flashy developer product, where the reason to, lets say, overlook Emacs is even more obvious.
  - pxc 4y ago
    > Doesn't matter what it is, it's probably a trivial emacs feature. :') Org mode is far from trivial! when a commenter posts about org mode on an article like this, consider it an invitation to explore something deep and wonderful rather than a claim that the problem being solved is trivial!
    - OJFord 4y ago
      To be clear, I was just paraphrasing OP - I'm not an emacs user (nothing against it, just never got into it.. and yes I do use vim, on Arch btw).
- joerdav 4y ago
kodablah 4y ago
Like "literate programming" + "shell scripts". Ideally the latter can be a better, cross-platform language (e.g. "```python" code fences).
- dschep 4y ago
  I had exactly the same idea regarding ```python fences. I filed an issue if you wanna give it a thumbs up: https://github.com/joerdav/xc/issues/42 https://github.com/joerdav/xc/issues/42
proxysna 4y ago
Why not Makefile ?
- jonathankoren 4y ago
  TIL Make is the Zoidberg of technologies. Also non trivial Makefiles quickly become an unintelligible mess of rules both explicit and implicit, and an endless stack of variables that never feel fully defined.
  - semi-extrinsic 4y ago
    I was going to make a funny comment about Make, but then I decided I can just link the Make documentation's page "The Two Flavors of Variables" [1] which links to definitions of the four different flavors of variables. [1] https://www.gnu.org/software/make/manual/html_node/Flavors.html https://www.gnu.org/software/make/manual/html_node/Flavors.h...
    - beepbooptheory 4y ago
      There are four kinds of assignment, the first applies to recursive variable flavors, the rest to simple ones. It's maybe something to learn, but why is this a bad thing? People make good jokes but anyone asserting that official GNU documentation is anything short of wonderful needs to learn to read again.
- habitue 4y ago
  The key insight here seems to be that you usually need to specify the tasks for a repo twice: once in your task runner (Makefile, package.json, bazel BUILD file) and then again in your readme so that people can learn what custom tasks exist and what they do. This tool tries out the idea that maybe if the readme was structured the right way, it could double as the task runner specification. That way, even if you don't have the tool installed (maybe because you're browsing the repo online instead of in your editor) you can still see what the tasks do alongside their instructions
  - 4y ago
praveen9920 4y ago
Maybe I’m outdated but why not use simple make files?
- ricardobeat 4y ago
  PHONY: comment comment: TEXT=“yes, I also wonder why people don’t use simple makefiles” $(MAKE) publish publish: echo “$(TEXT)” Some “simple” things like passing arguments to a task become incredibly complex. I’ve been playing around with https://github.com/casey/just https://github.com/casey/just and so far it’s been very pleasant.
- spicybright 4y ago
  Mostly preference from what I can see. Some people don't like parts of how make works. Personally I have no problems with makefiles but I see the appeal.
- davidpfarrell 4y ago
  If you like to use make as a task runner, may I recommend you take a look at my entry into the ecosystem: * https://github.com/TekWizely/run https://github.com/TekWizely/run Feels like make (by design) but purpose-built for managing small tasks.
- joerdav 4y ago
  The reason I prefer xc to makefiles is it is more discoverable I think, when you have a project with a Makefile you often have some documentation in the README that outlines what each Makefile command does. I thought why not cut out the middle man!
  - zvr 4y ago
    You can definitely add the documentation to your Makefile and have a target ("help" is commonly used) that displays it! Examples of such an approach abound.
davidpfarrell 4y ago
To the extent that posts like these evolve into discussing the merits of Make as a task runner, I would like to offer my tool for review: * https://github.com/TekWizely/run https://github.com/TekWizely/run I built it to feel like make, but be better at managing tasks and wrappers. If you are evaluating task runners and appreciate the simplicity of Make's syntax, I hope you'll give Run a try.
tanepiper 4y ago
A few years back on my project Takeoff (https://github.com/takeoff-env/takeoff https://github.com/takeoff-env/takeoff) - a tool for creating docker environments for local dev - I used maid (https://github.com/egoist/maid https://github.com/egoist/maid) for exactly this, using a markdown file as a task runner. I liked the idea, although didn't go much further with the project.
pachico 4y ago
Great work! I really like it! BTW, it would be great if it had .env support!
schemescape 4y ago
My initial reaction was: why Markdown? That's (despite its name) a markup language, not some sort of data/configuration format. But then I saw that it allows you to consolidate the documentation and the tasks into a single place, and now I think this is a great idea! Other comments have pointed it out, but the only issue I see is that shell commands aren't trivially usable on e.g. Windows. I'm sure there are good solutions out there for this, but I haven't found them yet...
- joerdav 4y ago
  Most tasks should work trivially on windows actually, xc is shipped with a shell interpreted. The only constraint at the moment is if an executable is used that isn't available on the system.
  - schemescape 4y ago
    I didn’t realize that, thanks! I’ll definitely give this a try…
cdaringe 4y ago
I authored a markdown player, for which one may run arbitrary code/commands, capture output, and self embed. Literate programming, md targeted. https://deno.land/x/markdown_player@v1.2.3 https://deno.land/x/markdown_player@v1.2.3 OCaml users also like omd. Iirc omd is a bit ocaml specific tho
airtonix 4y ago
[dead]

113 points by joerdav 4y ago

Nice to see other markdown-based task runners! I'm the creator of mask which is another alternative, written in Rust. Our approaches slightly differ. It looks like xc parses the README.md file for commands while mask looks for a maskfile.md by default, though you can provide a --maskfile arg to specify any markdown file that follows the expected format. https://github.com/jacobdeichert/mask https://github.com/jacobdeichert/mask

teknopaul 4y ago
Why not a simple well commented shell script? Most code editor will syntax highlight it a lot better.
- boxed 4y ago
  In the docs for iommi we've tried both the documentation-centric and the code-centric approaches. We started with the documentation-centric approach because we had a lot of documentation and wanted to check if the code examples in them worked. So I wrote a bad parser that extracted the code from rST and put them into python files that we then ran. This worked... ok-ish. Then we switched to the code-centric approach that we use today where we write tests that we generate the documentation from. There are specially marked documentation strings plus some tags for parts of the tests that shouldn't be included in the generated documentation, and some other cool little features like saving down the output of test requests to their own html files and then embedding them with iframes instead of using screenshots. I am a programmer so I like the code-centric approach more, but they are pretty similar. The big thing is that you need to control the file format/parser yourself.
- snake_case 4y ago
  Before mask, I used a custom bash command runner which relied on a directory structure to implement the command and subcommand tree. This was pretty simple and nice to use. So if it’s working for you, there’s no reason to look for alternatives. Mask takes advantage of the markdown structure in a few ways. Headings define top-level commands and subheadings represent nested subcommands, which makes it extremely easy to structure a command tree. Also, mask checks the code block lang code (ruby, python, js, fish, etc…) and executes the script using that runtime as long as you have it installed. There’s other features, but those two are great examples why markdown works well as a command definition format.
- airtonix 4y ago
  [dead]
- e12e 4y ago
  > Most code editor will syntax highlight it a lot better. Really? I thought most now understood markdown code blocks with language tag?
MuffinFlavored 4y ago
Let me ask a silly question. Use case, I'm on Mac OS X (aka not Linux, not Windows). I have an Apple M1 processor (aka not x86_64). I have 8GB of RAM (silly mistake on my end, I know). Therefore, Docker + virtual machine based solutions are too expensive memory wise. However, I also like to think in terms of "let me separate this functionality into say... a Kubernetes workload like a pod" There's no good containerization solution for Mac OS. Can something like `mask` be used to achieve basically a "poor man's k8s" for a long running service? Basically, run these 5 or 6 services in parallel, let me be able to see their logs individually. If not, I completely understand. I just can't tell if there is an actual need for a solution like this, if it already exists and I just can't find it, etc.
- spmurrayzzz 4y ago
  Have you tried using a process manager like supervisord[1] or pm2[2]? The former has a pretty clean declarative interface that you could view as a replacement for something like a docker-compose file or a k8s pod config. You don't get any containerization/isolation benefits obviously, but it sounds like you've already accepted that. [1] http://supervisord.org/ http://supervisord.org/ [2] https://github.com/Unitech/pm2 https://github.com/Unitech/pm2
  - snake_case 4y ago
    And another possibly lighter alternative is something like concurrently: https://github.com/open-cli-tools/concurrently https://github.com/open-cli-tools/concurrently
  - qbasic_forever 4y ago
    Foreman is a nice, easy to use local process manager. It's what heroku and such use with a simple Procfile for defining multiple processes, but it would work fine for running multiple developer tools, testing services/dependencies, etc. There are multiple implementations of it, heres the canonical one: https://github.com/ddollar/foreman https://github.com/ddollar/foreman And heres a go version that's easier to install and use: https://github.com/ddollar/forego https://github.com/ddollar/forego

alwaysbeconsing 4y ago

Your doc links are slightly broken. On the page https://xcfile.dev/task-syntax/ https://xcfile.dev/task-syntax/ the links don't resolve, although the ones in the sidebar do. The former are missing the trailing slash that the sidebar links contain: https://xcfile.dev/task-syntax/task-name https://xcfile.dev/task-syntax/task-name vs. https://xcfile.dev/task-syntax/task-name/ https://xcfile.dev/task-syntax/task-name/

joerdav 4y ago
Thanks for pointing that out, I recently did a rewrite of the docs. Seems to be one that works locally but not on remote. Thanks!

CharlieDigital 4y ago

This actually reminds me a lot of Gauge from Thoughtworks: https://github.com/getgauge/gauge https://github.com/getgauge/gauge It's typically paired with Taiko for test automation, but generally speaking it's a markdown to logical instruction engine. I dig it, but also worth taking a look at what the Thoughtworks team has done especially around the VS Code tooling and language server work that they did to bring intellisense into their Markdown templates.

qbasic_forever 4y ago
Wow I had not heard of gauge until now but it looks really cool! Seems like a nice more general behavior driven development tool. Very cool!

sigmonsays 4y ago

It's interesting to see emacs features reinvented this is a trival thing to achieve in emacs or org mode or babel in a number of ways. Why is copy and paste hard to do though?

OJFord 4y ago
This comment appears in any submission, doesn't it? Doesn't matter what it is, it's probably a trivial emacs feature. :') (It probably genuinely is in the infamous Dropbox thread somewhere - 'not only that but it's also built in to emacs'!)
- mLuby 4y ago
  Which suggests the wrapper is the blocker. (I have no horse in the emacs vim holy war; this argument could apply to macOS just as easily.)
  - beepbooptheory 4y ago
    Sometimes maybe, I can see that here. But a lot of the times its people pushing their flashy developer product, where the reason to, lets say, overlook Emacs is even more obvious.
- pxc 4y ago
  > Doesn't matter what it is, it's probably a trivial emacs feature. :') Org mode is far from trivial! when a commenter posts about org mode on an article like this, consider it an invitation to explore something deep and wonderful rather than a claim that the problem being solved is trivial!
  - OJFord 4y ago
    To be clear, I was just paraphrasing OP - I'm not an emacs user (nothing against it, just never got into it.. and yes I do use vim, on Arch btw).
joerdav 4y ago

kodablah 4y ago

Like "literate programming" + "shell scripts". Ideally the latter can be a better, cross-platform language (e.g. "```python" code fences).

dschep 4y ago
I had exactly the same idea regarding ```python fences. I filed an issue if you wanna give it a thumbs up: https://github.com/joerdav/xc/issues/42 https://github.com/joerdav/xc/issues/42

proxysna 4y ago

Why not Makefile ?

jonathankoren 4y ago
TIL Make is the Zoidberg of technologies. Also non trivial Makefiles quickly become an unintelligible mess of rules both explicit and implicit, and an endless stack of variables that never feel fully defined.
- semi-extrinsic 4y ago
  I was going to make a funny comment about Make, but then I decided I can just link the Make documentation's page "The Two Flavors of Variables" [1] which links to definitions of the four different flavors of variables. [1] https://www.gnu.org/software/make/manual/html_node/Flavors.html https://www.gnu.org/software/make/manual/html_node/Flavors.h...
  - beepbooptheory 4y ago
    There are four kinds of assignment, the first applies to recursive variable flavors, the rest to simple ones. It's maybe something to learn, but why is this a bad thing? People make good jokes but anyone asserting that official GNU documentation is anything short of wonderful needs to learn to read again.
habitue 4y ago
The key insight here seems to be that you usually need to specify the tasks for a repo twice: once in your task runner (Makefile, package.json, bazel BUILD file) and then again in your readme so that people can learn what custom tasks exist and what they do. This tool tries out the idea that maybe if the readme was structured the right way, it could double as the task runner specification. That way, even if you don't have the tool installed (maybe because you're browsing the repo online instead of in your editor) you can still see what the tasks do alongside their instructions
- 4y ago

praveen9920 4y ago

Maybe I’m outdated but why not use simple make files?

ricardobeat 4y ago
PHONY: comment comment: TEXT=“yes, I also wonder why people don’t use simple makefiles” $(MAKE) publish publish: echo “$(TEXT)” Some “simple” things like passing arguments to a task become incredibly complex. I’ve been playing around with https://github.com/casey/just https://github.com/casey/just and so far it’s been very pleasant.
spicybright 4y ago
Mostly preference from what I can see. Some people don't like parts of how make works. Personally I have no problems with makefiles but I see the appeal.
davidpfarrell 4y ago
If you like to use make as a task runner, may I recommend you take a look at my entry into the ecosystem: * https://github.com/TekWizely/run https://github.com/TekWizely/run Feels like make (by design) but purpose-built for managing small tasks.
joerdav 4y ago
The reason I prefer xc to makefiles is it is more discoverable I think, when you have a project with a Makefile you often have some documentation in the README that outlines what each Makefile command does. I thought why not cut out the middle man!
- zvr 4y ago
  You can definitely add the documentation to your Makefile and have a target ("help" is commonly used) that displays it! Examples of such an approach abound.

davidpfarrell 4y ago

To the extent that posts like these evolve into discussing the merits of Make as a task runner, I would like to offer my tool for review: * https://github.com/TekWizely/run https://github.com/TekWizely/run I built it to feel like make, but be better at managing tasks and wrappers. If you are evaluating task runners and appreciate the simplicity of Make's syntax, I hope you'll give Run a try.

tanepiper 4y ago

A few years back on my project Takeoff (https://github.com/takeoff-env/takeoff https://github.com/takeoff-env/takeoff) - a tool for creating docker environments for local dev - I used maid (https://github.com/egoist/maid https://github.com/egoist/maid) for exactly this, using a markdown file as a task runner. I liked the idea, although didn't go much further with the project.

pachico 4y ago

Great work! I really like it! BTW, it would be great if it had .env support!

schemescape 4y ago

My initial reaction was: why Markdown? That's (despite its name) a markup language, not some sort of data/configuration format. But then I saw that it allows you to consolidate the documentation and the tasks into a single place, and now I think this is a great idea! Other comments have pointed it out, but the only issue I see is that shell commands aren't trivially usable on e.g. Windows. I'm sure there are good solutions out there for this, but I haven't found them yet...

joerdav 4y ago
Most tasks should work trivially on windows actually, xc is shipped with a shell interpreted. The only constraint at the moment is if an executable is used that isn't available on the system.
- schemescape 4y ago
  I didn’t realize that, thanks! I’ll definitely give this a try…

cdaringe 4y ago

I authored a markdown player, for which one may run arbitrary code/commands, capture output, and self embed. Literate programming, md targeted. https://deno.land/x/markdown_player@v1.2.3 https://deno.land/x/markdown_player@v1.2.3 OCaml users also like omd. Iirc omd is a bit ocaml specific tho

airtonix 4y ago

[dead]