6 ms·

Show HN: Maid – Markdown driven task runner

83 points by 0x142857 8y ago

actionowl 8y ago
Neat. It's like documentation that is also code instead of the other way around.
- sleepychu 8y ago
  Yeah, a pretty nice alternative to a wiki page full of commands you have to run. Would be nice if there was some kind of <YOUR_HOST_HERE> equivalent. I have to write that in wiki pages a lot.
- andreareina 8y ago
  The term is "literate programming", if you'd like to learn more. It's how I keep most of my notes.
edejong 8y ago
Would be nice if tooling took the principle of least astonishment[1] into account. A markdown which is also a build file with specific rules on headings and first block quotes? High astonishment. Literate programming without text semantics? Medium astonishment. Declarative dependency tracking build tool, such as CMake? Low astonishment. [1] https://en.wikipedia.org/wiki/Principle_of_least_astonishment https://en.wikipedia.org/wiki/Principle_of_least_astonishmen...
- 0x142857 8y ago
  > A markdown which is also a build file with specific rules on headings and first block quotes? High astonishment. This is wrong in the first place.
christophilus 8y ago
This is pretty cool! I've thought about putting something like this together for building (web) front-end sources from markdown. It's probably a terrible idea, but I thought it'd be helpful for writing e.g. a programming book that was itself executable.
hndyh 8y ago
The name seems demeaning to women.
- djsumdog 8y ago
  I honestly can't tell if you're serious. (@see Poe's Law)
  - qwer 8y ago
    I wouldn't have noticed if it hadn't been pointed out but it's definitely a term for a pretty outdated female-only position so I think the point isn't so extreme. I can almost never detect this kind of stuff because it doesn't affect me directly and I'm sure the dev meant no ill-will, but once pointed out, I can see how projects named like this would get under people's skin. "made, the MArkDown Executor"? :D It's pretty easy to be more inclusive.
    - wattengard 8y ago
      The suffix "ake" is "standard" for these task runners... Cake (C#), Fake (F#) etc... So of course it should be called Make! ... ... wait ... ... shit ...
- wingerlang 8y ago
  Would you consider "Alfred" (the osx app) demeaning to men? Or Bartender (the osx app) demeaning to both genders? Both of their purpose is to essentially help do stuff for you.
- andrew_ 8y ago
  Jeez. There's always someone that's going to seem to be offended. For clarity, the author should be just as ridiculous and claim to use the early-English definition of the phrase; a virgin. Or claim that it has something to do with Minute Maid® juices. For reference, "maid" as someone who cleans up a place is not seen as derogatory nor demeaning to everyone, as there are swaths of trademarks [1] registered using that term, and innumerable mobile maid services available in the U.S. which use that term in their company name. Or maybe that comment was just a troll job. [1] http://tmsearch.uspto.gov/bin/showfield?f=toc&state=4807%3Aw6uj5h.1.1&p_search=searchss&p_L=50&BackReference=&p_plural=yes&p_s_PARA1=&p_tagrepl~%3A=PARA1%24LD&expr=PARA1+AND+PARA2&p_s_PARA2=maid&p_tagrepl~%3A=PARA2%24COMB&p_op_ALL=AND&a_default=search&a_search=Submit+Query&a_search=Submit+Query http://tmsearch.uspto.gov/bin/showfield?f=toc&state=4807%3Aw...
reacharavindh 8y ago
Really like this idea. I'm going to build something like this for the scripts I write for administration and operations. Script and documentation in the same file. Pass it through another script and it spits out HTML documentation. I suppose Sphinx(Python) or equivalent in other languages already do this, but the ease of use of Markdown is lucrative. In my case Python - the language already recognizes lines starting with # as comments. Goes on to check out the rest of the stuff.
- vanderZwan 8y ago
  What's the benefit over having a doc comment style that can be transformed into documentation?
antjanus 8y ago
Is there something like this but for regular code? I think it's called verbose programming or literal programming? I can't remember but if anyone knows of similar projects, please let me know!
- nailuj 8y ago
  You mean notebook-style programming environments like Jupyter? They have kernels for a lot of languages. I think Emacs' org-mode can execute code embedded within the document as well. The concept you're thinking of is called Literate Programming.
- tjallingt 8y ago
  Its called literate programming; for example https://wiki.haskell.org/Literate_programming https://wiki.haskell.org/Literate_programming
- Jtsummers 8y ago
  Literate programming. You'd write a collection of blocks and "tangle" them into a program (weave usually means generating the documentation). A common workflow for me when prototyping something, and then I let it grow into a fullblown program, with org goes something like this: * Main This program will print out "Hello, world!". #+NAME: main #+BEGIN_SRC c :tangle hello.c :noweb yes <<includes>> int main(void) { <<main-body>> } #+END_SRC * Includes - stdio :: this is included to get =printf= - others :: if I had other includes I'd explain why here #+NAME: includes #+BEGIN_SRC c #include <stdio.h> #+END_SRC * Main body Maybe an explanation of the algorithm #+NAME: main-body #+BEGIN_SRC c printf("Hello, world!\n"); #+END_SRC Literate programming doesn't care about the order, I can put that main at the bottom or in the middle. This allows me to introduce program concepts in a logical fashion that is different from what the compiler wants or needs. This allows me to, for instance, group all data structures and even share code across them where this would be less feasible in straight C. References like the `<<main-body>>` can be shared by multiple code blocks. Obviously, as programmers, we want to minimize repetition. But sometimes a language forces us to repeat ourselves. If it gives us a way to include common source blocks, that's awesome (like you can make a header file called common.h that every one of your source files includes). But that's feasible in every language. Nor are the macro systems ideal in every case where it is technically feasible. As I write the code, usually in one pane of tmux, I'll tangle it. Then in a separate pane (usually on the right) I'll execute the code. Or I may have something that monitors the files for changes and builds it automatically and runs tests. EDIT: This is also an effective technique for developing an understanding of a new code base. Bring all the code into one big source block in a new org file (or whatever you use). Tangle it and make sure it's the same as the original (this process gets repeated a lot, make it a script). Grab segments of the code and extract them into new blocks (with a properly named reference). I usually do this with large chunks (whole class declarations, struct declarations, or function definitions). Document what the code does and its relation to other segments. Repeatedly extract segments until they're down to reasonable chunks (a 500 line function? It's probably worth going down to 10-50 smaller blocks covering each logical section).
djsumdog 8y ago
So we're going to take something that's not really a standard to begin with (Markdown is about as dependent on the renderer as HTML) and add some code blocks? Hmm... I can see the advantage of having a runnable readme, and documentation with the code. I guess it's sorta like a Zeppelin notebook. Huh .. at first I though this project was kinda ridiculous, but I do like the concept for including a maid file in a project and people can get started right away, with sections available for auto installing the packages and running the program on a local user account.
- albedoa 8y ago
  > So we're going to take something that's not really a standard to begin with (Markdown is about as dependent on the renderer as HTML) and add some code blocks? Hmm... It uses a specific renderer. The renderer adheres to a spec that defines code blocks.
_Marak_ 8y ago
This seems like a cool idea, but I'm hesitant to run it locally. Is an early version of the application with no unit tests and it eagerly calls `sh` command with string input ( see: https://github.com/egoist/maid/blob/master/lib/index.js https://github.com/egoist/maid/blob/master/lib/index.js ) I would be worried that some parse error or non-obvious typo in the Markdown format could result in `sh` accidentally running something destructive locally. Still though, I like the idea and am now following the project, you should keep working on it! You may want to consider running the Node tasks in a separate subprocess instead of using `require-from-string` module in parent process. You could also try using https://github.com/stackvana/microcule https://github.com/stackvana/microcule to add support for subprocess execution and multiple programming languages.
- 0x142857 8y ago
  yeah unit tests are coming.. > I would be worried that some parse error or non-obvious typo in the Markdown format well I couldn't help this too much since it uses markdown-it under the hood to parse the markdown, i think it's safe as long as you don't write destructive shell script yourself.. > You may want to consider running the Node tasks in a separate subprocess instead of using `require-from-string` module in parent process. will give this a try. also PR welcome xD
  - _Marak_ 8y ago
    That sounds good! I'll keep an eye on the Github Issues and may make a PR.
- andrew_ 8y ago
  `require-from-string` also presents all kinds of problems with regard to the require cache. Notably, cosmiconfig recently dropped use of that module.
acobster 8y ago
To me this sounds like Org Mode in Emacs, except that it's editor-agnostic while still being runnable. I haven't used it, but I'm intrigued. Can someone who's familiar with Org Mode tell me if this is a fair comparison? I think Org Mode has a wider scope, but some of the same benefits.
- andreareina 8y ago
  org mode is much more evolved than this. That does mean that it's less lightweight and requires the use of emacs, but I honestly think that org mode (and magit) by themselves are reasons to use emacs, even if just for those packages. Features of org mode that I use: * Full editor support for the language you're writing -- syntax highlighting, structural editing, etc * On-line evaluation of code blocks, with the results appearing after said code block * Threading of data through multiple code blocks -- I can perform an SQL query and transparently pass the result to gnuplot for visualization -- emacs takes care of translating the intermediate data between formats * Full noweb-style tangling of source and many more.
  - lilactown 8y ago
    I do wish there was a more standalone way to interact with org-mode files. Being the lone Emacs user on a team feels like I lose a lot of the power of org files. I would love to write our tasks / build config out in org-mode, but running that on someone elses machine w/o Emacs, CI, etc. seems not highly supported.
    - andreareina 8y ago
      I think the easiest answer to that is to ship a stripped-down emacs that starts in org mode, probably with keys remapped to the "standard" set (`C-a` for select all, etc).
blackbear_ 8y ago
org-mode + org-babel reinvented?
spotlmnop 8y ago
Is this handmaid?
skratlo 8y ago
The world definitely needs this, it has 230 stars in just 10 hours!!! What a useful tool, I think a maidfile would fit very well into the short list of *files of https://github.com/tootsuite/mastodon https://github.com/tootsuite/mastodon
newlaptopnewacc 8y ago
Many positive comments - am I the only one who doesn't like these declarative yaml, markdown, dsl things? No offence to the author(s), it's probably well implemented, I just can't stand these things primarily because of the lack of types, I never know what I'm looking at. I would much rather read comments and code.
- y4mi 8y ago
  I doubt this is meant as a production tool. Its probably just a proof of concept to explore an interesting idea. I cant agree on your point though. While there is a place for custom build tools, its incredibly rare. And if you're in a situation where you actually need to build a custom build tool for your project, you should consider fixing the reason why you'd need that. Builds shouldn't need such a complicated process..
staltz 8y ago
I wish the whole package.json could be replaced with something like this.
zabil 8y ago
We built something similar(markdown + runner) for testing at http://github.com/getgauge/gauge http://github.com/getgauge/gauge

83 points by 0x142857 8y ago

actionowl 8y ago

Neat. It's like documentation that is also code instead of the other way around.

sleepychu 8y ago
Yeah, a pretty nice alternative to a wiki page full of commands you have to run. Would be nice if there was some kind of <YOUR_HOST_HERE> equivalent. I have to write that in wiki pages a lot.
andreareina 8y ago
The term is "literate programming", if you'd like to learn more. It's how I keep most of my notes.

edejong 8y ago

Would be nice if tooling took the principle of least astonishment[1] into account. A markdown which is also a build file with specific rules on headings and first block quotes? High astonishment. Literate programming without text semantics? Medium astonishment. Declarative dependency tracking build tool, such as CMake? Low astonishment. [1] https://en.wikipedia.org/wiki/Principle_of_least_astonishment https://en.wikipedia.org/wiki/Principle_of_least_astonishmen...

0x142857 8y ago
> A markdown which is also a build file with specific rules on headings and first block quotes? High astonishment. This is wrong in the first place.

christophilus 8y ago

This is pretty cool! I've thought about putting something like this together for building (web) front-end sources from markdown. It's probably a terrible idea, but I thought it'd be helpful for writing e.g. a programming book that was itself executable.

hndyh 8y ago

The name seems demeaning to women.

djsumdog 8y ago
I honestly can't tell if you're serious. (@see Poe's Law)
- qwer 8y ago
  I wouldn't have noticed if it hadn't been pointed out but it's definitely a term for a pretty outdated female-only position so I think the point isn't so extreme. I can almost never detect this kind of stuff because it doesn't affect me directly and I'm sure the dev meant no ill-will, but once pointed out, I can see how projects named like this would get under people's skin. "made, the MArkDown Executor"? :D It's pretty easy to be more inclusive.
  - wattengard 8y ago
    The suffix "ake" is "standard" for these task runners... Cake (C#), Fake (F#) etc... So of course it should be called Make! ... ... wait ... ... shit ...
wingerlang 8y ago
Would you consider "Alfred" (the osx app) demeaning to men? Or Bartender (the osx app) demeaning to both genders? Both of their purpose is to essentially help do stuff for you.
andrew_ 8y ago
Jeez. There's always someone that's going to seem to be offended. For clarity, the author should be just as ridiculous and claim to use the early-English definition of the phrase; a virgin. Or claim that it has something to do with Minute Maid® juices. For reference, "maid" as someone who cleans up a place is not seen as derogatory nor demeaning to everyone, as there are swaths of trademarks [1] registered using that term, and innumerable mobile maid services available in the U.S. which use that term in their company name. Or maybe that comment was just a troll job. [1] http://tmsearch.uspto.gov/bin/showfield?f=toc&state=4807%3Aw6uj5h.1.1&p_search=searchss&p_L=50&BackReference=&p_plural=yes&p_s_PARA1=&p_tagrepl~%3A=PARA1%24LD&expr=PARA1+AND+PARA2&p_s_PARA2=maid&p_tagrepl~%3A=PARA2%24COMB&p_op_ALL=AND&a_default=search&a_search=Submit+Query&a_search=Submit+Query http://tmsearch.uspto.gov/bin/showfield?f=toc&state=4807%3Aw...

reacharavindh 8y ago

Really like this idea. I'm going to build something like this for the scripts I write for administration and operations. Script and documentation in the same file. Pass it through another script and it spits out HTML documentation. I suppose Sphinx(Python) or equivalent in other languages already do this, but the ease of use of Markdown is lucrative. In my case Python - the language already recognizes lines starting with # as comments. Goes on to check out the rest of the stuff.

vanderZwan 8y ago
What's the benefit over having a doc comment style that can be transformed into documentation?

antjanus 8y ago

Is there something like this but for regular code? I think it's called verbose programming or literal programming? I can't remember but if anyone knows of similar projects, please let me know!

nailuj 8y ago
You mean notebook-style programming environments like Jupyter? They have kernels for a lot of languages. I think Emacs' org-mode can execute code embedded within the document as well. The concept you're thinking of is called Literate Programming.
tjallingt 8y ago
Its called literate programming; for example https://wiki.haskell.org/Literate_programming https://wiki.haskell.org/Literate_programming
Jtsummers 8y ago
Literate programming. You'd write a collection of blocks and "tangle" them into a program (weave usually means generating the documentation). A common workflow for me when prototyping something, and then I let it grow into a fullblown program, with org goes something like this: * Main This program will print out "Hello, world!". #+NAME: main #+BEGIN_SRC c :tangle hello.c :noweb yes <<includes>> int main(void) { <<main-body>> } #+END_SRC * Includes - stdio :: this is included to get =printf= - others :: if I had other includes I'd explain why here #+NAME: includes #+BEGIN_SRC c #include <stdio.h> #+END_SRC * Main body Maybe an explanation of the algorithm #+NAME: main-body #+BEGIN_SRC c printf("Hello, world!\n"); #+END_SRC Literate programming doesn't care about the order, I can put that main at the bottom or in the middle. This allows me to introduce program concepts in a logical fashion that is different from what the compiler wants or needs. This allows me to, for instance, group all data structures and even share code across them where this would be less feasible in straight C. References like the `<<main-body>>` can be shared by multiple code blocks. Obviously, as programmers, we want to minimize repetition. But sometimes a language forces us to repeat ourselves. If it gives us a way to include common source blocks, that's awesome (like you can make a header file called common.h that every one of your source files includes). But that's feasible in every language. Nor are the macro systems ideal in every case where it is technically feasible. As I write the code, usually in one pane of tmux, I'll tangle it. Then in a separate pane (usually on the right) I'll execute the code. Or I may have something that monitors the files for changes and builds it automatically and runs tests. EDIT: This is also an effective technique for developing an understanding of a new code base. Bring all the code into one big source block in a new org file (or whatever you use). Tangle it and make sure it's the same as the original (this process gets repeated a lot, make it a script). Grab segments of the code and extract them into new blocks (with a properly named reference). I usually do this with large chunks (whole class declarations, struct declarations, or function definitions). Document what the code does and its relation to other segments. Repeatedly extract segments until they're down to reasonable chunks (a 500 line function? It's probably worth going down to 10-50 smaller blocks covering each logical section).

djsumdog 8y ago

So we're going to take something that's not really a standard to begin with (Markdown is about as dependent on the renderer as HTML) and add some code blocks? Hmm... I can see the advantage of having a runnable readme, and documentation with the code. I guess it's sorta like a Zeppelin notebook. Huh .. at first I though this project was kinda ridiculous, but I do like the concept for including a maid file in a project and people can get started right away, with sections available for auto installing the packages and running the program on a local user account.

albedoa 8y ago
> So we're going to take something that's not really a standard to begin with (Markdown is about as dependent on the renderer as HTML) and add some code blocks? Hmm... It uses a specific renderer. The renderer adheres to a spec that defines code blocks.

_Marak_ 8y ago

This seems like a cool idea, but I'm hesitant to run it locally. Is an early version of the application with no unit tests and it eagerly calls `sh` command with string input ( see: https://github.com/egoist/maid/blob/master/lib/index.js https://github.com/egoist/maid/blob/master/lib/index.js ) I would be worried that some parse error or non-obvious typo in the Markdown format could result in `sh` accidentally running something destructive locally. Still though, I like the idea and am now following the project, you should keep working on it! You may want to consider running the Node tasks in a separate subprocess instead of using `require-from-string` module in parent process. You could also try using https://github.com/stackvana/microcule https://github.com/stackvana/microcule to add support for subprocess execution and multiple programming languages.

0x142857 8y ago
yeah unit tests are coming.. > I would be worried that some parse error or non-obvious typo in the Markdown format well I couldn't help this too much since it uses markdown-it under the hood to parse the markdown, i think it's safe as long as you don't write destructive shell script yourself.. > You may want to consider running the Node tasks in a separate subprocess instead of using `require-from-string` module in parent process. will give this a try. also PR welcome xD
- _Marak_ 8y ago
  That sounds good! I'll keep an eye on the Github Issues and may make a PR.
andrew_ 8y ago
`require-from-string` also presents all kinds of problems with regard to the require cache. Notably, cosmiconfig recently dropped use of that module.

acobster 8y ago

To me this sounds like Org Mode in Emacs, except that it's editor-agnostic while still being runnable. I haven't used it, but I'm intrigued. Can someone who's familiar with Org Mode tell me if this is a fair comparison? I think Org Mode has a wider scope, but some of the same benefits.

andreareina 8y ago
org mode is much more evolved than this. That does mean that it's less lightweight and requires the use of emacs, but I honestly think that org mode (and magit) by themselves are reasons to use emacs, even if just for those packages. Features of org mode that I use: * Full editor support for the language you're writing -- syntax highlighting, structural editing, etc * On-line evaluation of code blocks, with the results appearing after said code block * Threading of data through multiple code blocks -- I can perform an SQL query and transparently pass the result to gnuplot for visualization -- emacs takes care of translating the intermediate data between formats * Full noweb-style tangling of source and many more.
- lilactown 8y ago
  I do wish there was a more standalone way to interact with org-mode files. Being the lone Emacs user on a team feels like I lose a lot of the power of org files. I would love to write our tasks / build config out in org-mode, but running that on someone elses machine w/o Emacs, CI, etc. seems not highly supported.
  - andreareina 8y ago
    I think the easiest answer to that is to ship a stripped-down emacs that starts in org mode, probably with keys remapped to the "standard" set (`C-a` for select all, etc).

blackbear_ 8y ago

org-mode + org-babel reinvented?

spotlmnop 8y ago

Is this handmaid?

skratlo 8y ago

The world definitely needs this, it has 230 stars in just 10 hours!!! What a useful tool, I think a maidfile would fit very well into the short list of *files of https://github.com/tootsuite/mastodon https://github.com/tootsuite/mastodon

newlaptopnewacc 8y ago

Many positive comments - am I the only one who doesn't like these declarative yaml, markdown, dsl things? No offence to the author(s), it's probably well implemented, I just can't stand these things primarily because of the lack of types, I never know what I'm looking at. I would much rather read comments and code.

y4mi 8y ago
I doubt this is meant as a production tool. Its probably just a proof of concept to explore an interesting idea. I cant agree on your point though. While there is a place for custom build tools, its incredibly rare. And if you're in a situation where you actually need to build a custom build tool for your project, you should consider fixing the reason why you'd need that. Builds shouldn't need such a complicated process..

staltz 8y ago

I wish the whole package.json could be replaced with something like this.

zabil 8y ago

We built something similar(markdown + runner) for testing at http://github.com/getgauge/gauge http://github.com/getgauge/gauge