8 ms·

Sorry for the plug, but I can't help myself everytime I read about task managers to notice you have to learn yet another DSL. DSL are the plague of our field,

Sorry for the plug, but I can't help myself everytime I read about task managers to notice you have to learn yet another DSL. DSL are the plague of our field, for one that is useful like SQL, you have a hundred that turn your life into hell. Make's DSL is already full of gotchas, but templated YAML based DSL are just insanity. They have so many footguns, and so little flexibility and ability to be debugged. After trying a lot of task runners and builders, I finally settled on doit: https://www.bitecode.dev/p/doit-the-goodest-python-task-runner https://www.bitecode.dev/p/doit-the-goodest-python-task-runn... - It makes the simple things simple, and the hard things possible. - It scales up (you get targets, dependencies, manual scripting, etc), but it also scales down (you can define a simple list of shell commands to run and that's it). - Instead of a DSL, you have regular Python. Which for all it's faults, is an excellent scripting language. It has great tooling support, and comes with a debugger. - To avoid people abusing the expressiveness of Python, the default syntax to create tasks encourages a declarative style. I've been using doit for years, and every single project I introduced it to was improved by it. Of course, it comes with it's own problems (non python dev can dislike having to install that, and it doesn't yet provide a stand alone executable), but the ROI is way better than the alternatives I tried.

emptysongglass 3y ago
For Pythonistas or people interested in Python, I can't plug the commenter's newsletter enough: it is an absolute treasure trove of good advice for new and old users of Python. They did a whole series on the right way to install and use Python [1]. If you follow their advice you'll never struggle with any of the problems people complain of encountering with Python. [1] https://www.bitecode.dev/p/installing-python-the-bare-minimum https://www.bitecode.dev/p/installing-python-the-bare-minimu...
- BiteCode_dev 3y ago
  Cheers :)
rewmie 3y ago
> DSL are the plague of our field, for one that is useful like SQL, you have a hundred that turn your life into hell. The same can be said about using general purpose programming languages to specify stuff declaratively, but instead of implementation the problems happen in each and every single use of it. It only takes a single user that either does not comply with standards or feels they are particularly clever to mess everything for everyone.
- BiteCode_dev 3y ago
  The dose makes the poison
sdflhasjd 3y ago
> templated YAML based DSL are just insanity The truest statement of all of HN It started off with configuration files, but then we needed logic, control flow, or variables; and for some reason nobody goes "ok, maybe we need an actual progamming language now" and instead we've created horrors beyond our comprehension. Azure Pipelines is the worst I've used: three types of variables, conditional statements return string "True" and "False", and of course it's mostly undebuggable.
- BiteCode_dev 3y ago
  I have hopes that one day something like CUElang (https://cuelang.org https://cuelang.org) will become popular to be the configuration language of everything. Just enough logic to get by, but not turing complete, plus strong typing and embedded schemas. However, for now there is just a single implementation in GO.
  - gavinhoward 3y ago
    Oh interesting. I came up with basically the same thing as CUE (as described in [1]) on my own, except for the heavy schema stuff. I think the schema stuff is unfortunately what's going to keep it from adoption. I mean, I didn't add schema stuff explicitly in, but you can define schemas in mine, which makes it easier. But making it explicit raises the bar for an implementation and may scare people away from writing one. [1]: https://cuelang.org/docs/tutorials/tour/intro/json/ https://cuelang.org/docs/tutorials/tour/intro/json/
- hitchstory 3y ago
  Templated YAML just means that something somebody thought was configuration either wasn't or wasnt under every circumstance. It's not really a problem with YAML. In most cases I think it's the fault of the creators of the YAML DSL (e.g. ansible) but with CI pipelines people really just should be writing scripts 100% in other languages and wiring them up to events with YAML. I dont have any sympathy for people using them as programming languages. I'm sure Microsoft is happy to indulge it though because hey, vendor lock-in.
veltas 3y ago
No need to apologise, I don't like YAML, I like Python, and I agree: I'd rather have a domain specific library than a DSL anyday. The best of both worlds is a good programming language that allows expressing domain specific libraries almost as if they were domain specific languages, without going so declarative that people don't know how to script anything anymore. I don't know if Python/doit really achieved that, but its task definitions seem almost as readable as the YAML, in a real language where I can do anything I want without asking permission. So I would always choose doit over Taskfile. But a better comparison is between doit and make. I much prefer make's basic syntax, although I dislike newer features which inevitably have stretched the syntax beyond its original simplicity, but conceivably you can achieve something similar to make in a language but as a library rather than a DSL, and it would be immediately friendlier and more powerful than make. Depending on what I was doing I would use doit, but for something simple I've still got make (and it would be easy to convert if desired, in fact I have some projects in mind that could do with this!).
- BiteCode_dev 3y ago
  Yes. I said it in another comment as well, but I'm rooting for a solid non turing complete conf language like CUElang (https://cuelang.org https://cuelang.org) to become popular. For now we can already use it to check and generate regular JSON or YAML, which is a good compat story. But imagine having this in CI instead of YAML files.
  - rad_gruchalski 3y ago
    I prefer Jsonnet for JSON and YAML
    - BiteCode_dev 3y ago
      There is also Dhall. I've tried the 3 of them, eventually I found CUElang was the best balance between expressiveness, ability to enforce and check for constraints, and easy of use. But I would welcome any of them replacing the YAML mess we currently have in sysadmin.
tjoff 3y ago
Python is such a massive dependency though, people who code in python rarely realize how big of a barrier it is for people not in that ecosystem. Package management is insane and every project has its own way of dealing with it. Installing python by itself is a chore on many operating systems.
- BiteCode_dev 3y ago
  Agreed. I wish more projects would compile their Python tool into a stand alone executable so that the users wouldn't have to deal with the installation process. I will at some point write about how to do that with Python, so that the knowledge will more broadly available and maybe the community will grow the habit of doing it. Meanwhile, I don't have more to offer than the "Relieving your Python packaging pain" procedure: https://www.bitecode.dev/p/relieving-your-python-packaging-pain https://www.bitecode.dev/p/relieving-your-python-packaging-p... It's not perfect, but it's an improvement for any people that don't do a lot of Python but has to.
  - tjoff 3y ago
    Excellent, one of the problems with this is that you can't google it. Well, of course you can but you'll get a million contradicting resources and all that float to the top are too basic anyway. I really enjoyed your post, saved and will pass it along. The mentioned "follow-up" article: https://www.bitecode.dev/p/back-to-basics-with-pip-and-venv https://www.bitecode.dev/p/back-to-basics-with-pip-and-venv (if you find the time, please add a link to it to make it even more complete)
    - BiteCode_dev 3y ago
      Thanks a lot, I planned to do it and I forgot. I've just added the link and it's indeed way better like this. Substack doesn't really have any way to make a series of articles, or to bundle stuff together, so linking manually is the only thing you can do that will give a little structure. One more reason to move off it when I find the time.
- deleted 3y ago
mongol 3y ago
I am thinking there must be a doit kind of tool but using bash shell instead. Anyone aware of one?
- BiteCode_dev 3y ago
  People that love bash usually are comfortable with make.
- pacifika 3y ago
  No guarantees, but I use https://github.com/svandragt/Taskfile https://github.com/svandragt/Taskfile Unfortunately it’s named the same.
theshrike79 3y ago
This is the exact reason why I moved to magefiles[0] in my Go projects. I had a semi-working Makefile base I spent days honing years ago, but then I ran into a few walls (can't remember which). Enter magefiles, I could use the same language in them as I could with my other code and stuff just worked. [0] https://magefile.org https://magefile.org
Brian_K_White 3y ago
Rather than downvote, since it's valid to state an opinion, I'll instead just do the same: Ugh. No.
zelphirkalt 3y ago
I guess that could work for a Python project, where you already have Python installed. But still you would need to set up a venv or similar to keep your system Python clean. Then you cannot run the tasks without creating that venv or similar. How to create the venv and install all the correct dev dependencies into it? You can't use the task runner yet, because you still need to install that ... Maybe a Makefile?
- BiteCode_dev 3y ago
  That's why I'm advocating for compiling tools stand alone in another comments. Having to install doit first is indeed going to be a barrier of entry for a lot of people. It's still worth it in my experience, but I get how it can deter users.

emptysongglass 3y ago

For Pythonistas or people interested in Python, I can't plug the commenter's newsletter enough: it is an absolute treasure trove of good advice for new and old users of Python. They did a whole series on the right way to install and use Python [1]. If you follow their advice you'll never struggle with any of the problems people complain of encountering with Python. [1] https://www.bitecode.dev/p/installing-python-the-bare-minimum https://www.bitecode.dev/p/installing-python-the-bare-minimu...

BiteCode_dev 3y ago
Cheers :)

rewmie 3y ago

> DSL are the plague of our field, for one that is useful like SQL, you have a hundred that turn your life into hell. The same can be said about using general purpose programming languages to specify stuff declaratively, but instead of implementation the problems happen in each and every single use of it. It only takes a single user that either does not comply with standards or feels they are particularly clever to mess everything for everyone.

BiteCode_dev 3y ago
The dose makes the poison

sdflhasjd 3y ago

> templated YAML based DSL are just insanity The truest statement of all of HN It started off with configuration files, but then we needed logic, control flow, or variables; and for some reason nobody goes "ok, maybe we need an actual progamming language now" and instead we've created horrors beyond our comprehension. Azure Pipelines is the worst I've used: three types of variables, conditional statements return string "True" and "False", and of course it's mostly undebuggable.

BiteCode_dev 3y ago
I have hopes that one day something like CUElang (https://cuelang.org https://cuelang.org) will become popular to be the configuration language of everything. Just enough logic to get by, but not turing complete, plus strong typing and embedded schemas. However, for now there is just a single implementation in GO.
- gavinhoward 3y ago
  Oh interesting. I came up with basically the same thing as CUE (as described in [1]) on my own, except for the heavy schema stuff. I think the schema stuff is unfortunately what's going to keep it from adoption. I mean, I didn't add schema stuff explicitly in, but you can define schemas in mine, which makes it easier. But making it explicit raises the bar for an implementation and may scare people away from writing one. [1]: https://cuelang.org/docs/tutorials/tour/intro/json/ https://cuelang.org/docs/tutorials/tour/intro/json/
hitchstory 3y ago
Templated YAML just means that something somebody thought was configuration either wasn't or wasnt under every circumstance. It's not really a problem with YAML. In most cases I think it's the fault of the creators of the YAML DSL (e.g. ansible) but with CI pipelines people really just should be writing scripts 100% in other languages and wiring them up to events with YAML. I dont have any sympathy for people using them as programming languages. I'm sure Microsoft is happy to indulge it though because hey, vendor lock-in.

veltas 3y ago

No need to apologise, I don't like YAML, I like Python, and I agree: I'd rather have a domain specific library than a DSL anyday. The best of both worlds is a good programming language that allows expressing domain specific libraries almost as if they were domain specific languages, without going so declarative that people don't know how to script anything anymore. I don't know if Python/doit really achieved that, but its task definitions seem almost as readable as the YAML, in a real language where I can do anything I want without asking permission. So I would always choose doit over Taskfile. But a better comparison is between doit and make. I much prefer make's basic syntax, although I dislike newer features which inevitably have stretched the syntax beyond its original simplicity, but conceivably you can achieve something similar to make in a language but as a library rather than a DSL, and it would be immediately friendlier and more powerful than make. Depending on what I was doing I would use doit, but for something simple I've still got make (and it would be easy to convert if desired, in fact I have some projects in mind that could do with this!).

BiteCode_dev 3y ago
Yes. I said it in another comment as well, but I'm rooting for a solid non turing complete conf language like CUElang (https://cuelang.org https://cuelang.org) to become popular. For now we can already use it to check and generate regular JSON or YAML, which is a good compat story. But imagine having this in CI instead of YAML files.
- rad_gruchalski 3y ago
  I prefer Jsonnet for JSON and YAML
  - BiteCode_dev 3y ago
    There is also Dhall. I've tried the 3 of them, eventually I found CUElang was the best balance between expressiveness, ability to enforce and check for constraints, and easy of use. But I would welcome any of them replacing the YAML mess we currently have in sysadmin.

tjoff 3y ago

Python is such a massive dependency though, people who code in python rarely realize how big of a barrier it is for people not in that ecosystem. Package management is insane and every project has its own way of dealing with it. Installing python by itself is a chore on many operating systems.

BiteCode_dev 3y ago
Agreed. I wish more projects would compile their Python tool into a stand alone executable so that the users wouldn't have to deal with the installation process. I will at some point write about how to do that with Python, so that the knowledge will more broadly available and maybe the community will grow the habit of doing it. Meanwhile, I don't have more to offer than the "Relieving your Python packaging pain" procedure: https://www.bitecode.dev/p/relieving-your-python-packaging-pain https://www.bitecode.dev/p/relieving-your-python-packaging-p... It's not perfect, but it's an improvement for any people that don't do a lot of Python but has to.
- tjoff 3y ago
  Excellent, one of the problems with this is that you can't google it. Well, of course you can but you'll get a million contradicting resources and all that float to the top are too basic anyway. I really enjoyed your post, saved and will pass it along. The mentioned "follow-up" article: https://www.bitecode.dev/p/back-to-basics-with-pip-and-venv https://www.bitecode.dev/p/back-to-basics-with-pip-and-venv (if you find the time, please add a link to it to make it even more complete)
  - BiteCode_dev 3y ago
    Thanks a lot, I planned to do it and I forgot. I've just added the link and it's indeed way better like this. Substack doesn't really have any way to make a series of articles, or to bundle stuff together, so linking manually is the only thing you can do that will give a little structure. One more reason to move off it when I find the time.
deleted 3y ago

mongol 3y ago

I am thinking there must be a doit kind of tool but using bash shell instead. Anyone aware of one?

BiteCode_dev 3y ago
People that love bash usually are comfortable with make.
pacifika 3y ago
No guarantees, but I use https://github.com/svandragt/Taskfile https://github.com/svandragt/Taskfile Unfortunately it’s named the same.

theshrike79 3y ago

This is the exact reason why I moved to magefiles[0] in my Go projects. I had a semi-working Makefile base I spent days honing years ago, but then I ran into a few walls (can't remember which). Enter magefiles, I could use the same language in them as I could with my other code and stuff just worked. [0] https://magefile.org https://magefile.org

Brian_K_White 3y ago

Rather than downvote, since it's valid to state an opinion, I'll instead just do the same: Ugh. No.

zelphirkalt 3y ago

I guess that could work for a Python project, where you already have Python installed. But still you would need to set up a venv or similar to keep your system Python clean. Then you cannot run the tasks without creating that venv or similar. How to create the venv and install all the correct dev dependencies into it? You can't use the task runner yet, because you still need to install that ... Maybe a Makefile?

BiteCode_dev 3y ago
That's why I'm advocating for compiling tools stand alone in another comments. Having to install doit first is indeed going to be a barrier of entry for a lot of people. It's still worth it in my experience, but I get how it can deter users.