2 ms·
For historical reasons my team uses Makefiles for executing data science workflows[1]. They are auto-documenting which makes it not just helpful on me when I fo
by kelseyfrog 5y ago
For historical reasons my team uses Makefiles for executing data science workflows[1]. They are auto-documenting which makes it not just helpful on me when I forget an underused target, but it's perhaps even more useful when onboarding new team members. Rather than directing them to some out of band docs, it's convenient to simply direct them to use make to get the help they need[2].
1. There are reasons, like consistency of interface which I value even more than autodocumentation, but that is outside the scope of this discussion.
2. For small orders of help.
- nerdponx 5y agoFor data science specifically, I would strongly suggest looking into DVC: https://dvc.org/ https://dvc.org/. You can easily write DVC stage files by hand as a straightforward Makefile replacement, and integrate other features into your workflow as needed/desired.
- andreineculau 5y agoThis is exactly my experience which lead me to create https://github.com/ysoftwareab/yplatform https://github.com/ysoftwareab/yplatform - with a consistent make interface https://github.com/ysoftwareab/yplatform/tree/master/build.mk https://github.com/ysoftwareab/yplatform/tree/master/build.m... PS: quite feature complete but not yet well marketed so to speak. I'm actually recording an asciinema session this week in order for a visitor to grasp quicker the mentioned benefits.