I think flow based programming principles can do wonders for workflow systems and composable analytics pipelines. We have recently used Spotify's luigi workflo

I think flow based programming principles can do wonders for workflow systems and composable analytics pipelines. We have recently used Spotify's luigi workflow system a lot, and ran into some tricky problems with a bit more complex workflows, that was only solved after we incorporated a bit of FBP principles (in/out ports and separate network definition, to be exact). Have documented a bit of it here: https://medium.com/@saml/loosely-coupled-tasks-in-luigi-workflows-6840d32e2824 https://medium.com/@saml/loosely-coupled-tasks-in-luigi-work... (fairly detailed and technical post though, not sure how easy it is to follow). At least in my experience so far, FBP principles really help to create the kind of modularity and composability that is needed in these kind of applications.