4 ms·

Ah yes, a home-made framework to generate configurations for your framework that's supposed to make your life easier. That way you can maintain your code that m

Ah yes, a home-made framework to generate configurations for your framework that's supposed to make your life easier. That way you can maintain your code that maintains your configs that make it easier to run your code that you have to maintain!

pharmakom 4y ago
This can be a much better approach than upgrading the DAG description language to a true programming language. It forces anything complex to happen at build time where it can do less damage. Plus, we can often use the same library to do static analysis on the output
- savin-goyal 4y ago
  Metaflow provides a similar concept to interface with Step Functions and Argo Workflows in Python - https://docs.metaflow.org/going-to-production-with-metaflow/scheduling-metaflow-flows/scheduling-with-aws-step-functions https://docs.metaflow.org/going-to-production-with-metaflow/...
latchkey 4y ago
Actually, yes. It allows for easier unit and integration testing as well. The original complaint is that things were getting hard to read and they wished there was code for this. It seems logical to create a framework for the json configuration files so that they can be easily mocked and tested. As someone who greatly values spending time on automated testing, it seems weird to not think of it this way. Quick google shows that others have done things like this already... [1] https://noise.getoto.net/2021/10/14/using-jsonpath-effectively-in-aws-step-functions/ https://noise.getoto.net/2021/10/14/using-jsonpath-effective... [2] https://aws.amazon.com/about-aws/whats-new/2022/01/aws-step-functions-support-workflows/ https://aws.amazon.com/about-aws/whats-new/2022/01/aws-step-... [3] https://docs.aws.amazon.com/step-functions/latest/dg/sfn-local-test-sm-exec.html https://docs.aws.amazon.com/step-functions/latest/dg/sfn-loc...

4 ms·

Ah yes, a home-made framework to generate configurations for your framework that's supposed to make your life easier. That way you can maintain your code that m

by entropicdrifter 4y ago

pharmakom 4y ago
This can be a much better approach than upgrading the DAG description language to a true programming language. It forces anything complex to happen at build time where it can do less damage. Plus, we can often use the same library to do static analysis on the output
- savin-goyal 4y ago
  Metaflow provides a similar concept to interface with Step Functions and Argo Workflows in Python - https://docs.metaflow.org/going-to-production-with-metaflow/scheduling-metaflow-flows/scheduling-with-aws-step-functions https://docs.metaflow.org/going-to-production-with-metaflow/...
latchkey 4y ago
Actually, yes. It allows for easier unit and integration testing as well. The original complaint is that things were getting hard to read and they wished there was code for this. It seems logical to create a framework for the json configuration files so that they can be easily mocked and tested. As someone who greatly values spending time on automated testing, it seems weird to not think of it this way. Quick google shows that others have done things like this already... [1] https://noise.getoto.net/2021/10/14/using-jsonpath-effectively-in-aws-step-functions/ https://noise.getoto.net/2021/10/14/using-jsonpath-effective... [2] https://aws.amazon.com/about-aws/whats-new/2022/01/aws-step-functions-support-workflows/ https://aws.amazon.com/about-aws/whats-new/2022/01/aws-step-... [3] https://docs.aws.amazon.com/step-functions/latest/dg/sfn-local-test-sm-exec.html https://docs.aws.amazon.com/step-functions/latest/dg/sfn-loc...