4 ms·
I've gone through a large number of these, and I think that Airflow is the best on Kubernetes for managed orchestration. The things I like are: * Source con
by dmayle 6y ago
I've gone through a large number of these, and I think that Airflow is the best on Kubernetes for managed orchestration. The things I like are:
* Source control for workflows/DAGs (using git-sync)
* Tracking/retries with SLAs
* Jobs run in Kubernetes
* Web UI for management
* Fully open source
I also use Argo Workflows, because I like its native handling of Kubernetes objects (e.g. the ability to manage and update a deployment as one of the steps), but it just doesn't have the orchestration/tracking side of things very well managed yet
- tiagod 6y agoI absolutely love Cadence/Temporal. It can do anything the others can do (and you could implement any other engine with it), without any DSLs.
- silasb 6y agoJust found out about Temporal and it looks interesting. I eager to jump in but our organization primarily uses Ruby. I know the big difference between Cadence vs Temporal is the fact they are using GRPC which seems much easier to adopt.
- ttsda 6y agoI also find the API to be more consistent on Temporal (I began implementing a project in Cadence before finding out about Temporal)
- mfateev 6y agoTemporal has an externally contributed Ruby client. It is going to be open sourced very soon.
- silasb 6y agoVery exciting news. Is there a way that I can subscribe to get info on when it'll be released?
- mfateev 6y agoThere is #ruby-sdk channel in the Temporal Slack: https://join.slack.com/t/temporalio/shared_invite/zt-c1e99p8g-beF7~ZZW2HP6gGStXD8Nuw https://join.slack.com/t/temporalio/shared_invite/zt-c1e99p8...
- deleted 6y ago[deleted]
- cpdean 6y agoI think this is the link to cadence: https://github.com/uber/cadence https://github.com/uber/cadence and temporal: https://github.com/temporalio/temporal https://github.com/temporalio/temporal
- ttsda 6y agoYep, this is it. They're developed by the same team, Temporal is still beta but I believe the production version is coming out late-June. I can't find an easy way to explain everything it does, but it pretty much allows you to write naive functions with no error handling, with month-long sleeps, auto-retries on unreliable function calls, etc etc. It also gives you a web interface when you can inspect the running functions, and allows for external code (and other workflow functions) to signal/query the running workflows.
- autogenerated 6y agoMoving off of Airflow and to Cadence/Temporal was the single biggest relief in terms of maintainability, operational ease and scalability. Also +1 on being free of any DSL.
- ttsda 6y agoI'm currently moving from a custom yaml DSL-based engine to Temporal and it's the best architectural decision I've taken in a long time. I researched a lot and couldn't find anything that even came close to the freedom it provides.
- alexg123 6y agoCurious about this. Can you elaborate more? Also happy to hop on a call/zoom if you don't want to share publicly (email me at saguziel@gmail.com). I'm working on something similar.
- bjacobso 6y agoI'd also like to +1 on Cadence/Temporal. It's a really great mental model and framework for thinking about workflows. This is more of a proof of concept but it can also support DSLs (although we found the go client is easier to understand than DSLs): https://github.com/checkr/states-language-cadence https://github.com/checkr/states-language-cadence
- mfateev 6y agoAs a team lead of the Cadence/Temporal it is really amazing to hear such a feedback. Thank you! BTW. You can AMA about the technology here.
- ttsda 6y agoThank you for your work! I believe once it's properly documented and marketed, it will be very popular.