rywalker

9 ms·

61.
by rywalker 6y ago
We've outlined some of the key new features of Airflow 2.0 here - https://www.astronomer.io/airflow - and it's available on Astronomer Cloud today.
62.
by rywalker 6y ago
firing your devops person? maybe wait a few weeks to be sure!
63.
by rywalker 6y ago
service never came up; also can't delete it :( "Environments with CREATING status must complete previous operation before initiating a new operation." can't email support for help (i only have basic plan) any AWSMWAA ppl
64.
by rywalker 6y ago
if you like YAML-based DAGs, you can do that in Airflow with the dag-factory extension https://github.com/ajbosco/dag-factory
65.
by rywalker 6y ago
fyi airflow 2.0 ships in the next few weeks, we're made a lot of improvements https://www.youtube.com/playlist?list=PLCi-q9vYo4x-PESoBcXN0...
66.
by rywalker 6y ago
i'd let you know if their service came up :) been waiting an hour after hitting the "create" button disclosure: co-founder of astronomer
67.
by rywalker 6y ago
You could certainly execute a Terraform script from an Airflow DAG, but I wouldn't say it's a common use case.
68.
by rywalker 6y ago
Yes, we need to improve the backfills in Airflow. We're working on making the scheduler HA and more performant, reach out to me if you'd like to collaborate on your use case (ry at astronomer dot io)
69.
by rywalker 6y ago
1) An official Helm chart is coming very soon. We (Astronomer) have a commercial platform that aims to solve this completely including observability, configuration and deployment. Happy to team up to improve Airflow's open-source k8s s
70.
by rywalker 6y ago
We were a very small company when we discovered Airflow, just saying, and it provided a lot of value to us vs. alternatives. We fell in love with Airflow's capability to allow us to write one small python program that generated hundred
71.
by rywalker 6y ago
re: ETL framework - you can get a lot done with the built-in Airflow Operators, including the PythonOperator (and bring in any python dependency you like) and BashOperator (call any CLI, etc.) - it's not drag-and-drop, but I've fo
72.
by rywalker 6y ago
It's not quite "open source", better labelled as "source available"
73.
by rywalker 6y ago
Implying that Airflow is a simple mixture of a few ingredients is selling it quite short. There are a lot of knobs and switches in Airflow (i.e. features) that have been built and battle-tested over a lot of users. It has quite a lot of dep
74.
Profiling the Airflow Scheduler(astronomer.io)
2 points by rywalker 7y ago|0 comments
75.
Why Airflow?(astronomer.io)
3 points by rywalker 7y ago|0 comments
76.
Outsourced IT for Startups?
1 points by rywalker 7y ago|1 comments
77.
by rywalker 7y ago
No.
78.
by rywalker 7y ago
yep, just wasted 10 min on this :)
79.
The Autonomous Dataflow Service(ascend.io)
1 points by rywalker 7y ago|0 comments
80.
Announcing Astronomer v0.9, a Platform to Run Apache Airflow on Kubernetes(astronomer.io)
4 points by rywalker 7y ago|0 comments
81.
by rywalker 8y ago
Yes I need to hire a React dev right now :)
82.
by rywalker 8y ago
Super exited to see the project get out of incubation. It's an amazing tool.
83.
by rywalker 8y ago
Great overview. Do you plan to open source the DAG dependency graph tool?
84.
by rywalker 8y ago
I'd suggest using a service like Segment or MetaRouter.io to collect the event data, then use Airflow to prep it.
85.
by rywalker 8y ago
+1 from me as well. Well done.
86.
by rywalker 8y ago
This is likely a middle case, not enough traction to get next round of funding as a viable product/company. I personally use Astro on my iPhone (just uninstalled this morning when I got their notification). It was good app, but no way
87.
by rywalker 8y ago
My guess is that you had an underpowered database instance backing Airflow.
88.
by rywalker 9y ago
Here we have a company that is simply opening up visibility of the code of their commercial offering, which will enhance customer relationships vs. keeping the code black box —— and it's observed as user-hostile? Open core isn't e
89.
Opening Up Astronomer Open(astronomer.io)
1 points by rywalker 9y ago|0 comments
90.
by rywalker 9y ago
They are working to resolve https://medium.com/@dwalsh.sdlr/hi-ezequiel-6855beab08cb Understand the pause, and if we were a social networking company I'd be more concerned - but already our decision to use GraphQL

More ›