4 ms·
Can you expand on that last sentence? Not that I’d disagree, I’m just curious about what specifically you’re referring to. My team is currently underwater supp
by pgwhalen 8y ago
Can you expand on that last sentence? Not that I’d disagree, I’m just curious about what specifically you’re referring to.
My team is currently underwater supporting a decades old system of batch ETL, where it’s a major challenge to understand dependencies between jobs, and therefore what to rerun when something fails. We’re exploring a move to streaming ETL (Kafka streams being the leading candidate) as a way of making the dependencies explicit: rather than having to rerun stored procs or Talend jobs that we depend on certain data, we run a fleet of 24/7 microservices that transform and move data downstream whenever it is available.