3 ms·
There's no silver bullet. Ultimately you're responsible for copying records from a source, no tool can magically solve that for you. If you change your schema o
by xyzzy_plugh 5y ago
There's no silver bullet. Ultimately you're responsible for copying records from a source, no tool can magically solve that for you. If you change your schema or need to backfill, it's good to have strong tools and frameworks to handle that.
Ultimately data pipelines are code, and need to be treated as code. If a library or API changes, you make the associated changes required.
The problem with all of these tools, CDC or not, select * or not, is they still just punt the problem to you. Instead, I've had wild success building tooling from first principles to fetch records in batches or streams, as a first class API rather than a backdoor hack.