3 ms·
We use Stitch[1] for this. You can selectively replicate some tables from any "source" to any selected destination[2]. I believe 1 destination is there on the f
by sidmitra 5y ago
We use Stitch[1] for this. You can selectively replicate some tables from any "source" to any selected destination[2]. I believe 1 destination is there on the free plan. You can point to any self hosted(or cloud) Postgres DBs. There are a bunch of supported destinations eg. Postgres, Snowflake etc. The sources also can be a varied like Google sheets.
We use this to send relevant tables to analytics teams outside of engineering. There's a bit of tweaking initially(i.e. having a timestamp column like 'last_updated` helps to replicate only those rows)
Having a separate readonly DB user is also pretty common. We use that internally inside engineering(where we trust them slightly more). Important bit is to always add a statement_timeout so people cannot not run random slow queries on your production database that are hung for days.
I've also had my eye on https://www.singer.io/ https://www.singer.io/ , which seems like an open source ETL tool similar to Stitch. But i've not experimented with it locally yet. It uses "taps" and "targets" nomenclature. And there's a healthy ecosystem of opensource extensions of creating your own taps/targets.
[1] https://www.stitchdata.com/ https://www.stitchdata.com/
[2] https://www.stitchdata.com/integrations/destinations/ https://www.stitchdata.com/integrations/destinations/