4 ms·
I'm not familiar with S3 on datalake setup. When replicating a db table to S3, what format will be used? And I'm wondering if it's possible to update the S3 fi
by wejick 2y ago
I'm not familiar with S3 on datalake setup. When replicating a db table to S3, what format will be used?
And I'm wondering if it's possible to update the S3 files to reflect latest incoming changes on the db table?
- mritchie712 2y agoThe file format is often Parquet. The "table format" depend on what data lake you're using (e.g. Iceberg, Delta, etc.). If you know Python, here's[0] a practical example of how Iceberg works. 0 - https://www.definite.app/blog/iceberg-query-engine https://www.definite.app/blog/iceberg-query-engine