I've mentioend this whenever iceberg comes up. It's wild how immature the ecosystem is still. Duckdb itself lacks the ability to write iceberg.... https://duck

I've mentioend this whenever iceberg comes up. It's wild how immature the ecosystem is still. Duckdb itself lacks the ability to write iceberg.... https://duckdb.org/docs/stable/extensions/iceberg/overview.html#limitations https://duckdb.org/docs/stable/extensions/iceberg/overview.h... Apache iceberg go ? Nope https://github.com/apache/iceberg-go?tab=readme-ov-file#readwrite-data-support https://github.com/apache/iceberg-go?tab=readme-ov-file#read... Basically java iceberg is the only mature way to do this, it's not a very accessible ecosystem. For a side project I'm using pyiceberg to sink streaming data to iceberg (using DuckDB as the stream processor): https://sql-flow.com/docs/tutorials/iceberg-sink https://sql-flow.com/docs/tutorials/iceberg-sink It's basically a workaround for DuckDB's lack of native support. I am very happy with the Pyicerbg library as a user, It was very easy and the native Arrow support is a glimpse into the future. Arrow as an interchange format is quite amazing. Just open up the iceberg table and append Arrow dataframes to it! https://github.com/turbolytics/sql-flow https://github.com/turbolytics/sql-flow Arrow is quite spectacular and it's cool to see the industry moving to standardize on it as a dataframe. For example, Clickhouse python also support arrow-based insertion: https://sql-flow.com/docs/tutorials/clickhouse-sink https://sql-flow.com/docs/tutorials/clickhouse-sink This makes the glue code trivial to sink into these different systems as long as arrow is used.