2 ms·
This is such a big deal. Trying to get stuff done with geospatial data in native formats in big companies has been somewhat painful for a while, because a lot o
by ZeroCool2u 2y ago
This is such a big deal. Trying to get stuff done with geospatial data in native formats in big companies has been somewhat painful for a while, because a lot of them tend to default to using DataBricks and Delta format.
At least now there's a great open source tech stack combo with Trino as your query engine (which is decidedly a lot less annoying to run at scale than OSS Spark) and Iceberg as your storage format that a lot of your "Enterprise Architecture" types can be comfortable enough with.
- tiems 2y agoSince it’s getting added to the parquet spec itself, it should hopefully make its way to delta too soon.
- Mosarwat 2y agoPlease note that not all query engines supports the native geo type in Iceberg yet. The first one to support it is Apache Sedona, which works well with Spark: https://github.com/apache/sedona https://github.com/apache/sedona However, the ultimate goal is to make more engines (e.g., Arrow, Trino...) support the geo type too