2 ms·
We also had issues with Glue/Athena, never got parquet to work right, have schema issues with schemaless data coming from MongoDB, and in general it is obtuse a
by dumbfounder 6y ago
We also had issues with Glue/Athena, never got parquet to work right, have schema issues with schemaless data coming from MongoDB, and in general it is obtuse and hard to work with. We did a data lab with AWS using Glue and it was an exercise in pain. Then we did a POC with Snowflake and we cried with joy at how easy it was to work with large amounts of data. But our ELT was very light, not true transformation of data, more rearranging. I still think we will need Glue for some workloads and I pray this makes our lives easier.
- chrisjc 6y agoSounds very similar to what we're dealing with, although never turned to Glue to try and resolve our challenge. Ended up going with an ETL as a service (Alooma and now transitioning to ETLWorks) to extract and load our data into Snowflake.