2 ms·
1. Great explanation of ELT, for a long time ETL monkey this broke through for me. 2. GDPR/COPPA means you really have to think about your data lake before dump
by banku_brougham 5y ago
1. Great explanation of ELT, for a long time ETL monkey this broke through for me.
2. GDPR/COPPA means you really have to think about your data lake before dumping verbatim into long term storage.
- jpgvm 5y agoUltimately regulations will change over time. I think the most important thing to keep in mind is storing data in a form that is amenable to large scale batch transformation. This way you can respond by kicking off a task to "rewrite the world" to scrub fields or drop records. Bug reason why I encourage stuff like Avro, Parquet or the even just loading to BigQuery.