2 ms·
Based on my own personal experience, a data lake is more of a data dumpster fire: a bunch of CSV, XML, or JSON files in random directories (typically in cloud o
by icedchai 3y ago
Based on my own personal experience, a data lake is more of a data dumpster fire: a bunch of CSV, XML, or JSON files in random directories (typically in cloud object storage like S3 or a clone) plus sort of system that processes that data (Amazon Athena, Spark, Hadoop, DuckDB, whatever) to generate reports. In some projects I've seen, the size of the "lake" has been vastly over stated and could be stored in RAM on any modern system.