3 ms·
I have 100% seen data swamps in the wild: > We see customers creating big data graveyards, dumping everything into Hadoop distributed file system (HDFS) and ho
by dijksterhuis 6y ago
I have 100% seen data swamps in the wild:
> We see customers creating big data graveyards, dumping everything into Hadoop distributed file system (HDFS) and hoping to do something with it down the road. But then they just lose track of what’s there. The main challenge is not creating a data lake, but taking advantage of the opportunities it presents.
https://en.wikipedia.org/wiki/Data_lake https://en.wikipedia.org/wiki/Data_lake
> If you use your data lake as the source for your warehouse then you’re forced to keep it clean
So you apply a schema...? Determining what should and should not live in your storage? What format it should be in etc.?
That's not a data lake in my mind -- it's just the ETL staging area.
This is why I dislike the term data lake. It's so vague and loose that it could mean 300 different things to 300 different experts. Makes it a great term for snake oil marketing though.