3 ms·
The number of definitions here suggest its not well defined, one more :) A data lake is when you store your data in object storage (S3, GCS etc) as opposed to
by cavisne 8y ago
The number of definitions here suggest its not well defined, one more :)
A data lake is when you store your data in object storage (S3, GCS etc) as opposed to a filesystem (HDFS) or some indexed datastore (Redshift etc).
This potentially saves a lot of money because you can scale compute separately from storage, and object storage can be extremely cost effective compared to running a distributed filesystem.
Where the two overlap is when you store something like parquet in object storage, the file format is somewhat indexed already so you spend a bit more money preprocessing it but save a lot of money querying it.
I think whether its "raw" json or log files or preprocessed parquet doesnt really differentiate whether its a data lake or not
- kakoni 8y agoBut interestingly delta.io only supports HDFS currently.