3 ms·
Wait what? I thought spark is designed to work on top of a shared nothing cluster? Have you any resources describing the architecture of databricks specifically
by anonymousDan 6y ago
Wait what? I thought spark is designed to work on top of a shared nothing cluster? Have you any resources describing the architecture of databricks specifically?
- snidane 6y agoIn Databricks you only get cloud storage (s3) as a persistent storage for data. So the data has to move from s3 to compute nodes. Self hosted HDFS will work more like shared nothing.
- anonymousDan 6y agoAh,thanks, makes sense.