2 ms·
It takes around 20 minutes to spin up HDInsight clusters. So it is not instant but I wonder if databricks on Azure can do any better given they are running in c
by mobileexpert 9y ago
It takes around 20 minutes to spin up HDInsight clusters. So it is not instant but I wonder if databricks on Azure can do any better given they are running in customer environments so wont have a big pool of preallocated multitenant machines available.
- rdeboo 9y agoAFAIK it is not multitenant, you get your own machines. They spin up the VMs with either open source Spark or the Databricks runtime - you get to choose the distribution and version before spinning up the cluster. If you have enough workloads you can run your own pool of VMs to provide a 'serverless' experience to your Spark users: https://databricks.com/blog/2017/06/07/databricks-serverless-next-generation-resource-management-for-apache-spark.html https://databricks.com/blog/2017/06/07/databricks-serverless...