3 ms·
hmm.. atleast databricks claims that the GPU clusters are beta, but generally available https://docs.databricks.com/user-guide/clusters/gpu.html https://docs.d
by sandGorgon 10y ago
hmm.. atleast databricks claims that the GPU clusters are beta, but generally available
https://docs.databricks.com/user-guide/clusters/gpu.html https://docs.databricks.com/user-guide/clusters/gpu.html
https://databricks.com/blog/2016/12/21/deep-learning-on-databricks.html https://databricks.com/blog/2016/12/21/deep-learning-on-data...
In addition, IBM has multiple projects around GPU aware spark - https://github.com/IBMSparkGPU https://github.com/IBMSparkGPU
http://www.spark.tc/gpu-acceleration-on-apache-spark-2/ http://www.spark.tc/gpu-acceleration-on-apache-spark-2/
- nl 10y agoYes the Databricks stuff is available on their cluster. It isn't part of Spark the Open Source project. Yes, as I said elsewhere there are plenty of projects to enable GPU usage via Spark. Have you actually tried them though? I have (eg https://github.com/IBMSparkGPU/GPUEnabler/issues/25 https://github.com/IBMSparkGPU/GPUEnabler/issues/25 ) and there are... issues.