3 ms·

Snowflake and Databricks are different, sometimes complementary technologies. You can store data in Snowflake & query it with Databricks for example: https://g

Snowflake and Databricks are different, sometimes complementary technologies. You can store data in Snowflake & query it with Databricks for example: https://github.com/snowflakedb/spark-snowflake https://github.com/snowflakedb/spark-snowflake Snowflake predicate pushdown filtering seems quite promising: https://www.snowflake.com/blog/snowflake-spark-part-2-pushing-query-processing/ https://www.snowflake.com/blog/snowflake-spark-part-2-pushin... Think both these companies can win.

jayparth 6y ago
There's definitely place in the industry for a no 1 and a no 2, but no. I don't think they're complementary, I think they're competitive. Both are trying to own the whole storage + compute layer eventually. I guess you can see Snowflake as a storage layer but they have attached scalable compute to it. Databricks started out as scalable compute and is now becoming more focused on storage. The idea of storing your data in Snowflake and querying it in Databricks is pretty silly. Why would you want to do that? Why not just use Snowflake's compute? Sure you could argue Spark has some transformations that are hard to express in SQL, but that is why Snowflake introduced Snowpark.