5 ms·

Databricks is way more than hadoop or spark. A great analogy - Spark is a great engine but you need to design and build all of the other subsystems. Databricks

Databricks is way more than hadoop or spark. A great analogy - Spark is a great engine but you need to design and build all of the other subsystems. Databricks is an F1 car - everything is built out. You get in and drive - FAST.

glogla 5y ago
> Databricks is an F1 car F1 cars really unreliable and need a lot of engineers to keep running, are very expensive, and completely impractical in normal use. They are fast but only on very specific roads, they couldn't survive on normal roads. What do you know, you might be right! :D
- drawturkey 5y ago
  You nailed it. Meanwhile the rest of the world just needs a camry.
dreyfan 5y ago
Databricks is a shit platform that encourages terrible data practices and accretion of technical debt.
- exsmelliarmus 5y ago
  Seems pretty good to us! Can you give more information?
  - 0x500x79 5y ago
    As people noted elsewhere, you have to be VERY careful with using databricks for a full data warehouse due to the fact that it drives you to notebook driven development and scheduling of those notebooks when data pipelines should follow similar development practices as other software projects. Great for proof of concepts, but when you start to build out complete pipelines please look into how to make the pipelines more sustainable and maintainable.
- matt123456789 5y ago
  As with other offerings in this space, the key to managing technical debt is to get functions out of notebooks ASAP, stage intermediate results where appropriate, and turn everything into jobs.
- fs111 5y ago
  Finally somebody that has used Databricks! I can't believe all the praise I read elsewhere in the comments here. Databricks is broken in so many ways, it is beyond me how anyone can like using this.
  - deleted 5y ago
    [deleted]
fs111 5y ago
> Databricks is an F1 car - everything is built out. You get in and drive - FAST. found the databricks employee

glogla 5y ago

> Databricks is an F1 car F1 cars really unreliable and need a lot of engineers to keep running, are very expensive, and completely impractical in normal use. They are fast but only on very specific roads, they couldn't survive on normal roads. What do you know, you might be right! :D

drawturkey 5y ago
You nailed it. Meanwhile the rest of the world just needs a camry.

dreyfan 5y ago

Databricks is a shit platform that encourages terrible data practices and accretion of technical debt.

exsmelliarmus 5y ago
Seems pretty good to us! Can you give more information?
- 0x500x79 5y ago
  As people noted elsewhere, you have to be VERY careful with using databricks for a full data warehouse due to the fact that it drives you to notebook driven development and scheduling of those notebooks when data pipelines should follow similar development practices as other software projects. Great for proof of concepts, but when you start to build out complete pipelines please look into how to make the pipelines more sustainable and maintainable.
matt123456789 5y ago
As with other offerings in this space, the key to managing technical debt is to get functions out of notebooks ASAP, stage intermediate results where appropriate, and turn everything into jobs.
fs111 5y ago
Finally somebody that has used Databricks! I can't believe all the praise I read elsewhere in the comments here. Databricks is broken in so many ways, it is beyond me how anyone can like using this.
- deleted 5y ago
  [deleted]

fs111 5y ago

> Databricks is an F1 car - everything is built out. You get in and drive - FAST. found the databricks employee