Interestingly, my team is actually moving off Databricks. I have anecdotally heard the same from other teams in the industry (buy side finance). We found that

Interestingly, my team is actually moving off Databricks. I have anecdotally heard the same from other teams in the industry (buy side finance). We found that notebook-based development is actually an antipattern for software engineering. It was ostensibly helpful for the narrower "data science" use case, but we have a much more robust ETL and research platform we built on our own using Pandas, Dask, Prefect and AWS. And personally I hated writing code in notebooks. If you're attached to that, you can basically get the same thing by using PyCharm in scientific mode with cell execution.