3 ms·
You can use Databricks Repos (https://docs.databricks.com/repos/index.html https://docs.databricks.com/repos/index.html) specifically files in repos (https://do
by alexott 4y ago
You can use Databricks Repos (https://docs.databricks.com/repos/index.html https://docs.databricks.com/repos/index.html) specifically files in repos (https://docs.databricks.com/repos/work-with-notebooks-other-files.html#work-with-python-and-r-modules https://docs.databricks.com/repos/work-with-notebooks-other-...) functionality that allows to use Python files (not notebooks!) as Python modules.
Another alternative is to split notebooks into “library notebooks” that just define transformations, and “orchestration notebooks” that use code library notebooks to execute a “business logic”.
In both approaches you can do code testing, etc.
P.S. I have a demo of both approaches here: https://github.com/alexott/databricks-nutter-repos-demo https://github.com/alexott/databricks-nutter-repos-demo
- samuell 4y agoSweet, many thanks! Just looked into this, and this will be a huge improvement! Actually pleasantly surprised they store notebooks as .py files. Yay. (Looking at you, .ipynb files :/)