3 ms·
The last point was for teams that only rely on notebooks, sorry if I didn't make that clear. You're right that all those issues can be sidestepped if you build
by MrPowers 6y ago
The last point was for teams that only rely on notebooks, sorry if I didn't make that clear.
You're right that all those issues can be sidestepped if you build projects in version controlled Git repos, test the code, and deploy JAR / Wheel files.
Speaking of testing, can you let me know if this PySpark testing fix worked for you ;) https://github.com/MrPowers/chispa/issues/6 https://github.com/MrPowers/chispa/issues/6
- alexott 6y agoI’m sorry for delay, will fix ASAP... My point is that you can do that even without jars/wheels - you can do VC and tests of notebooks. For example, https://github.com/alexott/databricks-nutter-projects-demo https://github.com/alexott/databricks-nutter-projects-demo
- Someone 6y agoAs alexott says, you can link notebooks to a git repository and commit and rollback them from the Databricks UI (https://docs.databricks.com/notebooks/github-version-control.html https://docs.databricks.com/notebooks/github-version-control...) Problem is that it’s limited. You can’t, for example, commit multiple files in one go (that improves a little bit in https://docs.databricks.com/projects.html https://docs.databricks.com/projects.html), or merge changes a colleague made with your changes. You also have to use the UI Databricks provides. You can’t use a git CLI or whatever GUI you prefer. (all AFAIK, but I’m fairly certain about it)