2 ms·
Ask HN: BI Developers, how do you manage your queries/scripts
Hi experts,
I'm wondering how do you manage and organize your queries, in case you work as a BI/DWH Developer who writes a lot of queries (but no DBA type of work).
On my side, I'm using Datagrip and my workflow looks like this:
- A main folder for the product (say a game)
- Two Level I folders, one for dashboards and one for reports
- Many Level II folders, each for one project (say, a feature of the game that product team needs a dashboard or several reports)
And then i put all of my SQL/Bash/Python scripts and Excel/Tableau workbooks under Level II folders, and occasionally create Level III folders if it's a big project that spans more than 4 sprints (2 months).
However so far I find there is a major flaw of this method:
- Difficult to locate frequently-used queries. These are NOT boiler-plate ones (I use Datagrip templates for these), but usually difficult and hacky ones that can be used in multiple projects, or multiple times in one project.
I'm wondering how do you organize your files/folders? There must be a better way to do this.