6 ms·

Creator here! Thanks for submitting - I hope people find this interesting. I’m excited to explore more use cases beyond listing and reformatting git log output!

Creator here! Thanks for submitting - I hope people find this interesting. I’m excited to explore more use cases beyond listing and reformatting git log output! There’s a similar project called gitbase, but I had difficulty just spinning that up for ad-hoc CLI querying so created this project in hope of a better UX for that type of use case. It uses SQLite virtual tables, which was something I had been meaning to play around with. This seemed like a cool use of them!

vcryan 6y ago
Can you query multiple repos at with one query? Interesting tool!
- patrickdevivo 6y ago
  At the moment, not easily, but someone else just asked about that and I think it's worth looking into! It makes sense and could be a cool use case to run queries on all the key repos in an org or project
  - princesse 6y ago
    This would make the difference between neat and useful for my team.
    - vcryan 6y ago
      same!
diegoholiveira 6y ago
A friend of mine did something similar a few years ago: https://github.com/filhodanuvem/gitql https://github.com/filhodanuvem/gitql
sdesol 6y ago
Cool. It's taken me years to find an efficient (and most importantly, easily debuggable) way to transfer Git's history to SQL. Since you are looking for use cases, I'll share with you some of the ways that I'm leveraging SQL to help drive code analytics for Git. Once Git's history is in an easy to query state, you can do some interesting things. For example, if you aggregate all the contributors in a Git repo, you can produce something like this: https://imgur.com/tT4DSJc https://imgur.com/tT4DSJc which makes it very easy to identify project investment/commitment. In the above example, you can see that Microsoft is heavily invested in vscode, as a lot of the developers that are contributing to it, have been doing so for more than 3 years. And if you aggregate contributions by file types, you can see how people are contributing to it as well. In the case of vscode, the contributions are mainly TypeScript contributions. Here is another contributor example, which shows GitLab contributions: https://imgur.com/3UU6Kda https://imgur.com/3UU6Kda What the above analytics shows, is GitLab has a lot of contributors and a lot of them are new contributors (6 months or less), which makes sense since they were hiring aggressively not too long ago. Not sure if this is still the case with Covid-19, but this can be easily confirmed 6 months from now, with the same chart. Now for something more interesting in my opinion, which is code review analytics. https://imgur.com/xWH6QFl https://imgur.com/xWH6QFl It has taken a lot of research and development to get to this point, but once you can easily query Git, you can surface very interesting things by cross referencing it with external systems, like GitHub's pull request system. In the pull requests screen shot, I created a window that only considers open pull requests that were updated within the last 30 days. With this type of window, I can see what has changed across dozens, if not hundreds or thousands of pull requests. For example, I can easily identify file collisions, between pull requests. When was their last commit, and so forth. I'm still working on refining my code review analytics, but the goal is to get it to an advanced state, where you can see exactly what is happening between pull requests and to derive insights from those requests. So those are just some of the use cases that I've developed, which leverages being able to query a Git's history with SQL.
- jacques_chester 6y ago
  Do you have a schema somewhere? I've been working on a dataset that includes git repositories and I've been muddling through it slowly. It's built around a dataset of ~126k builds I collected some time back, plus ~5k Pivotal Tracker stories for the same time period. Covers about 2.5 years of 3 teams. The hardest parts have been (1) dealing with actual lines, which I gave up on and (2) very busy robot repos with hundreds of thousands of commits. My goal is to release the data as a single integrated set, but there's a ways to go. For one thing I need to find everyone in it to ask if they're OK with me doing so.
Matt3o12_ 6y ago
That seems like a really cool project, but I couldn't figure out how exactly the where filter was implemented. Can you point me to the code that implements it?

vcryan 6y ago

Can you query multiple repos at with one query? Interesting tool!

patrickdevivo 6y ago
At the moment, not easily, but someone else just asked about that and I think it's worth looking into! It makes sense and could be a cool use case to run queries on all the key repos in an org or project
- princesse 6y ago
  This would make the difference between neat and useful for my team.
  - vcryan 6y ago
    same!

diegoholiveira 6y ago

A friend of mine did something similar a few years ago: https://github.com/filhodanuvem/gitql https://github.com/filhodanuvem/gitql

sdesol 6y ago

Cool. It's taken me years to find an efficient (and most importantly, easily debuggable) way to transfer Git's history to SQL. Since you are looking for use cases, I'll share with you some of the ways that I'm leveraging SQL to help drive code analytics for Git. Once Git's history is in an easy to query state, you can do some interesting things. For example, if you aggregate all the contributors in a Git repo, you can produce something like this: https://imgur.com/tT4DSJc https://imgur.com/tT4DSJc which makes it very easy to identify project investment/commitment. In the above example, you can see that Microsoft is heavily invested in vscode, as a lot of the developers that are contributing to it, have been doing so for more than 3 years. And if you aggregate contributions by file types, you can see how people are contributing to it as well. In the case of vscode, the contributions are mainly TypeScript contributions. Here is another contributor example, which shows GitLab contributions: https://imgur.com/3UU6Kda https://imgur.com/3UU6Kda What the above analytics shows, is GitLab has a lot of contributors and a lot of them are new contributors (6 months or less), which makes sense since they were hiring aggressively not too long ago. Not sure if this is still the case with Covid-19, but this can be easily confirmed 6 months from now, with the same chart. Now for something more interesting in my opinion, which is code review analytics. https://imgur.com/xWH6QFl https://imgur.com/xWH6QFl It has taken a lot of research and development to get to this point, but once you can easily query Git, you can surface very interesting things by cross referencing it with external systems, like GitHub's pull request system. In the pull requests screen shot, I created a window that only considers open pull requests that were updated within the last 30 days. With this type of window, I can see what has changed across dozens, if not hundreds or thousands of pull requests. For example, I can easily identify file collisions, between pull requests. When was their last commit, and so forth. I'm still working on refining my code review analytics, but the goal is to get it to an advanced state, where you can see exactly what is happening between pull requests and to derive insights from those requests. So those are just some of the use cases that I've developed, which leverages being able to query a Git's history with SQL.

jacques_chester 6y ago
Do you have a schema somewhere? I've been working on a dataset that includes git repositories and I've been muddling through it slowly. It's built around a dataset of ~126k builds I collected some time back, plus ~5k Pivotal Tracker stories for the same time period. Covers about 2.5 years of 3 teams. The hardest parts have been (1) dealing with actual lines, which I gave up on and (2) very busy robot repos with hundreds of thousands of commits. My goal is to release the data as a single integrated set, but there's a ways to go. For one thing I need to find everyone in it to ask if they're OK with me doing so.

Matt3o12_ 6y ago

That seems like a really cool project, but I couldn't figure out how exactly the where filter was implemented. Can you point me to the code that implements it?