3 ms·

Oh this sounds cool! When I was writing this post I was thinking about ASTs and transforming them into domain-specific semantic graphs. E.g. that `run_config`

by feifan 6y ago

Oh this sounds cool! When I was writing this post I was thinking about ASTs and transforming them into domain-specific semantic graphs. E.g. that `run_config` example would be a generic "method call" node in an AST, but a domain-specific AST crawler could recognize that a "method call node whose name is `run_config`" should be replaced with a semantically-meaningful `run_config` node. And maybe that would be an interesting way to build up a conceptual graph of a codebase?

lstamour 6y ago
I'm with you on this one. I'd actually suggest https://github.com/CoatiSoftware/Sourcetrail https://github.com/CoatiSoftware/Sourcetrail could be extended to do this, though I haven't found the time yet for my own codebases. For example https://github.com/CoatiSoftware/SourcetrailPythonIndexer https://github.com/CoatiSoftware/SourcetrailPythonIndexer and under the hood the file format is SQLite: https://github.com/CoatiSoftware/SourcetrailDB https://github.com/CoatiSoftware/SourcetrailDB
gitrog 6y ago
Although I understand what we're trying to achieve here is easier code navigation and manipulation on a much broader scale with preferably a single tool, I couldn't shake the feeling that some of the examples you gave have existing solutions. The `run_config` one stood out the most. Isn't it perfectly feasible to have a `jobs` and a related `run_configs` table, instead of them only being objects? You could even put them in a separate database, much like one does with the auditing tables in a Rails application, or with reporting tables that need to be accessible by the likes of PowerBI. That separate database could have it's own light-weight GUI, something that just plugs your data structure into the interface, like Django's admin back-end.
- feifan 6y ago
  Sure, but then you'd have the "config" for a job separate from its implementation. If I'm looking at the code for a job, how do I easily find when it's scheduled to run? If I'm looking at the config database, how do I easily get to the implementation of the job? Putting aside the feasibility of migrating a large codebase that exists one way or the other, my point here is that these shouldn't be separate tools.

by feifan 6y ago

lstamour 6y ago

I'm with you on this one. I'd actually suggest https://github.com/CoatiSoftware/Sourcetrail https://github.com/CoatiSoftware/Sourcetrail could be extended to do this, though I haven't found the time yet for my own codebases. For example https://github.com/CoatiSoftware/SourcetrailPythonIndexer https://github.com/CoatiSoftware/SourcetrailPythonIndexer and under the hood the file format is SQLite: https://github.com/CoatiSoftware/SourcetrailDB https://github.com/CoatiSoftware/SourcetrailDB

gitrog 6y ago

Although I understand what we're trying to achieve here is easier code navigation and manipulation on a much broader scale with preferably a single tool, I couldn't shake the feeling that some of the examples you gave have existing solutions. The `run_config` one stood out the most. Isn't it perfectly feasible to have a `jobs` and a related `run_configs` table, instead of them only being objects? You could even put them in a separate database, much like one does with the auditing tables in a Rails application, or with reporting tables that need to be accessible by the likes of PowerBI. That separate database could have it's own light-weight GUI, something that just plugs your data structure into the interface, like Django's admin back-end.

feifan 6y ago
Sure, but then you'd have the "config" for a job separate from its implementation. If I'm looking at the code for a job, how do I easily find when it's scheduled to run? If I'm looking at the config database, how do I easily get to the implementation of the job? Putting aside the feasibility of migrating a large codebase that exists one way or the other, my point here is that these shouldn't be separate tools.