7 ms·

SCM as a database for the code

95 points by gritzko 8mo ago

mtsolitary 8mo ago
I’ve recently been thinking about this too, here is my idea: https://clintonboys.com/projects/lit/ https://clintonboys.com/projects/lit/
- gethly 8mo ago
  > Like everyone else with a keyboard, a brain and a vested interest in their future as a programmer, I’ve been using AI agents to write a lot of code recently. alrighty then..
- fragmede 8mo ago
  Fascinating! https://github.com/clintonboys/lit-demo-crud https://github.com/clintonboys/lit-demo-crud isn't deep enough to really get a feel for how well this would work in practice.
- ferroman 8mo ago
  Using prompt as source of truth is not reliable, because outputs for the same prompt may vary
  - mtsolitary 8mo ago
    Of course. But if this is the direction we’re going in, we need to try and do something no? Did you read the post I linked?
sublinear 8mo ago
Missing 4 out of 5 parts. The attempt to wrest control away from the open source community doesn't even have the effort to keep going.
- gritzko 8mo ago
  Yes, parts will keep coming, this weekend and the next week.
- bonzini 8mo ago
  The gist was created 1 hour before your comment.
forrestthewoods 8mo ago
Definitely agree that Git is mediocre-at-best VCS tool. This has always been the case. But LLMs are finally forcing the issue. It’s a shame a whole generation programmers has only used Git/GitHub and think it’s good. Monorepo and large binary file support is TheWay. A good cross-platform virtual file system (VFS) is necessary; a good open source one doesn’t exist today. Ideally it comes with a copy-on-write system for cross-repo blob caching. But I suppose that’s optional. It lets you commit toolchains for open source projects which is a dream of mine. Not sure I agree that LSP like features need to be built in. That feels wrong. That’s just a layer on top. Do think that agent prompts/plans/summaries need to be a first class part of commits/merges. Not sure the full set of features required here.
- JodieBenitez 8mo ago
  > It’s a shame a whole generation programmers has only used Git/GitHub and think it’s good. Well... I used SVN before that and it was way worse.
  - EliRivers 8mo ago
    Continuing the theme, a new starter at my place (with about a decade of various experience, including an international financial services information player whose name is well known) had never used git or indeed any distributed, modern source control system. HN is a tiny bubble. The majority of the world's software engineers are barely using source control, don't do code reviews, don't have continuous build systems, don't have configuration controlled release versions, don't do almost anything that most of HN's visitors think are the basic table stakes just to conduct software engineering.
  - ryanm101 8mo ago
    Clearly the poster has never encountered cvs or cvs-next. And clear OP hasn't heard of vss..
    - fragmede 8mo ago
      RCS, anyone?
solarized 8mo ago
Talk is cheap. Show me the code.
nylonstrung 8mo ago
Trustfall seems really promising for querying files as if they were a db https://github.com/obi1kenobi/trustfall https://github.com/obi1kenobi/trustfall
hole_in_foot 8mo ago
[dead]
charcircuit 8mo ago
>I definitely reject the "git compatible" approach If your version control system is not compatible with GitHub it will be dead on arrival. The value of allowing people to gradually adopt a new solution can not be understated. There is also value in being compatible with existing git integrations or scripts in projects build systems.
- quadrifoliate 8mo ago
  Based on reading this, I don't see anything that would prevent keeping track of a repo tracked by this database with Git (and therefore GitHub) in addition to the database. I think the "compatible" bit means more that you have to think in terms of Git concepts everywhere. Curious what the author thinks though, looks like it's posted by them.
  - gritzko 8mo ago
    Technically, exporting changes either way is not a challenge. It only becomes difficult if we have multiple gateways for some reason. One way to do it is to use the new system for the messy part and git/GitHub for "publication".
- conartist6 8mo ago
  A system as described could be forwards compatible with git without being backwards compatible with git. In other words let people migrate easily, but don't force the new system to have all the same flaws of the old
  - ongy 8mo ago
    What issues do you see in git's data model to abandon it as wire format for syncing?
    - conartist6 8mo ago
      I wouldn't say I want to abandon anything git is doing as much as evolve it. Objects need to be able to contain syntax tree nodes, and patches need to be able to target changes to particular locations in a syntax tree instead of just by line/col.
deleted 8mo ago
[deleted]
benrutter 8mo ago
> The monorepo problem: git has difficulty dividing the codebase into modules and joining them back Can anyone explain this one? I use monorepos everyday and although tools like precommit can get a bit messy, I've never found git itself to be the issue?
- gritzko 8mo ago
  Based on my personal experience, big-corp monorepos have all features of a black hole: they try to suck in all the existing code (vendor it) and once some code starts living in a monorepo, there is no way to separate it as it becomes entangled with the entire thing. There is no way to "blend it in", so to say. This subject deserves a study of its own, but big-big-tech tends to use other things than git.
  - ongy 8mo ago
    That black hole behavior is a result of corporate processes though. Not a result of git. Business continuity (no uncontrolled external dependencies) and corporate security teams wanting to be able to scan everything. Also wanting to update everyone's dependencies when they backport something. Once you got those requirements, most of the benefits of multi-repo / roundtripping over releases just don't hold anymore. The entanglement can be stronger, but if teams build clean APIs it's no harder than removing it from a cluster of individual repositories. That might be a pretty load bearing if though.
  - rbsmith 8mo ago
    > there is no way to separate it There is git subtree --help May get complicated at the edges, as files move across boundaries. It's a hard problem. And for some subset of those problems, subtree does give a way to split and join.
- PunchyHamster 8mo ago
  The author doesn't know how to use git or how git works. If he knew how to use it, he'd be annoyed at some edge cases. If he knew how it works, he'd know the storage subsystem is flexible enough to implement any kind of new VCS on top of it. The storage format doesn't need to change to improve/replace the user facing part
panstromek 8mo ago
I think agree (but I think I think about this maybe a one level higher). I wrote about this a while ago in https://yoyo-code.com/programming-breakthroughs-we-need/#editing-code-in-general-doesn-t-work-well https://yoyo-code.com/programming-breakthroughs-we-need/#edi... . One interesting thing I got in replies is Unison language (content adressed functions, function is defined by AST). Also, I recommend checking Dion language demo (experimental project which stores program as AST). In general I think there's a missing piece between text and storage. Structural editing is likely a dead end, writing text seems superior, but storage format as text is just fundamentally problematic. I think we need a good bridge that allows editing via text, but storage like structured database (I'd go as far as say relational database, maybe). This would unlock a lot of IDE-like features for simple programmatic usage, or manipulating langauge semantics in some interesting ways, but challenge is of course how to keep the mapping between textual input in shape.
- conartist6 8mo ago
  Come the BABLR side. We have cookies! In all seriousness this is being done. By me. I would say structural editing is not a dead end, because as you mention projects like Unison and Smalltalk show us that storing structures is compatible with having syntax. The real problem is that we need a common way of storing parse tree structures so that we can build a semantic editor that works on the syntax of many programming languages
  - panstromek 8mo ago
    I think neither Unison nor Smalltalk use structural editing, though. [edit] on the level of a code in a function at least.
    - conartist6 8mo ago
      No, I know that. But we do have an example of something that does: the web browser.
- fuhsnn 8mo ago
  Structural diff tools like difftastic[1] is a good middle ground and still underexplored IMO. [1] https://github.com/Wilfred/difftastic https://github.com/Wilfred/difftastic
PunchyHamster 8mo ago
No we don't. And you can build near any VCS of your dream while still using Git as storage backend, as it is database of a linked snapshots + metadata. Bonus benefit: it will work with existing tooling The whole article is "I don't know how git works, let's make something from scratch"
- conartist6 8mo ago
  A syntax tree node does not fit into a git object. Too many children. This doesn't mean we shouldn't keep everything that's great about git in a next-gen solution, but it does mean that we'll have to build some new tools to experiment with features like semantic patching and merging. Also I checked the author out and can confirm that they know how git works in detail.
  - ongy 8mo ago
    Why do you think it has too many children? If we are talking direct descendents, I have seen way larger directories in file systems (git managed) than I've ever seen in an AST. I don't think there's a limit in git. The structure might be a bit deep for git and thus some things might be unoptimized, but the shape is the same. Tree.
    - conartist6 8mo ago
      Directories use the `tree` object type in git whereas files use `blob`. What I understand you to suggest is using the tree nodes instead of the blob nodes as the primary type of data. This is an interesting idea for how to reuse more of git's infrastructure, but it wouldn't be backwards compatible in the traditional sense either. If you checked out the contents of that repo you'd get every node in the syntax tree as a file, and let's just say that syntax nodes as directories aren't going to be compatible with any existing tools. But even if I wanted to embrace it I still think I'd hit problems with the assumptions baked into the `tree` object type in git. Directories use a fundamentally different model than syntax trees do. Directories tend to look like `<Parent><Child/></>` while syntax trees tend to look like `<Person> child: <Person /> </>`. There's no room in git's `tree` objects to put the extra information you need, and eventually the exercise would just start to feel like putting a square peg in a round hole. Instead of learning that I should use exactly git's data structure to preserve compatibility, I think my learning should be that a successful structure needs to be well-suited to the purpose it is being used for.
gfody 8mo ago
I've had this idea too, and think about it everytime I'm on a PR with lots of whitespace/non-functional noise how nice it would be if source code wern't just text and I could be looking at a cleaner higher level diff instead.. I think you have to go higher than AST though, it should at least be language-aware
- gritzko 8mo ago
  (Author) In my current codebase, I preserve the whitespace nodes. Whitespace changes would not affect the other nodes though. My first attempt to recover whitespace algorithmically not exactly failed, but more like I was unable to verify it is OK enough. We clang-format or go fmt the entire thing anyway, and whitespace changes are mostly noise, but I did not find 100% sure approach yet.
  - gfody 8mo ago
    I think about eg the "using" section at the top of a .cs file where order doesn't matter and it's common for folks to use the "Remove and Sort Usings" feature in VS.. if that were modeled as a set then diffs would consist only of added/removed items and a re-ordering wouldn't even be representable. And then every other manner of refactor that noises up a PR: renaming stuff, moving code around, etc. in my fantasies some perfect high-level model would separate everything that matters from everything that doesn't and when viewing PRs or change history we could tick "ignore superficial changes" to cut thru all the noise when looking for something specific ..to my mind such a thing could only be language-specific and the model for C# is probably something similar to Roslyn's interior (it keeps "trivia" nodes separate but still models the using section as a list for some reason) and having it all in a queryable database would be glorious for change analysis
- zelphirkalt 8mo ago
  Some languages are unfortunately whitespace sensitive, so a generic VCS cannot discard whitespace at all. But maybe the diffing tools themselves could be made language aware and hide not meaningful changes.
deafpolygon 8mo ago
No, we don’t. The genius of git is in its simplicity (while also being pretty damn complicated too).
OhNoNotAgain_99 8mo ago
[dead]
whazor 8mo ago
A big challenge will be the unfamiliarity of such a new system. Many people have found that coding agents work really well with terminals, unix tooling, and file systems. It's proven tech. Where-as doing DB queries to navigate code would be quite unfamiliar.
DannyBee 8mo ago
Somebody call the visualage and clearcase folks and let them know their time has come again!
- gritzko 8mo ago
  https://wiki.c2.com/?VisualAge https://wiki.c2.com/?VisualAge
- guerrilla 8mo ago
  > clear case Please no.
  - DannyBee 8mo ago
    What, you dont like 6 hour code checkouts of 10 megabyte projects? All you had to do was access it over VPN and you could go on vacation because nobody would expect you to get anything done!
    - guerrilla 8mo ago
      That's what I remember most about it: how much it lowered expectations on us. We barely ever worked because most of the time we couldn't. It was always broken (not just slow). This became a part of the culture where management didn't even care what we did. They knew eventually things would get done (late) but we'd also have to wait. That was just part of it.
      - DannyBee 8mo ago
        I agree. I will say the slowness i could kind of understand - just poor network protocol optimization/etc on their part, etc. Not that i excused it, but i at least could understand how you get there - you are always on a fast network and so you just don't notice what it's like when you aren't. The brokeness was always what got me. I had to believe either they had no unit tests, or 1000's of them were failing and they released it anyway. Because it was so fragile that it would have been impossible to test it and not notice easily broken things.
stared 8mo ago
To me, git works. And LLMs understand that, unlike some yet-to-come-tool. If you create a new tool for version control, go for it. Then see how it fares in benchmarks (for end-to-end tools) or vox populi - if people use you new tool/skill/workflow.
procaryote 8mo ago
One fundamental deal-breaking problem with structure aware version control is that your vcs now needs to know all the versions of all the languages you're writing in. It gets non-trivial fast.
- conartist6 8mo ago
  It does! So extensible parser definition is another key piece of the technological puzzle along with common formats for parse trees.
hallh 8mo ago
We've tackled this problem slightly differently where I work. We have AI agents contribute in a large legacy codebase, and without proper guidance, the agents quickly get lost or reimplement existing functionality. To help the agents understand the codebase, we indexed our code into a graph database using an AST, allowing the agent to easily find linked pages, features, databases, tests, etc from any one point in the code, which helped it produce much more accurate plans with less human intervention and guidance. This is combined with semantic search, where we've indexed the code based on our application's terminology, so when an agent is asked to investigate a task or bug for a specific feature, it'll find the place in the code that implements that feature, and can navigate the graph of dependencies from there to get the big picture. We provide these tools to the coding agents via MCP and it has worked really well for us. Devs and QAs can find the blast radius of bugs and critical changes very quickly, and the first draft quality of AI generated plans requires much less feedback and corrections for larger changes. In our case, I doubt that a general purpose AST would work as well. It might be better than a simple grep, especially for indirect dependencies or relationships. But IMO, it'd be far more interesting to start looking at application frameworks or even programming languages that provide this direct traversability out of the box. I remember when reading about Wasp[0] that I thought it would be interesting to see it go this way, and provide tooling specifically for AI agents. [0] https://wasp.sh/ https://wasp.sh/
- SOLAR_FIELDS 8mo ago
  I’m currently experimenting with something similar on a smaller scale using continue.dev’s code indexing implementation to expose a context mcp server for both semantic and code search. Tricky part is of course context management.
- andout_ 8mo ago
  This is close to what we're doing with [Encore](https://encore.cloud https://encore.cloud). The framework parses your application code through static analysis at compile time to build a full graph of services, APIs, databases, queues, cron jobs, and their dependencies. It uses that graph to provision infrastructure, generate architecture diagrams, API docs, and wire up observability automatically. The interesting side effect is that AI tools get this traversability for free. When business logic and infrastructure declarations live in the same code, an AI agent doesn't need a separate graph database or MCP tool to understand what a service depends on or what infrastructure it needs. It's all in the type signatures. The agent generates standard TypeScript or Go, and the framework handles everything from there to running in production. Our users see this work really well with AI agents as the agent can scaffold a complete service with databases and pub/sub, and it's deployable immediately because the framework already understands what the code needs.
moezd 8mo ago
Git works universally as a storage backend, with some reasoning capacity thanks to commit history. It didn't need to include semantics about the code or build a tree of knowledge. That would be against Linux philosophy: Do one thing and do it well. You can build whatever you want on top to help your AI agents. That would be actually beneficial so that we stop feeding raw text to this insane machinery for once.
WorldMaker 8mo ago
A CRDT-like approach to version control is usable today in darcs and pijul. Neither of them currently have a database for backend, though, but exploring both as alternatives to git and alternative's to git's ideas/dominance is still a handy idea.
yencabulator 8mo ago
This comes across as a shallow understanding of git, with the author very eager bash it instead of understanding it. > On top of that, git is not syntax-aware, so false conflicts are pretty common. If you have a syntax-aware merge tool, you can tell git to use it. Git does not bundle such for all the languages in the world, or force the user into a specific language (as the author seems to intend to do). > Fundamentally, git merges are an act of will, they are not deterministic. They are deterministic, though? It seems author is confused about the fact that humans can add edits on top to resolve conflicts. > [Part II] With remote branches different from local branches They're both just refs. > staging and stash different from commits, plus the worktree Jj unifies all these while being "just git" underneath (for most users). > GET //branch2 switching the branch; GET with side effects?
- yencabulator 8mo ago
  *eager to bash git

95 points by gritzko 8mo ago

I’ve recently been thinking about this too, here is my idea: https://clintonboys.com/projects/lit/ https://clintonboys.com/projects/lit/

gethly 8mo ago
> Like everyone else with a keyboard, a brain and a vested interest in their future as a programmer, I’ve been using AI agents to write a lot of code recently. alrighty then..
fragmede 8mo ago
Fascinating! https://github.com/clintonboys/lit-demo-crud https://github.com/clintonboys/lit-demo-crud isn't deep enough to really get a feel for how well this would work in practice.
ferroman 8mo ago
Using prompt as source of truth is not reliable, because outputs for the same prompt may vary
- mtsolitary 8mo ago
  Of course. But if this is the direction we’re going in, we need to try and do something no? Did you read the post I linked?

sublinear 8mo ago

Missing 4 out of 5 parts. The attempt to wrest control away from the open source community doesn't even have the effort to keep going.

gritzko 8mo ago
Yes, parts will keep coming, this weekend and the next week.
bonzini 8mo ago
The gist was created 1 hour before your comment.

forrestthewoods 8mo ago

Definitely agree that Git is mediocre-at-best VCS tool. This has always been the case. But LLMs are finally forcing the issue. It’s a shame a whole generation programmers has only used Git/GitHub and think it’s good. Monorepo and large binary file support is TheWay. A good cross-platform virtual file system (VFS) is necessary; a good open source one doesn’t exist today. Ideally it comes with a copy-on-write system for cross-repo blob caching. But I suppose that’s optional. It lets you commit toolchains for open source projects which is a dream of mine. Not sure I agree that LSP like features need to be built in. That feels wrong. That’s just a layer on top. Do think that agent prompts/plans/summaries need to be a first class part of commits/merges. Not sure the full set of features required here.

JodieBenitez 8mo ago
> It’s a shame a whole generation programmers has only used Git/GitHub and think it’s good. Well... I used SVN before that and it was way worse.
- EliRivers 8mo ago
  Continuing the theme, a new starter at my place (with about a decade of various experience, including an international financial services information player whose name is well known) had never used git or indeed any distributed, modern source control system. HN is a tiny bubble. The majority of the world's software engineers are barely using source control, don't do code reviews, don't have continuous build systems, don't have configuration controlled release versions, don't do almost anything that most of HN's visitors think are the basic table stakes just to conduct software engineering.
- ryanm101 8mo ago
  Clearly the poster has never encountered cvs or cvs-next. And clear OP hasn't heard of vss..
  - fragmede 8mo ago
    RCS, anyone?

solarized 8mo ago

Talk is cheap. Show me the code.

nylonstrung 8mo ago

Trustfall seems really promising for querying files as if they were a db https://github.com/obi1kenobi/trustfall https://github.com/obi1kenobi/trustfall

hole_in_foot 8mo ago

[dead]

charcircuit 8mo ago

>I definitely reject the "git compatible" approach If your version control system is not compatible with GitHub it will be dead on arrival. The value of allowing people to gradually adopt a new solution can not be understated. There is also value in being compatible with existing git integrations or scripts in projects build systems.

quadrifoliate 8mo ago
Based on reading this, I don't see anything that would prevent keeping track of a repo tracked by this database with Git (and therefore GitHub) in addition to the database. I think the "compatible" bit means more that you have to think in terms of Git concepts everywhere. Curious what the author thinks though, looks like it's posted by them.
- gritzko 8mo ago
  Technically, exporting changes either way is not a challenge. It only becomes difficult if we have multiple gateways for some reason. One way to do it is to use the new system for the messy part and git/GitHub for "publication".
conartist6 8mo ago
A system as described could be forwards compatible with git without being backwards compatible with git. In other words let people migrate easily, but don't force the new system to have all the same flaws of the old
- ongy 8mo ago
  What issues do you see in git's data model to abandon it as wire format for syncing?
  - conartist6 8mo ago
    I wouldn't say I want to abandon anything git is doing as much as evolve it. Objects need to be able to contain syntax tree nodes, and patches need to be able to target changes to particular locations in a syntax tree instead of just by line/col.

deleted 8mo ago

[deleted]

benrutter 8mo ago

> The monorepo problem: git has difficulty dividing the codebase into modules and joining them back Can anyone explain this one? I use monorepos everyday and although tools like precommit can get a bit messy, I've never found git itself to be the issue?

gritzko 8mo ago
Based on my personal experience, big-corp monorepos have all features of a black hole: they try to suck in all the existing code (vendor it) and once some code starts living in a monorepo, there is no way to separate it as it becomes entangled with the entire thing. There is no way to "blend it in", so to say. This subject deserves a study of its own, but big-big-tech tends to use other things than git.
- ongy 8mo ago
  That black hole behavior is a result of corporate processes though. Not a result of git. Business continuity (no uncontrolled external dependencies) and corporate security teams wanting to be able to scan everything. Also wanting to update everyone's dependencies when they backport something. Once you got those requirements, most of the benefits of multi-repo / roundtripping over releases just don't hold anymore. The entanglement can be stronger, but if teams build clean APIs it's no harder than removing it from a cluster of individual repositories. That might be a pretty load bearing if though.
- rbsmith 8mo ago
  > there is no way to separate it There is git subtree --help May get complicated at the edges, as files move across boundaries. It's a hard problem. And for some subset of those problems, subtree does give a way to split and join.
PunchyHamster 8mo ago
The author doesn't know how to use git or how git works. If he knew how to use it, he'd be annoyed at some edge cases. If he knew how it works, he'd know the storage subsystem is flexible enough to implement any kind of new VCS on top of it. The storage format doesn't need to change to improve/replace the user facing part

panstromek 8mo ago

I think agree (but I think I think about this maybe a one level higher). I wrote about this a while ago in https://yoyo-code.com/programming-breakthroughs-we-need/#editing-code-in-general-doesn-t-work-well https://yoyo-code.com/programming-breakthroughs-we-need/#edi... . One interesting thing I got in replies is Unison language (content adressed functions, function is defined by AST). Also, I recommend checking Dion language demo (experimental project which stores program as AST). In general I think there's a missing piece between text and storage. Structural editing is likely a dead end, writing text seems superior, but storage format as text is just fundamentally problematic. I think we need a good bridge that allows editing via text, but storage like structured database (I'd go as far as say relational database, maybe). This would unlock a lot of IDE-like features for simple programmatic usage, or manipulating langauge semantics in some interesting ways, but challenge is of course how to keep the mapping between textual input in shape.

conartist6 8mo ago
Come the BABLR side. We have cookies! In all seriousness this is being done. By me. I would say structural editing is not a dead end, because as you mention projects like Unison and Smalltalk show us that storing structures is compatible with having syntax. The real problem is that we need a common way of storing parse tree structures so that we can build a semantic editor that works on the syntax of many programming languages
- panstromek 8mo ago
  I think neither Unison nor Smalltalk use structural editing, though. [edit] on the level of a code in a function at least.
  - conartist6 8mo ago
    No, I know that. But we do have an example of something that does: the web browser.
fuhsnn 8mo ago
Structural diff tools like difftastic[1] is a good middle ground and still underexplored IMO. [1] https://github.com/Wilfred/difftastic https://github.com/Wilfred/difftastic

PunchyHamster 8mo ago

No we don't. And you can build near any VCS of your dream while still using Git as storage backend, as it is database of a linked snapshots + metadata. Bonus benefit: it will work with existing tooling The whole article is "I don't know how git works, let's make something from scratch"

conartist6 8mo ago
A syntax tree node does not fit into a git object. Too many children. This doesn't mean we shouldn't keep everything that's great about git in a next-gen solution, but it does mean that we'll have to build some new tools to experiment with features like semantic patching and merging. Also I checked the author out and can confirm that they know how git works in detail.
- ongy 8mo ago
  Why do you think it has too many children? If we are talking direct descendents, I have seen way larger directories in file systems (git managed) than I've ever seen in an AST. I don't think there's a limit in git. The structure might be a bit deep for git and thus some things might be unoptimized, but the shape is the same. Tree.
  - conartist6 8mo ago
    Directories use the `tree` object type in git whereas files use `blob`. What I understand you to suggest is using the tree nodes instead of the blob nodes as the primary type of data. This is an interesting idea for how to reuse more of git's infrastructure, but it wouldn't be backwards compatible in the traditional sense either. If you checked out the contents of that repo you'd get every node in the syntax tree as a file, and let's just say that syntax nodes as directories aren't going to be compatible with any existing tools. But even if I wanted to embrace it I still think I'd hit problems with the assumptions baked into the `tree` object type in git. Directories use a fundamentally different model than syntax trees do. Directories tend to look like `<Parent><Child/></>` while syntax trees tend to look like `<Person> child: <Person /> </>`. There's no room in git's `tree` objects to put the extra information you need, and eventually the exercise would just start to feel like putting a square peg in a round hole. Instead of learning that I should use exactly git's data structure to preserve compatibility, I think my learning should be that a successful structure needs to be well-suited to the purpose it is being used for.

gfody 8mo ago

I've had this idea too, and think about it everytime I'm on a PR with lots of whitespace/non-functional noise how nice it would be if source code wern't just text and I could be looking at a cleaner higher level diff instead.. I think you have to go higher than AST though, it should at least be language-aware

gritzko 8mo ago
(Author) In my current codebase, I preserve the whitespace nodes. Whitespace changes would not affect the other nodes though. My first attempt to recover whitespace algorithmically not exactly failed, but more like I was unable to verify it is OK enough. We clang-format or go fmt the entire thing anyway, and whitespace changes are mostly noise, but I did not find 100% sure approach yet.
- gfody 8mo ago
  I think about eg the "using" section at the top of a .cs file where order doesn't matter and it's common for folks to use the "Remove and Sort Usings" feature in VS.. if that were modeled as a set then diffs would consist only of added/removed items and a re-ordering wouldn't even be representable. And then every other manner of refactor that noises up a PR: renaming stuff, moving code around, etc. in my fantasies some perfect high-level model would separate everything that matters from everything that doesn't and when viewing PRs or change history we could tick "ignore superficial changes" to cut thru all the noise when looking for something specific ..to my mind such a thing could only be language-specific and the model for C# is probably something similar to Roslyn's interior (it keeps "trivia" nodes separate but still models the using section as a list for some reason) and having it all in a queryable database would be glorious for change analysis
zelphirkalt 8mo ago
Some languages are unfortunately whitespace sensitive, so a generic VCS cannot discard whitespace at all. But maybe the diffing tools themselves could be made language aware and hide not meaningful changes.

deafpolygon 8mo ago

No, we don’t. The genius of git is in its simplicity (while also being pretty damn complicated too).

OhNoNotAgain_99 8mo ago

[dead]

whazor 8mo ago

A big challenge will be the unfamiliarity of such a new system. Many people have found that coding agents work really well with terminals, unix tooling, and file systems. It's proven tech. Where-as doing DB queries to navigate code would be quite unfamiliar.

DannyBee 8mo ago

Somebody call the visualage and clearcase folks and let them know their time has come again!

gritzko 8mo ago
https://wiki.c2.com/?VisualAge https://wiki.c2.com/?VisualAge
guerrilla 8mo ago
> clear case Please no.
- DannyBee 8mo ago
  What, you dont like 6 hour code checkouts of 10 megabyte projects? All you had to do was access it over VPN and you could go on vacation because nobody would expect you to get anything done!
  - guerrilla 8mo ago
    That's what I remember most about it: how much it lowered expectations on us. We barely ever worked because most of the time we couldn't. It was always broken (not just slow). This became a part of the culture where management didn't even care what we did. They knew eventually things would get done (late) but we'd also have to wait. That was just part of it.
    - DannyBee 8mo ago
      I agree. I will say the slowness i could kind of understand - just poor network protocol optimization/etc on their part, etc. Not that i excused it, but i at least could understand how you get there - you are always on a fast network and so you just don't notice what it's like when you aren't. The brokeness was always what got me. I had to believe either they had no unit tests, or 1000's of them were failing and they released it anyway. Because it was so fragile that it would have been impossible to test it and not notice easily broken things.

stared 8mo ago

To me, git works. And LLMs understand that, unlike some yet-to-come-tool. If you create a new tool for version control, go for it. Then see how it fares in benchmarks (for end-to-end tools) or vox populi - if people use you new tool/skill/workflow.

procaryote 8mo ago

One fundamental deal-breaking problem with structure aware version control is that your vcs now needs to know all the versions of all the languages you're writing in. It gets non-trivial fast.

conartist6 8mo ago
It does! So extensible parser definition is another key piece of the technological puzzle along with common formats for parse trees.

hallh 8mo ago

We've tackled this problem slightly differently where I work. We have AI agents contribute in a large legacy codebase, and without proper guidance, the agents quickly get lost or reimplement existing functionality. To help the agents understand the codebase, we indexed our code into a graph database using an AST, allowing the agent to easily find linked pages, features, databases, tests, etc from any one point in the code, which helped it produce much more accurate plans with less human intervention and guidance. This is combined with semantic search, where we've indexed the code based on our application's terminology, so when an agent is asked to investigate a task or bug for a specific feature, it'll find the place in the code that implements that feature, and can navigate the graph of dependencies from there to get the big picture. We provide these tools to the coding agents via MCP and it has worked really well for us. Devs and QAs can find the blast radius of bugs and critical changes very quickly, and the first draft quality of AI generated plans requires much less feedback and corrections for larger changes. In our case, I doubt that a general purpose AST would work as well. It might be better than a simple grep, especially for indirect dependencies or relationships. But IMO, it'd be far more interesting to start looking at application frameworks or even programming languages that provide this direct traversability out of the box. I remember when reading about Wasp[0] that I thought it would be interesting to see it go this way, and provide tooling specifically for AI agents. [0] https://wasp.sh/ https://wasp.sh/

SOLAR_FIELDS 8mo ago
I’m currently experimenting with something similar on a smaller scale using continue.dev’s code indexing implementation to expose a context mcp server for both semantic and code search. Tricky part is of course context management.
andout_ 8mo ago
This is close to what we're doing with [Encore](https://encore.cloud https://encore.cloud). The framework parses your application code through static analysis at compile time to build a full graph of services, APIs, databases, queues, cron jobs, and their dependencies. It uses that graph to provision infrastructure, generate architecture diagrams, API docs, and wire up observability automatically. The interesting side effect is that AI tools get this traversability for free. When business logic and infrastructure declarations live in the same code, an AI agent doesn't need a separate graph database or MCP tool to understand what a service depends on or what infrastructure it needs. It's all in the type signatures. The agent generates standard TypeScript or Go, and the framework handles everything from there to running in production. Our users see this work really well with AI agents as the agent can scaffold a complete service with databases and pub/sub, and it's deployable immediately because the framework already understands what the code needs.

moezd 8mo ago

Git works universally as a storage backend, with some reasoning capacity thanks to commit history. It didn't need to include semantics about the code or build a tree of knowledge. That would be against Linux philosophy: Do one thing and do it well. You can build whatever you want on top to help your AI agents. That would be actually beneficial so that we stop feeding raw text to this insane machinery for once.

WorldMaker 8mo ago

A CRDT-like approach to version control is usable today in darcs and pijul. Neither of them currently have a database for backend, though, but exploring both as alternatives to git and alternative's to git's ideas/dominance is still a handy idea.

yencabulator 8mo ago

This comes across as a shallow understanding of git, with the author very eager bash it instead of understanding it. > On top of that, git is not syntax-aware, so false conflicts are pretty common. If you have a syntax-aware merge tool, you can tell git to use it. Git does not bundle such for all the languages in the world, or force the user into a specific language (as the author seems to intend to do). > Fundamentally, git merges are an act of will, they are not deterministic. They are deterministic, though? It seems author is confused about the fact that humans can add edits on top to resolve conflicts. > [Part II] With remote branches different from local branches They're both just refs. > staging and stash different from commits, plus the worktree Jj unifies all these while being "just git" underneath (for most users). > GET //branch2 switching the branch; GET with side effects?

yencabulator 8mo ago
*eager to bash git