3 ms·

mmm. interesting and fun concept, but it seems to me like the text is actually the right layer for storing and expressing changes since that is what gets read,

by a-dub 7mo ago

mmm. interesting and fun concept, but it seems to me like the text is actually the right layer for storing and expressing changes since that is what gets read, changed and reasoned about. why does it make more sense to use asts here? are these asts fully normalized or do (x) and ((x)) produce different trees, yet still express the same thing? why change what is being stored and tracked when the language aware metadata for each change can be generated after the fact (or alongside the changes)? (adding transform layers between what appears and what gets stored/tracked seems like it could get confusing?)

deleted 7mo ago
[deleted]
bri3d 7mo ago
100% agree. I think AST-driven tooling is very valuable (most big companies have internal tools akin to each operation Beagle provides, and Linux have Coccinelle / Spatch for example), but it's still just easier implemented as a layer on top of source code than the fundamental source of truth. There are some clever things that can be done with merge/split using CRDTs as the stored transformation, but they're hard to reason about compared to just semantic merge tools, and don't outweigh the cognitive overhead IMO. Having worked for many years with programming systems which were natively expressed as trees - often just operation trees and object graphs, discarding the notion of syntax completely, this layer is incredibly difficult for humans to reason about, especially when it comes to diffs, and usually at the end you end up having to build a system which can produce and act upon text-based diffs anyway. I think there's some notion of these kinds of revision management tools being useful for an LLM, but again, at that point you might as well run them aside (just perform the source -> AST transformation at each commit) rather than use them as the core storage.
- ragall 7mo ago
  > but it's still just easier implemented as a layer on top of source code than the fundamental source of truth Easier but much less valuable.
  - a-dub 7mo ago
    you can parse the text at any time pretty much for free and use anything you learn to be smarter about manipulating the text. you can literally replace the default diff program with one that parses the source files to do a better job today.
    - gritzko 7mo ago
      CRDT's trick is metadata. Good old diff guesses the changes by solving the longest-common-subsequence problem. There is always some degree of confusion as changes accumulate. CRDTs can know the exact changes, or at least guess less.
    - derriz 7mo ago
      This is the fundamental idea behind git - to fully compute/derive diffs from snapshots (commits) and to only store snapshots. While brilliant in some ways - particularly the simplifications it allows in terms of implementation, I’ve always felt that dropping all information about how a new commit was derived from its parent(s) was wasteful. There have been a number of occasions where I wished that git recorded a rename/mv somehow - it’s particularly annoying when you squash some commits and suddenly it no longer recognizes that a file was renamed where previously it was able to determine this. Now your history is broken - “git blame” fails to provide useful information, etc. There are other ways of storing history and revisions which don’t have this issue - git isn’t the end of the line in terms of version control evolution.
      - bri3d 7mo ago
        I agree with this, I just don't think I agree with the Beagle approach (CRDT on AST as the source of truth) vs. the Git method (bytewise files as the source of truth) with something alongside. Like, I think it's way easier to add a parallel construction to Git (via a formal method or even a magic file) which includes the CRDT for the AST than it is to make that the base unit of truth. It still lets you answer and interact at the higher level with "oh, this commit changed $SYM1 to $SYM2" without also destroying byte-level file information that someone finds important, and without changing the main abstraction from the human-space to the computer-space.
ibejoeb 7mo ago
> why does it make more sense to use asts here For one, it eliminates a class of merge conflict that arises strictly from text formatting. I always liked the idea of storing code in abstraction, especially editors supported edit-time formatting. I enjoy working on other people's code, but I don't think anybody likes the tedium of complying with style guides, especially ones that are enforced at the SCM level, which adds friction to creating even local, temporary revisions. This kind of thing would obviate that. That's why I also appreciate strict and deterministic systems like rustfmt. Unison goes a little further, which is neat but I think they're struggling getting adoption because of that, even though I'm pretty sure they've got some better tooling for working outside the whole ecosystem. These decoupled tools are probably a good way to go. I was messing around with a file-less paradigm that would present a source tree in arbitrary ways, like just showing a individual functions, so you have the things you're working on co-located rather than switching between files. Kind of like the old VB IDE.
- psadri 7mo ago
  Related, I’d love an editor that’d let me view/edit identifier names in snake_case and save them as camelCase on disk. If anyone knows of such a thing - please let me know!
  - ibejoeb 7mo ago
    Sure. Presumably you could have localized source presentation, too. But, yeah, I think a personalized development environment with all of your preferences preserved and that don't interfere with whatever the upstream standard is would be a nice upgrade.
  - Wilfred 7mo ago
    This is actually possible with glasses-mode in Emacs: https://codelearn.me/2025/02/24/emacs-glasses-mode.html https://codelearn.me/2025/02/24/emacs-glasses-mode.html I think it sees very little usage though.
- micw 7mo ago
  An AST based conflict resolver could eliminate the same kind of merge conflicts on a text based RCS
CuriouslyC 7mo ago
One nice thing about serializing/transmitting AST changes is that it makes it much easier to to compose and transform change sets. The text based diff method works fine if everyone is working off a head, but when you're trying to compose a release from a lot of branches it's usually a huge mess. Text based diffs also make maintaining forks harder. Git is going to become a big bottleneck as agents get better.
- a-dub 7mo ago
  what do you actually gain over enforced formatting? first you should not be composing releases at the end from conflicting branches, you should be integrating branches and testing each one in sequence and then cutting releases. if there are changes to the base for a given branch, that means that branch has to be updated and re-tested. simple as that. storing changes as normalized trees rather than normalized text doesn't really buy you anything except for maybe slightly smarter automatic merge conflict resolution but even then it needs to be analyzed and tested.
  - CuriouslyC 7mo ago
    Diffs are fragile, and while I agree with that process in a world where humans do all the work and you aren't cutting a dozen different releases, I think that's a world we're rapidly moving away from.
    - a-dub 7mo ago
      in that case you probably flag a bunch of prs for release and it linearizes their order and rebases and tests each one a step ahead of your review (responding to any changes you make as you go).
sse 7mo ago
Having a VCS that stores changes as refactorings combined with an editor that reports the refactorings directly to the VCS, without plain text files as intermediate format, would avoid losing information on the way. The downside is tight coupling between VCS and editor. It will be difficult to convince developers to use anything else than their favourite editor when they want to use your VCS. I wonder if you can solve it the language-server way, so that each editor that supports refactoring through language-server would support the VCS.

by a-dub 7mo ago

deleted 7mo ago

[deleted]

bri3d 7mo ago

100% agree. I think AST-driven tooling is very valuable (most big companies have internal tools akin to each operation Beagle provides, and Linux have Coccinelle / Spatch for example), but it's still just easier implemented as a layer on top of source code than the fundamental source of truth. There are some clever things that can be done with merge/split using CRDTs as the stored transformation, but they're hard to reason about compared to just semantic merge tools, and don't outweigh the cognitive overhead IMO. Having worked for many years with programming systems which were natively expressed as trees - often just operation trees and object graphs, discarding the notion of syntax completely, this layer is incredibly difficult for humans to reason about, especially when it comes to diffs, and usually at the end you end up having to build a system which can produce and act upon text-based diffs anyway. I think there's some notion of these kinds of revision management tools being useful for an LLM, but again, at that point you might as well run them aside (just perform the source -> AST transformation at each commit) rather than use them as the core storage.

ragall 7mo ago
> but it's still just easier implemented as a layer on top of source code than the fundamental source of truth Easier but much less valuable.
- a-dub 7mo ago
  you can parse the text at any time pretty much for free and use anything you learn to be smarter about manipulating the text. you can literally replace the default diff program with one that parses the source files to do a better job today.
  - gritzko 7mo ago
    CRDT's trick is metadata. Good old diff guesses the changes by solving the longest-common-subsequence problem. There is always some degree of confusion as changes accumulate. CRDTs can know the exact changes, or at least guess less.
  - derriz 7mo ago
    This is the fundamental idea behind git - to fully compute/derive diffs from snapshots (commits) and to only store snapshots. While brilliant in some ways - particularly the simplifications it allows in terms of implementation, I’ve always felt that dropping all information about how a new commit was derived from its parent(s) was wasteful. There have been a number of occasions where I wished that git recorded a rename/mv somehow - it’s particularly annoying when you squash some commits and suddenly it no longer recognizes that a file was renamed where previously it was able to determine this. Now your history is broken - “git blame” fails to provide useful information, etc. There are other ways of storing history and revisions which don’t have this issue - git isn’t the end of the line in terms of version control evolution.
    - bri3d 7mo ago
      I agree with this, I just don't think I agree with the Beagle approach (CRDT on AST as the source of truth) vs. the Git method (bytewise files as the source of truth) with something alongside. Like, I think it's way easier to add a parallel construction to Git (via a formal method or even a magic file) which includes the CRDT for the AST than it is to make that the base unit of truth. It still lets you answer and interact at the higher level with "oh, this commit changed $SYM1 to $SYM2" without also destroying byte-level file information that someone finds important, and without changing the main abstraction from the human-space to the computer-space.

ibejoeb 7mo ago

> why does it make more sense to use asts here For one, it eliminates a class of merge conflict that arises strictly from text formatting. I always liked the idea of storing code in abstraction, especially editors supported edit-time formatting. I enjoy working on other people's code, but I don't think anybody likes the tedium of complying with style guides, especially ones that are enforced at the SCM level, which adds friction to creating even local, temporary revisions. This kind of thing would obviate that. That's why I also appreciate strict and deterministic systems like rustfmt. Unison goes a little further, which is neat but I think they're struggling getting adoption because of that, even though I'm pretty sure they've got some better tooling for working outside the whole ecosystem. These decoupled tools are probably a good way to go. I was messing around with a file-less paradigm that would present a source tree in arbitrary ways, like just showing a individual functions, so you have the things you're working on co-located rather than switching between files. Kind of like the old VB IDE.

psadri 7mo ago
Related, I’d love an editor that’d let me view/edit identifier names in snake_case and save them as camelCase on disk. If anyone knows of such a thing - please let me know!
- ibejoeb 7mo ago
  Sure. Presumably you could have localized source presentation, too. But, yeah, I think a personalized development environment with all of your preferences preserved and that don't interfere with whatever the upstream standard is would be a nice upgrade.
- Wilfred 7mo ago
  This is actually possible with glasses-mode in Emacs: https://codelearn.me/2025/02/24/emacs-glasses-mode.html https://codelearn.me/2025/02/24/emacs-glasses-mode.html I think it sees very little usage though.
micw 7mo ago
An AST based conflict resolver could eliminate the same kind of merge conflicts on a text based RCS

CuriouslyC 7mo ago

One nice thing about serializing/transmitting AST changes is that it makes it much easier to to compose and transform change sets. The text based diff method works fine if everyone is working off a head, but when you're trying to compose a release from a lot of branches it's usually a huge mess. Text based diffs also make maintaining forks harder. Git is going to become a big bottleneck as agents get better.

a-dub 7mo ago
what do you actually gain over enforced formatting? first you should not be composing releases at the end from conflicting branches, you should be integrating branches and testing each one in sequence and then cutting releases. if there are changes to the base for a given branch, that means that branch has to be updated and re-tested. simple as that. storing changes as normalized trees rather than normalized text doesn't really buy you anything except for maybe slightly smarter automatic merge conflict resolution but even then it needs to be analyzed and tested.
- CuriouslyC 7mo ago
  Diffs are fragile, and while I agree with that process in a world where humans do all the work and you aren't cutting a dozen different releases, I think that's a world we're rapidly moving away from.
  - a-dub 7mo ago
    in that case you probably flag a bunch of prs for release and it linearizes their order and rebases and tests each one a step ahead of your review (responding to any changes you make as you go).

sse 7mo ago

Having a VCS that stores changes as refactorings combined with an editor that reports the refactorings directly to the VCS, without plain text files as intermediate format, would avoid losing information on the way. The downside is tight coupling between VCS and editor. It will be difficult to convince developers to use anything else than their favourite editor when they want to use your VCS. I wonder if you can solve it the language-server way, so that each editor that supports refactoring through language-server would support the VCS.