3 ms·

The linked page looks like a subsystem of some specific library, I am not sure if it is intended for general use. If it were intended to be general replacement

The linked page looks like a subsystem of some specific library, I am not sure if it is intended for general use. If it were intended to be general replacement for general purpose version control systems, I am not sure how storing AST is better than storing the original plain text files since the transformation from text to AST might be lossy. I might want to store files with no AST (e.g. plain text files), files with multiple AST (e.g. polyglots), multiple files with the same AST (e.g. files to test different code layout), broken AST (e.g. data files to be used as test cases). These use cases would be trivially supported by storing the original file as is, whereas storing any processed form of the file would require extra work.

sethev 7mo ago
It leans on tree-sitter for language handling, so i wonder if they're actually Concrete Syntax Trees.
gritzko 7mo ago
(Author) There is a fall-back general-text codec: tokens, no AST (e.g. for Markdown). If that fails (non UTF8), there is the general-blob final-fallback codec (the git mode). The way it makes an AST tree is non-lossy. Additionally, it stamps ids on the nodes, so merges do not get confused by renames, formatting changes and similar things. There is value in preserving structure this way that repeat parsing can not provide. In big-O terms, working with such an AST tree and a stack of its patches is not much different from stacks of binary diffs git is using. If I have k independent changesets, I have k^2 unplanned interactions and 2^k unplanned change combinations. Having a bunch of change sets, which I had not fully evaluated yet, esp in relation to one another, I would like k-way merges and repeat-merges to be seamless, non-intrusive and deterministic. git's merges are not. The project is experimental at this point.
- jhancock 7mo ago
  AST of what? Will it read my clojure code's forms as such? What if my source file has a paran balancing error? I feel I'm thinking of this at the wrong level/angle.
  - gritzko 7mo ago
    I cannot remember a case, in the last 10 years at least, when I committed code that does not compile. Why should I share that? Also, tree-sitter sort of handles that.
    - omoikane 7mo ago
      > code that does not compile. Why should I share that? If you collect test cases for compilers, for example. > tree-sitter sort of handles that My worry is that stability of committed ASTs would depend on tree-sitter being stable, and it might be difficult to guarantee that for languages are still in flux. Even most well established languages gain new grammar once every few years, sometimes in backward incompatible ways. Maybe you meant tree-sitter itself will also be versioned inside this repository?
      - gritzko 7mo ago
        Tree-sitter can parse somewhat-bad code. Also, there is an option to pick a codec for a particular file. Might use tree-sitter-C, might use general-text. The only issue here, you can't change the codec and keep nice diffs. So, these cases are handled.

sethev 7mo ago

It leans on tree-sitter for language handling, so i wonder if they're actually Concrete Syntax Trees.

gritzko 7mo ago

(Author) There is a fall-back general-text codec: tokens, no AST (e.g. for Markdown). If that fails (non UTF8), there is the general-blob final-fallback codec (the git mode). The way it makes an AST tree is non-lossy. Additionally, it stamps ids on the nodes, so merges do not get confused by renames, formatting changes and similar things. There is value in preserving structure this way that repeat parsing can not provide. In big-O terms, working with such an AST tree and a stack of its patches is not much different from stacks of binary diffs git is using. If I have k independent changesets, I have k^2 unplanned interactions and 2^k unplanned change combinations. Having a bunch of change sets, which I had not fully evaluated yet, esp in relation to one another, I would like k-way merges and repeat-merges to be seamless, non-intrusive and deterministic. git's merges are not. The project is experimental at this point.

jhancock 7mo ago
AST of what? Will it read my clojure code's forms as such? What if my source file has a paran balancing error? I feel I'm thinking of this at the wrong level/angle.
- gritzko 7mo ago
  I cannot remember a case, in the last 10 years at least, when I committed code that does not compile. Why should I share that? Also, tree-sitter sort of handles that.
  - omoikane 7mo ago
    > code that does not compile. Why should I share that? If you collect test cases for compilers, for example. > tree-sitter sort of handles that My worry is that stability of committed ASTs would depend on tree-sitter being stable, and it might be difficult to guarantee that for languages are still in flux. Even most well established languages gain new grammar once every few years, sometimes in backward incompatible ways. Maybe you meant tree-sitter itself will also be versioned inside this repository?
    - gritzko 7mo ago
      Tree-sitter can parse somewhat-bad code. Also, there is an option to pick a codec for a particular file. Might use tree-sitter-C, might use general-text. The only issue here, you can't change the codec and keep nice diffs. So, these cases are handled.