5 ms·

Can I use this to translate one language to another?

morgante 3y ago
In theory, you could write a full transpiler in GritQL. In practice, it's not really the ideal tool for translating between languages. It does work well for cases where the differences are minor (ex. converting from JavaScript to TypeScript, or between different SQL dialects).
- ayewo 3y ago
  But Grit should be able to handle translating a PHP dialect like Hacklang to a modern version of PHP? In your opinion, what tools would be ideal for general purpose transpilation between languages?
  - morgante 3y ago
    > But Grit should be able to handle translating a PHP dialect like Hacklang to a modern version of PHP? Yeah, that should be doable (though GritQL doesn't have PHP support yet). > In your opinion, what tools would be ideal for general purpose transpilation between languages? Writing a transpiler is quite complicated and requires a lot of development to get right. I don't know of any high-quality generalized transpilers—you're best looking for tools specific to the language pair you're targeting.
- snthpy 3y ago
  I have the same question. I'm particularly interested in the SQL dialects case. Does the rewrite code also have to be syntactically correct or is that a simple text replacement because that's what I see as the main impediment; if I'm using the sqlite grammar to parse and produce postgresql output, the output won't conform to the grammar. Also how do I dynamically load custom tree-sitter grammars like in ast-grep?
  - morgante 3y ago
    > Does the rewrite code also have to be syntactically correct or is that a simple text replacement because that's what I see as the main impediment; if I'm using the sqlite grammar to parse and produce postgresql output, the output won't conform to the grammar. No, we don't require the output code to be syntactically correct. You can use the raw`code` modified to indicate a raw text replacement: https://docs.grit.io/language/patterns#raw-output https://docs.grit.io/language/patterns#raw-output > Also how do I dynamically load custom tree-sitter grammars like in ast-grep? We don't currently support dynamically loading grammars, mainly because our metavariable approach requires some modifications to the grammar.
    - snthpy 3y ago
      Thank you. The raw"..." directive looks perfect for what I'm looking for.
- snthpy 3y ago
  @morgante Are you aware of any tools that do codegen from tree-sitter ASTs? A simple application would be to produce a code formatter like Black or ruff for python but which could support any tree-sitter supported language. More specifically to my use case though, if I produced tree-sitter grammars with identical node structure and node names, e.g. for sqlite and postgresql, then wouldn't this make it simple to do full transpilation between the two?
  - morgante 3y ago
    You should check out https://github.com/tweag/topiary https://github.com/tweag/topiary Yes, theoretically if you had ~identical grammars you could use it to do a full transpilation. There's a lot of challenges with that though. Writing a correct grammar for 1 language is complicated enough, but writing one for two where all your nodes and fields end up the same is likely insurmountable. In practice, languages are either: - Far enough apart that any pure AST transformation is insufficient and you need an AI component to produce usable output - Close enough that you're better off just targeting the specific parts that differ and rewriting those while leaving the rest alone. I think GritQL can do well here.
    - snthpy 3y ago
      Awesome, thanks. Big fan of the Tweag folks and Nix and Nickel so will definitely check it out.

In theory, you could write a full transpiler in GritQL. In practice, it's not really the ideal tool for translating between languages. It does work well for cases where the differences are minor (ex. converting from JavaScript to TypeScript, or between different SQL dialects).

ayewo 3y ago
But Grit should be able to handle translating a PHP dialect like Hacklang to a modern version of PHP? In your opinion, what tools would be ideal for general purpose transpilation between languages?
- morgante 3y ago
  > But Grit should be able to handle translating a PHP dialect like Hacklang to a modern version of PHP? Yeah, that should be doable (though GritQL doesn't have PHP support yet). > In your opinion, what tools would be ideal for general purpose transpilation between languages? Writing a transpiler is quite complicated and requires a lot of development to get right. I don't know of any high-quality generalized transpilers—you're best looking for tools specific to the language pair you're targeting.
snthpy 3y ago
I have the same question. I'm particularly interested in the SQL dialects case. Does the rewrite code also have to be syntactically correct or is that a simple text replacement because that's what I see as the main impediment; if I'm using the sqlite grammar to parse and produce postgresql output, the output won't conform to the grammar. Also how do I dynamically load custom tree-sitter grammars like in ast-grep?
- morgante 3y ago
  > Does the rewrite code also have to be syntactically correct or is that a simple text replacement because that's what I see as the main impediment; if I'm using the sqlite grammar to parse and produce postgresql output, the output won't conform to the grammar. No, we don't require the output code to be syntactically correct. You can use the raw`code` modified to indicate a raw text replacement: https://docs.grit.io/language/patterns#raw-output https://docs.grit.io/language/patterns#raw-output > Also how do I dynamically load custom tree-sitter grammars like in ast-grep? We don't currently support dynamically loading grammars, mainly because our metavariable approach requires some modifications to the grammar.
  - snthpy 3y ago
    Thank you. The raw"..." directive looks perfect for what I'm looking for.
snthpy 3y ago
@morgante Are you aware of any tools that do codegen from tree-sitter ASTs? A simple application would be to produce a code formatter like Black or ruff for python but which could support any tree-sitter supported language. More specifically to my use case though, if I produced tree-sitter grammars with identical node structure and node names, e.g. for sqlite and postgresql, then wouldn't this make it simple to do full transpilation between the two?
- morgante 3y ago
  You should check out https://github.com/tweag/topiary https://github.com/tweag/topiary Yes, theoretically if you had ~identical grammars you could use it to do a full transpilation. There's a lot of challenges with that though. Writing a correct grammar for 1 language is complicated enough, but writing one for two where all your nodes and fields end up the same is likely insurmountable. In practice, languages are either: - Far enough apart that any pure AST transformation is insufficient and you need an AI component to produce usable output - Close enough that you're better off just targeting the specific parts that differ and rewriting those while leaving the rest alone. I think GritQL can do well here.
  - snthpy 3y ago
    Awesome, thanks. Big fan of the Tweag folks and Nix and Nickel so will definitely check it out.