4 ms·

Is it possible to losslessly translate from Hy or Python to the Abstract Syntax Tree and back? That is, is there way to have a code viewer that, instead of high

Is it possible to losslessly translate from Hy or Python to the Abstract Syntax Tree and back? That is, is there way to have a code viewer that, instead of highlighting syntax, changes it to another syntax on the fly? Could each member of a development team look at the same piece of code but the first sees Lisp syntax, the second sees Python, a third sees a C-style syntax, and a fourth has a mishmash of their own design? Obviously there are language differences that go beyond syntax, but the learning curve of reading unfamiliar syntax often seems to be a sticking point for people considering/learning a new language. For example, someone who has experience in curly-brace languages may initially have trouble adapting to the use of indentation in Python or parentheses in Lisp.

aasasd 5y ago
Lisp and Python have different semantics, even between Python and Hy: an obvious one is that Lisp's `if` and all other statements are expressions. Coming up with a syntax is not the hard part. A couple gotchas that basically all Lisps-on-other-languages have to keep in mind are: variables, since Lisps usually don't have the same scopes as anything imperative (with the ‘let’ blocks); and function names and possibly functions-as-variables, since Lisps are liberal with both. As a rule, you'd have to implement your own variable scopes and name mangling for where the semantics don't match. Plus macros are an obvious addition—IIRC Hy still has them in separate files (though I might confuse that with Fennel). And if you wish to have keywords as a separate type, that's usually entirely on you—afaik Clojure went full-in here, others not so much. > reading unfamiliar syntax often seems to be a sticking point for people considering/learning a new language Precisely the opposite of my and many other people's experience. Remembering whether blocks are delimited with braces, indentation or e.g. ‘then/end’ is a no-brainer. Learning the subtle differences in semantics, the type system, the library, package system and build environments—that's the bitch. Consider that there are dozens of even kinda-popular Lisps, despite them not having much syntax in the first place. They don't just differ in the function names. You seem to be imagining a ‘universal-ish intermediate language’, so to say, that would permit people with different experience to quickly adapt to an environment of a different language. Not possible, because a) semantics is where the difference is, you'd need a full-blown language translator, which still can be lossy or incomplete (e.g. no types in your standard Lisp); b) one would still have to deal with all the non-language parts of the environment, like the library, packages, the build system.
- asoneth 5y ago
  > You seem to be imagining a ‘universal-ish intermediate language’ To clarify, I do not mean that each developer would be using a different language -- they would all be writing "Python", they'd just using a superficially distinct syntax that each finds most pleasant/familiar/ergonomic. > Remembering whether blocks are delimited with braces, indentation or e.g. ‘then/end’ is a no-brainer. I tend to find the same, at least after an initial learning phase. In fact, I sometimes find distinct languages with similar syntax confusing since I subconsciously expect the same behavior and packages too. Having said that I've taught and TA'd Scheme, Java, Jess, and block languages like Scratch & Alice many years ago, and in those cases a significant number of students found unfamiliar syntax to be a major hurdle, especially if it was one of their first two programming languages. Also, judging by the volume of online complaints about Python's semantically meaningful indentation, Ada's array indexing, and Lisp's parentheses these kinds of conventions do seem like a concern even for experienced developers. > one would still have to deal with all the non-language parts of the environment, like the library, packages, the build system. Right, that would be the point, to be able to use a language's library/packages/build system without necessarily using its original syntax.
  - aasasd 5y ago
    Well, doing this on top of a single language is more feasible, however the Python-Lisp pair is still not the best choice due to Python's strict semantics of separation between statements and expressions: when you generate an AST from Lisp, you'd have some sorta verbose shims in place of Lisp's expression-ic statements—precisely ones that Hy now creates. And vice versa, a coder's normal Python might not map cleanly to Hy. If you're free to invent the languages in the first place, it would be kinda trivial to make several with the same semantics but different superficial syntax—basically just substitute one bunch of characters for another, no need to go via AST even. And these languages can then be transpiled to a single language of your choice, be that Python or JVM bytecode—if the semantics are made to be close enough.
    - gilch 5y ago
      What about a Lissp-Hebigo pair? https://github.com/gilch/hissp#hebigo https://github.com/gilch/hissp#hebigo Hissp takes a different approach than Hy. Where Hy has to use shims to pretend statements are expressions, Hissp just targets the expression subset in the first place. (Actually a somewhat smaller subset than that if you're not injecting any raw Python: literals, lambdas, identifiers, and calls.)
Foxboron 5y ago
There are code which takes the Python AST and attempts to produce the correct Python code. It has a few issues if i recall correctly but it should mostly^tm work. https://github.com/berkerpeksag/astor https://github.com/berkerpeksag/astor
- gilch 5y ago
  That capability is in the standard library now: https://docs.python.org/3/library/ast.html?highlight=unparse#ast.unparse https://docs.python.org/3/library/ast.html?highlight=unparse...
qubex 5y ago
Basically what you’re suggesting is a kind of AST decompiler (in an IDE front-end?) that allows developers to view and edit the code in an essentially arbitrary syntax?
- asoneth 5y ago
  Yes. They'd be using the same language, but see different syntax styles.
mark_l_watson 5y ago
If you do not use the contributed “let” macro, then auto generated Python code from Hy source code looks fine. If you look at the GitHub repo for the Hy book I wrote, you will see a Makefile target for generating Python code from the Hy examples: https://github.com/mark-watson/hy-lisp-python https://github.com/mark-watson/hy-lisp-python EDIT: not a Makefile target, I used a shell script create_python_source_from_hy.sh
gilch 5y ago
See hy2py: https://docs.hylang.org/en/master/language/cli.html#hy2py https://docs.hylang.org/en/master/language/cli.html#hy2py
gilch 5y ago
See JetBrains MPS: https://www.jetbrains.com/mps/ https://www.jetbrains.com/mps/ The "Projectional Editor" might be the kind of thing you're talking about. The meta language is stored as syntax trees rather than text and can be rendered in different ways.

aasasd 5y ago

Lisp and Python have different semantics, even between Python and Hy: an obvious one is that Lisp's `if` and all other statements are expressions. Coming up with a syntax is not the hard part. A couple gotchas that basically all Lisps-on-other-languages have to keep in mind are: variables, since Lisps usually don't have the same scopes as anything imperative (with the ‘let’ blocks); and function names and possibly functions-as-variables, since Lisps are liberal with both. As a rule, you'd have to implement your own variable scopes and name mangling for where the semantics don't match. Plus macros are an obvious addition—IIRC Hy still has them in separate files (though I might confuse that with Fennel). And if you wish to have keywords as a separate type, that's usually entirely on you—afaik Clojure went full-in here, others not so much. > reading unfamiliar syntax often seems to be a sticking point for people considering/learning a new language Precisely the opposite of my and many other people's experience. Remembering whether blocks are delimited with braces, indentation or e.g. ‘then/end’ is a no-brainer. Learning the subtle differences in semantics, the type system, the library, package system and build environments—that's the bitch. Consider that there are dozens of even kinda-popular Lisps, despite them not having much syntax in the first place. They don't just differ in the function names. You seem to be imagining a ‘universal-ish intermediate language’, so to say, that would permit people with different experience to quickly adapt to an environment of a different language. Not possible, because a) semantics is where the difference is, you'd need a full-blown language translator, which still can be lossy or incomplete (e.g. no types in your standard Lisp); b) one would still have to deal with all the non-language parts of the environment, like the library, packages, the build system.

asoneth 5y ago
> You seem to be imagining a ‘universal-ish intermediate language’ To clarify, I do not mean that each developer would be using a different language -- they would all be writing "Python", they'd just using a superficially distinct syntax that each finds most pleasant/familiar/ergonomic. > Remembering whether blocks are delimited with braces, indentation or e.g. ‘then/end’ is a no-brainer. I tend to find the same, at least after an initial learning phase. In fact, I sometimes find distinct languages with similar syntax confusing since I subconsciously expect the same behavior and packages too. Having said that I've taught and TA'd Scheme, Java, Jess, and block languages like Scratch & Alice many years ago, and in those cases a significant number of students found unfamiliar syntax to be a major hurdle, especially if it was one of their first two programming languages. Also, judging by the volume of online complaints about Python's semantically meaningful indentation, Ada's array indexing, and Lisp's parentheses these kinds of conventions do seem like a concern even for experienced developers. > one would still have to deal with all the non-language parts of the environment, like the library, packages, the build system. Right, that would be the point, to be able to use a language's library/packages/build system without necessarily using its original syntax.
- aasasd 5y ago
  Well, doing this on top of a single language is more feasible, however the Python-Lisp pair is still not the best choice due to Python's strict semantics of separation between statements and expressions: when you generate an AST from Lisp, you'd have some sorta verbose shims in place of Lisp's expression-ic statements—precisely ones that Hy now creates. And vice versa, a coder's normal Python might not map cleanly to Hy. If you're free to invent the languages in the first place, it would be kinda trivial to make several with the same semantics but different superficial syntax—basically just substitute one bunch of characters for another, no need to go via AST even. And these languages can then be transpiled to a single language of your choice, be that Python or JVM bytecode—if the semantics are made to be close enough.
  - gilch 5y ago
    What about a Lissp-Hebigo pair? https://github.com/gilch/hissp#hebigo https://github.com/gilch/hissp#hebigo Hissp takes a different approach than Hy. Where Hy has to use shims to pretend statements are expressions, Hissp just targets the expression subset in the first place. (Actually a somewhat smaller subset than that if you're not injecting any raw Python: literals, lambdas, identifiers, and calls.)

Foxboron 5y ago

There are code which takes the Python AST and attempts to produce the correct Python code. It has a few issues if i recall correctly but it should mostly^tm work. https://github.com/berkerpeksag/astor https://github.com/berkerpeksag/astor

gilch 5y ago
That capability is in the standard library now: https://docs.python.org/3/library/ast.html?highlight=unparse#ast.unparse https://docs.python.org/3/library/ast.html?highlight=unparse...

qubex 5y ago

Basically what you’re suggesting is a kind of AST decompiler (in an IDE front-end?) that allows developers to view and edit the code in an essentially arbitrary syntax?

asoneth 5y ago
Yes. They'd be using the same language, but see different syntax styles.

mark_l_watson 5y ago

If you do not use the contributed “let” macro, then auto generated Python code from Hy source code looks fine. If you look at the GitHub repo for the Hy book I wrote, you will see a Makefile target for generating Python code from the Hy examples: https://github.com/mark-watson/hy-lisp-python https://github.com/mark-watson/hy-lisp-python EDIT: not a Makefile target, I used a shell script create_python_source_from_hy.sh

gilch 5y ago

See hy2py: https://docs.hylang.org/en/master/language/cli.html#hy2py https://docs.hylang.org/en/master/language/cli.html#hy2py

gilch 5y ago

See JetBrains MPS: https://www.jetbrains.com/mps/ https://www.jetbrains.com/mps/ The "Projectional Editor" might be the kind of thing you're talking about. The meta language is stored as syntax trees rather than text and can be rendered in different ways.