3 ms·

Super cool. Bidirectional code generation is something I've spent a bit of time thinking about: I've been building a spreadsheet that generates Python code when

by narush 4y ago

Super cool. Bidirectional code generation is something I've spent a bit of time thinking about: I've been building a spreadsheet that generates Python code when you edit it [1], but some of our users also want the ability to edit the Python code they generate and have that reflect in the sheet itself. Template -> Code -> Template is one really hard part of this, and something this tool seems to take a really good approach to. If you're interested in related subjects (like transpiliation), I'd recommend this overview article as a great approach [2]. From this article: "Now, there is a single biggest mistake we see in persons trying to implement a transpiler without experience in this: they try to generate directly the code of the target language from the AST of the original language." Question to the OP - you mention you parse the Ruby AST - do you also transform this into an AST of your template syntax before generating the template? Aka, do you avoid this sin, or is it not an issue for you? With Mito, there is additional complexity beyond just going from Template -> Code -> Template, in that we also need to understand _which_ variables are being changed and in what way. This is necessary because a spreadsheet stores other data about your variables beyond just their current value. As an example, which of these columns in a dataframe are a result of a formula vs. being in the original dataset isn't something that is just stored in the dataframe itself. I haven't tried too hard, but I don't think there's a general solution; it feels like it requires some sort of symbolic execution in the general case, and is tough to do well even in simple cases. Our "fail loudly and early" equivalent feels like it would be a lot higher than the 10-20% this tool can deliver! Anyways, bidirectional spreadsehet code generation is low on the priorities... but it's a fun one to dream about :-) [1] https://trymito.io https://trymito.io [2] https://tomassetti.me/how-to-write-a-transpiler/ https://tomassetti.me/how-to-write-a-transpiler/

lucasluitjes 4y ago
Mito looks interesting! Regarding the sin, I don't think it entirely applies here, unless I'm misunderstanding you - I'm not super familiar with transpilers. In this case the template is also ruby code, but with placeholders for the values. So both the template and the input get parsed into ASTs, then both of them are recursively walked and compared on the fly. During that walk it builds up the data structure with initial values. In the code generation direction it walks one AST and does lookups and rewrites with values from the values. Does that make sense?
- narush 4y ago
  Gotcha. Pretty much the template language is close enough to the destination language that you don’t need an AST for the template language at all. That makes sense! Seems like a legit simplification to take advantage of. Do you have any plans to make the template language more advanced/complex?
  - lucasluitjes 4y ago
    Well, looking at the rails monocles (for matching controllers, views, etc) a common pattern is custom matchers/replacers for things like "ast.children.first.to_s.singularize.camelize", so a shorthand for defining those in both directions could be handy. Other than that, not right now. But if there are common situations where you end up writing the same matchers/replacers, those would be prime candidates for built-in placeholders.
InflexHQ 4y ago
Inflex (https://inflex.io/ https://inflex.io/) does this bidirectional editing: you can edit any data structure as either a graphical object or the associated code, and the other one updates appropriately. (See e.g. https://discourse.inflex.io/t/how-to-make-and-access-a-record-in-inflex/25 https://discourse.inflex.io/t/how-to-make-and-access-a-recor...) As for the problem of which things of a data frame are generated from a formula and which are from normal form, Inflex compares the structure with the AST of the source, making it easy to tell that [{foo: ...}] are normal form and so editable graphically, whereas xs.filter(..) is not. The neat thing is you can still edit formulae that are deep within a normal form nested structure. It helps that Inflex doesn’t have syntactic sugar, what’s parsed is what is in the final AST. It also has a symbolic evaluator, somewhat, so it’s fine to have a list of functions for example and edit the list. The evaluator produces the same AST, rather than an alien format. This bidirectionality also applies to rich text editors. (https://mobile.twitter.com/InflexHQ/status/1492356413326336002 https://mobile.twitter.com/InflexHQ/status/14923564133263360...) Mito has the advantage of being a familiar language and ecosystem, but Python itself has a traditional runtime, it’s imperative and not expression oriented, and lacks sound static type information, so it’s inherently more difficult to achieve some things with it that are easier with a typed pure functional language, especially a custom one.

by narush 4y ago

lucasluitjes 4y ago

Mito looks interesting! Regarding the sin, I don't think it entirely applies here, unless I'm misunderstanding you - I'm not super familiar with transpilers. In this case the template is also ruby code, but with placeholders for the values. So both the template and the input get parsed into ASTs, then both of them are recursively walked and compared on the fly. During that walk it builds up the data structure with initial values. In the code generation direction it walks one AST and does lookups and rewrites with values from the values. Does that make sense?

narush 4y ago
Gotcha. Pretty much the template language is close enough to the destination language that you don’t need an AST for the template language at all. That makes sense! Seems like a legit simplification to take advantage of. Do you have any plans to make the template language more advanced/complex?
- lucasluitjes 4y ago
  Well, looking at the rails monocles (for matching controllers, views, etc) a common pattern is custom matchers/replacers for things like "ast.children.first.to_s.singularize.camelize", so a shorthand for defining those in both directions could be handy. Other than that, not right now. But if there are common situations where you end up writing the same matchers/replacers, those would be prime candidates for built-in placeholders.

InflexHQ 4y ago

Inflex (https://inflex.io/ https://inflex.io/) does this bidirectional editing: you can edit any data structure as either a graphical object or the associated code, and the other one updates appropriately. (See e.g. https://discourse.inflex.io/t/how-to-make-and-access-a-record-in-inflex/25 https://discourse.inflex.io/t/how-to-make-and-access-a-recor...) As for the problem of which things of a data frame are generated from a formula and which are from normal form, Inflex compares the structure with the AST of the source, making it easy to tell that [{foo: ...}] are normal form and so editable graphically, whereas xs.filter(..) is not. The neat thing is you can still edit formulae that are deep within a normal form nested structure. It helps that Inflex doesn’t have syntactic sugar, what’s parsed is what is in the final AST. It also has a symbolic evaluator, somewhat, so it’s fine to have a list of functions for example and edit the list. The evaluator produces the same AST, rather than an alien format. This bidirectionality also applies to rich text editors. (https://mobile.twitter.com/InflexHQ/status/1492356413326336002 https://mobile.twitter.com/InflexHQ/status/14923564133263360...) Mito has the advantage of being a familiar language and ecosystem, but Python itself has a traditional runtime, it’s imperative and not expression oriented, and lacks sound static type information, so it’s inherently more difficult to achieve some things with it that are easier with a typed pure functional language, especially a custom one.