6 ms·

Writing code that writes code – Hack Codegen

FractalNerve 11y ago
Cheers for the efforts, but that's a poor DSL or an atempt to create an Aspect-oriented version of Hack. You've gorgeous tools at facebook like for example 'pfff' [0] to play with ASTs, these could be used to build wonderful things given enough time. I suppose the ultimative goal is to allow business people to design workflows and let the tool generate the required code automatically. (I guess that's what Flow Based Programming is trying to achieve.) Kinda possibly with high-level specifications defined in languages better suited to the task like Racket, Rebol/Red, Xtend/Scala etc. There is an interesting presentation by ThoughtWorks – Neil Ford: Building DSLs in Static and Dynamic Languages [1] However I thought you might find this the EPFL / ETH Summer School on DSL Design and Implementation very useful. They not only come with presentations, but also code in Scala, Racket, Haskell et. al. I am currently working in a huge project with about >30 people working on building an MDD (Model Driven Development) platform for visually/declaratively writing things for the systems we use internally. These will be able to create models out of code and even fix API incompatibilities automatically. [2] Excuse the Unicode, I got bored ⬬══════━━━━━━──⋯𝅖܅܅܅܅﴿⌁⏧⌁﴾܅܅܅܅𝅖⋯──━━━━━━═══════⬬ [0] https://github.com/facebook/pfff https://github.com/facebook/pfff [1] http://www.intertech.com/resource/usergroup/Neal_Ford-Building_DSLs_in_Static_and_Dynamic_Languages-handouts.pdf http://www.intertech.com/resource/usergroup/Neal_Ford-Buildi... [2] http://vjovanov.github.io/dsldi-summer-school/program.html http://vjovanov.github.io/dsldi-summer-school/program.html
PretzelFisch 11y ago
In my experience when codegen shows up as a solution, you took a wrong turn somewhere and are walking down the path of very brittle code that becomes a nightmare to maintain over time. Several times the code generation libraries don't get updated and your are left in a difficult spot when migrating your code to a newer version of your platform. but hey maybe that's just a .net and java thing.
- k__ 11y ago
  It isn't. I used at least one PHP to JavaScript code-gen in the past and it was rather bad. I also dislike this Rails/Ember-CLI scaffolding stuff. It simply feels like too much magic for me.
  - aikah 11y ago
    > I also dislike this Rails/Ember-CLI scaffolding stuff. It simply feels like too much magic for me. Good thing nobody forces you to use scaffolding in Rails. Code generation should always be a one shot , to get started on something. On the other hand the situation in Go land with "go generate" is much much worse, as code generators become dependencies , right in the code with pragmas ... The worst offender is PHP of course, with frameworks like Symfony or Doctrine (and its famous proxies because yes, Doctrine doesn't actually use the code you write but the one it generates and put in a proxy folder ) that use code compilers and generators for everything from routing to dependency injection to every configuration step . Although PHP frameworks usually do it on the fly so no manual step.
- fizx 11y ago
  There's sort of a survivorship bias in play here, where you only noticed the brittle codegen being brittle.
  - fapjacks 11y ago
    At the risk of sounding obtuse, then, where is the robust codegen code?
    - bartonfink 11y ago
      Compilers, for start.
      - fapjacks 11y ago
        Hah! Touche!
jeanetienne 11y ago
Am I the only one to be bothered by the fact that both examples of codegen/generated code shows that you have to write more codegen to generate less actual code? In the first example there more than twice the amount of code written in the codegen than in the generated code... I know they are simple example but still, if I have to write more code for all the simple getters/setters, it defeats the purpose of the codegen which is to "reduce boilerplate". Am I missing something?
- zeroonetwothree 11y ago
  Each codegen is run multiple times. Imagine you want to generate 100 similar classes with similar code except for a small difference.
  - epaga 11y ago
    But that would be a code smell akin to copy-pasting code wouldn't it? I'm having trouble seeing any real-world scenarios for this kind of codegen but since it seems to be in wide use at Facebook, I must just be missing something...
EGreg 11y ago
Real codegen would be like "Computer, make me a companion." "Here you go, would you like to customize this hologram?"
hnruss 11y ago
"Notice that the manual section uses an ID to match it with the corresponding section when regenerating the code so that is placed in the same location." public function getName(): string { /* BEGIN MANUAL SECTION User::geName */ Wonder how well that works if there's a typo in the section comment?
- zeroonetwothree 11y ago
  I believe the comment is generated by the codegen, so it won't have a typo.
  - hnruss 11y ago
    That makes more sense, thanks.

Cheers for the efforts, but that's a poor DSL or an atempt to create an Aspect-oriented version of Hack. You've gorgeous tools at facebook like for example 'pfff' [0] to play with ASTs, these could be used to build wonderful things given enough time. I suppose the ultimative goal is to allow business people to design workflows and let the tool generate the required code automatically. (I guess that's what Flow Based Programming is trying to achieve.) Kinda possibly with high-level specifications defined in languages better suited to the task like Racket, Rebol/Red, Xtend/Scala etc. There is an interesting presentation by ThoughtWorks – Neil Ford: Building DSLs in Static and Dynamic Languages [1] However I thought you might find this the EPFL / ETH Summer School on DSL Design and Implementation very useful. They not only come with presentations, but also code in Scala, Racket, Haskell et. al. I am currently working in a huge project with about >30 people working on building an MDD (Model Driven Development) platform for visually/declaratively writing things for the systems we use internally. These will be able to create models out of code and even fix API incompatibilities automatically. [2] Excuse the Unicode, I got bored ⬬══════━━━━━━──⋯𝅖܅܅܅܅﴿⌁⏧⌁﴾܅܅܅܅𝅖⋯──━━━━━━═══════⬬ [0] https://github.com/facebook/pfff https://github.com/facebook/pfff [1] http://www.intertech.com/resource/usergroup/Neal_Ford-Building_DSLs_in_Static_and_Dynamic_Languages-handouts.pdf http://www.intertech.com/resource/usergroup/Neal_Ford-Buildi... [2] http://vjovanov.github.io/dsldi-summer-school/program.html http://vjovanov.github.io/dsldi-summer-school/program.html

PretzelFisch 11y ago

In my experience when codegen shows up as a solution, you took a wrong turn somewhere and are walking down the path of very brittle code that becomes a nightmare to maintain over time. Several times the code generation libraries don't get updated and your are left in a difficult spot when migrating your code to a newer version of your platform. but hey maybe that's just a .net and java thing.

k__ 11y ago
It isn't. I used at least one PHP to JavaScript code-gen in the past and it was rather bad. I also dislike this Rails/Ember-CLI scaffolding stuff. It simply feels like too much magic for me.
- aikah 11y ago
  > I also dislike this Rails/Ember-CLI scaffolding stuff. It simply feels like too much magic for me. Good thing nobody forces you to use scaffolding in Rails. Code generation should always be a one shot , to get started on something. On the other hand the situation in Go land with "go generate" is much much worse, as code generators become dependencies , right in the code with pragmas ... The worst offender is PHP of course, with frameworks like Symfony or Doctrine (and its famous proxies because yes, Doctrine doesn't actually use the code you write but the one it generates and put in a proxy folder ) that use code compilers and generators for everything from routing to dependency injection to every configuration step . Although PHP frameworks usually do it on the fly so no manual step.
fizx 11y ago
There's sort of a survivorship bias in play here, where you only noticed the brittle codegen being brittle.
- fapjacks 11y ago
  At the risk of sounding obtuse, then, where is the robust codegen code?
  - bartonfink 11y ago
    Compilers, for start.
    - fapjacks 11y ago
      Hah! Touche!

jeanetienne 11y ago

Am I the only one to be bothered by the fact that both examples of codegen/generated code shows that you have to write more codegen to generate less actual code? In the first example there more than twice the amount of code written in the codegen than in the generated code... I know they are simple example but still, if I have to write more code for all the simple getters/setters, it defeats the purpose of the codegen which is to "reduce boilerplate". Am I missing something?

zeroonetwothree 11y ago
Each codegen is run multiple times. Imagine you want to generate 100 similar classes with similar code except for a small difference.
- epaga 11y ago
  But that would be a code smell akin to copy-pasting code wouldn't it? I'm having trouble seeing any real-world scenarios for this kind of codegen but since it seems to be in wide use at Facebook, I must just be missing something...

EGreg 11y ago

Real codegen would be like "Computer, make me a companion." "Here you go, would you like to customize this hologram?"

hnruss 11y ago

"Notice that the manual section uses an ID to match it with the corresponding section when regenerating the code so that is placed in the same location." public function getName(): string { /* BEGIN MANUAL SECTION User::geName */ Wonder how well that works if there's a typo in the section comment?

zeroonetwothree 11y ago
I believe the comment is generated by the codegen, so it won't have a typo.
- hnruss 11y ago
  That makes more sense, thanks.