4 ms·

In my experience when codegen shows up as a solution, you took a wrong turn somewhere and are walking down the path of very brittle code that becomes a nightmar

In my experience when codegen shows up as a solution, you took a wrong turn somewhere and are walking down the path of very brittle code that becomes a nightmare to maintain over time. Several times the code generation libraries don't get updated and your are left in a difficult spot when migrating your code to a newer version of your platform. but hey maybe that's just a .net and java thing.

k__ 11y ago
It isn't. I used at least one PHP to JavaScript code-gen in the past and it was rather bad. I also dislike this Rails/Ember-CLI scaffolding stuff. It simply feels like too much magic for me.
- aikah 11y ago
  > I also dislike this Rails/Ember-CLI scaffolding stuff. It simply feels like too much magic for me. Good thing nobody forces you to use scaffolding in Rails. Code generation should always be a one shot , to get started on something. On the other hand the situation in Go land with "go generate" is much much worse, as code generators become dependencies , right in the code with pragmas ... The worst offender is PHP of course, with frameworks like Symfony or Doctrine (and its famous proxies because yes, Doctrine doesn't actually use the code you write but the one it generates and put in a proxy folder ) that use code compilers and generators for everything from routing to dependency injection to every configuration step . Although PHP frameworks usually do it on the fly so no manual step.
fizx 11y ago
There's sort of a survivorship bias in play here, where you only noticed the brittle codegen being brittle.
- fapjacks 11y ago
  At the risk of sounding obtuse, then, where is the robust codegen code?
  - bartonfink 11y ago
    Compilers, for start.
    - fapjacks 11y ago
      Hah! Touche!
  - nostrademons 11y ago
    On a continuum from "so robust you don't even consider it codegen" to "well, some people use it": microcode, compilers, virtualization, visual interface builders like XCode and Android Studio, dynamic HTML pages, ORMs, protobufs & other schema definition languages, autoconf & friends, compile-to-C languages like early versions of C++ and Go, ES6 transpilers like Traceur and Babel, compile-to-JS languages like CoffeeScript. Probably others I've forgotten or don't know about. When it works, you forget that the only thing that computers understand is their native instruction set, and everything on top of that is built on some form of codegen. When it doesn't, you curse out the programmer who left you a pile of buggy scripts to maintain.
    - kazinator 11y ago
      Nice easter egg: "autoconf & friends" buried in the middle of a paragraph that appears to be about robust code generation.
      - nostrademons 11y ago
        "Well, some people use it." Particularly toward the end of the list, there'll be more and more examples where people hate the code generation aspect (for example, I've personally had bad experiences with any JS technology that requires a build step). That doesn't change the fact that they're widely used; autoconf, for example, was mandated for all Google-owned open-source C/C++ libraries, because despite its kludginess, it will build on virtually everything.
  - deleted 11y ago
    [deleted]
- Ologn 11y ago
  Also, codegen exists on a solution continuum. Codegen is a superior solution to manually pumping out lots of boilerplate code with slight variations. A solution that does not need lots of boilerplate code would be even better, but codegen is still better than the makeshift code that it originally replaced.
  - ArkyBeagle 11y ago
    There's an intermediate approach - use a generator to spew out tables of data that an engine-type thing uses to adapt to boilerplate style variations. This is totally a 'C' hacker's approach but it's not bad for some things - especially serialization and the like. But it will need some generic reference mechanism - as void * is used in 'C' - to work.
joshribakoff 11y ago
I agree. They're also generating code for a language they made up. Seems very risky. Whether using code generation or not, having lots of repetitive getters & setters is a code smell. I feel like people just do that to feel more "professional" since they heard OOP was good to use, when a simple associative array would have sufficed. The answer isn't to start generating the repetitive code in my opinion, the answer is to switch to a data structure which doesn't involve repetitive code in the first place. That just my opinion though & its subjective.
munificent 11y ago
I think the motivation is pretty clear here: * It was not typed, and with the advent of Hack, we wanted to have types. * The actual setter method was not defined. If you searched the codebase, you wouldn't find it. If you looked at the class, you couldn't see which methods you could call, which made it harder to discover other calls, such as ifNotNullSetName. * IDEs can't autocomplete the code. What this boils down to is metaprogramming. If you want to create an array that contains twenty items, you aren't going to do: array.add(1); array.add(2); array.add(3); ... You'll use a loop, some imperative code. That's because data structures like arrays are within the domain of things you can touch with code. But what about class definitions? What if you want to add twenty methods to a class? Fans of dynamic languages will rightly say that class definitions, functions, etc. should also be in the domain of what your code can affect. Smalltalkers and Rubyists will happily write imperative code to build and modify classes. This is really powerful. It's exciting stuff. At the same time, for better or worse, most programmers are not spending their day in a live editing environment. They aren't in a Smalltalk browser all day. They're in an IDE or text editor. That means the window into the program that we have while editing doesn't see any of the results of that metaprogramming. All of the static analysis tools, auto-completers, go-to-definition-ers, etc. can't handle dynamic metaprogramming because they run before any of that metaprogramming is invoked. Code generation is a way out of that problem. It lets you do imperative dynamic metaprogramming, but then bakes the result back out to the file system where the rest of your tool ecosystem can see it. It's not the most elegant solution in the world, but I think it beats trying to reinvent the entire world of tooling that programmers use all day: grep, text editors, IDEs, etc.
- nmrm2 11y ago
  > but I think it beats trying to reinvent the entire world of tooling that programmers use all day: grep, text editors, IDEs, etc. Exactly, but only because we don't have a any really good solutions to building tooling that supports meta-programming for typed languages. > the window into the program that we have while editing doesn't see any of the results of that metaprogramming I wonder if there's a way to build IDEs that provide support for metaprogramming. (Also, this was a really excellent comment.)
kazinator 11y ago
It's probably a .net and Java thing. You can find some crufty Lisp macros from 1983 (code that writes code, par excellence) and they still work today.
justifier 11y ago
do you consider a compiler 'codegen'?

k__ 11y ago

It isn't. I used at least one PHP to JavaScript code-gen in the past and it was rather bad. I also dislike this Rails/Ember-CLI scaffolding stuff. It simply feels like too much magic for me.

aikah 11y ago
> I also dislike this Rails/Ember-CLI scaffolding stuff. It simply feels like too much magic for me. Good thing nobody forces you to use scaffolding in Rails. Code generation should always be a one shot , to get started on something. On the other hand the situation in Go land with "go generate" is much much worse, as code generators become dependencies , right in the code with pragmas ... The worst offender is PHP of course, with frameworks like Symfony or Doctrine (and its famous proxies because yes, Doctrine doesn't actually use the code you write but the one it generates and put in a proxy folder ) that use code compilers and generators for everything from routing to dependency injection to every configuration step . Although PHP frameworks usually do it on the fly so no manual step.

fizx 11y ago

There's sort of a survivorship bias in play here, where you only noticed the brittle codegen being brittle.

fapjacks 11y ago
At the risk of sounding obtuse, then, where is the robust codegen code?
- bartonfink 11y ago
  Compilers, for start.
  - fapjacks 11y ago
    Hah! Touche!
- nostrademons 11y ago
  On a continuum from "so robust you don't even consider it codegen" to "well, some people use it": microcode, compilers, virtualization, visual interface builders like XCode and Android Studio, dynamic HTML pages, ORMs, protobufs & other schema definition languages, autoconf & friends, compile-to-C languages like early versions of C++ and Go, ES6 transpilers like Traceur and Babel, compile-to-JS languages like CoffeeScript. Probably others I've forgotten or don't know about. When it works, you forget that the only thing that computers understand is their native instruction set, and everything on top of that is built on some form of codegen. When it doesn't, you curse out the programmer who left you a pile of buggy scripts to maintain.
  - kazinator 11y ago
    Nice easter egg: "autoconf & friends" buried in the middle of a paragraph that appears to be about robust code generation.
    - nostrademons 11y ago
      "Well, some people use it." Particularly toward the end of the list, there'll be more and more examples where people hate the code generation aspect (for example, I've personally had bad experiences with any JS technology that requires a build step). That doesn't change the fact that they're widely used; autoconf, for example, was mandated for all Google-owned open-source C/C++ libraries, because despite its kludginess, it will build on virtually everything.
- deleted 11y ago
  [deleted]
Ologn 11y ago
Also, codegen exists on a solution continuum. Codegen is a superior solution to manually pumping out lots of boilerplate code with slight variations. A solution that does not need lots of boilerplate code would be even better, but codegen is still better than the makeshift code that it originally replaced.
- ArkyBeagle 11y ago
  There's an intermediate approach - use a generator to spew out tables of data that an engine-type thing uses to adapt to boilerplate style variations. This is totally a 'C' hacker's approach but it's not bad for some things - especially serialization and the like. But it will need some generic reference mechanism - as void * is used in 'C' - to work.

joshribakoff 11y ago

I agree. They're also generating code for a language they made up. Seems very risky. Whether using code generation or not, having lots of repetitive getters & setters is a code smell. I feel like people just do that to feel more "professional" since they heard OOP was good to use, when a simple associative array would have sufficed. The answer isn't to start generating the repetitive code in my opinion, the answer is to switch to a data structure which doesn't involve repetitive code in the first place. That just my opinion though & its subjective.

munificent 11y ago

I think the motivation is pretty clear here: * It was not typed, and with the advent of Hack, we wanted to have types. * The actual setter method was not defined. If you searched the codebase, you wouldn't find it. If you looked at the class, you couldn't see which methods you could call, which made it harder to discover other calls, such as ifNotNullSetName. * IDEs can't autocomplete the code. What this boils down to is metaprogramming. If you want to create an array that contains twenty items, you aren't going to do: array.add(1); array.add(2); array.add(3); ... You'll use a loop, some imperative code. That's because data structures like arrays are within the domain of things you can touch with code. But what about class definitions? What if you want to add twenty methods to a class? Fans of dynamic languages will rightly say that class definitions, functions, etc. should also be in the domain of what your code can affect. Smalltalkers and Rubyists will happily write imperative code to build and modify classes. This is really powerful. It's exciting stuff. At the same time, for better or worse, most programmers are not spending their day in a live editing environment. They aren't in a Smalltalk browser all day. They're in an IDE or text editor. That means the window into the program that we have while editing doesn't see any of the results of that metaprogramming. All of the static analysis tools, auto-completers, go-to-definition-ers, etc. can't handle dynamic metaprogramming because they run before any of that metaprogramming is invoked. Code generation is a way out of that problem. It lets you do imperative dynamic metaprogramming, but then bakes the result back out to the file system where the rest of your tool ecosystem can see it. It's not the most elegant solution in the world, but I think it beats trying to reinvent the entire world of tooling that programmers use all day: grep, text editors, IDEs, etc.

nmrm2 11y ago
> but I think it beats trying to reinvent the entire world of tooling that programmers use all day: grep, text editors, IDEs, etc. Exactly, but only because we don't have a any really good solutions to building tooling that supports meta-programming for typed languages. > the window into the program that we have while editing doesn't see any of the results of that metaprogramming I wonder if there's a way to build IDEs that provide support for metaprogramming. (Also, this was a really excellent comment.)

kazinator 11y ago

It's probably a .net and Java thing. You can find some crufty Lisp macros from 1983 (code that writes code, par excellence) and they still work today.

justifier 11y ago

do you consider a compiler 'codegen'?