4 ms·

> the "templates" we use for codegen need to be valid Typescript files, or we risk breakage if imports or types change. Any tips on how to do this? I’ve been r

• love2read 3y ago
  Out of curiosity, what are you trying to make?
  • wolfgang42 3y ago
    I’ve run into this in a few different contexts, which usually boil down to “I have a schema, and I’d like to automatically write code that uses it.” (Think things like API definitions, database models, binary serialization formats, and the like.) Trying to do it at runtime with something like io-ts results in a lot of indirection that’s hard to follow, and writing the files by hand results in silly mistakes because it’s tedious to get all the details right. Generating the files and checking them in gives you the best of both worlds: the readability of hand-written code, with the accuracy of automation. But I’ve never been able to get the generator itself as nice as I’d like; string templating mostly works adequately, but as the parent says it lacks integration with the regular toolchains.
• jitl 3y ago
  The way I do it is pretty dumb. We have files like src/test/_template.test.ts that are valid TS, that our codegen commands like `notion new test` regex-replace to “instantiate” the template. The convention we follow is to name replacement targets with double underscores. No need for some kind of specialized template library - just regex & replaceAll. The scripts that consume our templates are mostly very simple ~15 line affairs but there are a few very complex, 1000+ line affairs that use the Typescript type checker API to codegen model classes and test data generators from hand-written interface types. Those things read ASTs but writing ASTs seems like a huge waste of time to me. AST is only nice if you need to very carefully morph existing code, and even then I usually end up doing a string replacement of an AST node’s position in its original file. In all that code I think the only thing that writes using an AST is import statement stuff. Luckily that super complex stuff runs on every commit so it can’t break. It’s the one-off codegen commands like “make me a new component” or “make me an integration test” that are in danger of going stale, and even those we could write some CI tests for like “make a new thingy, check that it typechecks”. Anyways here’s a full example template file: /*__FILE_METADATA__*/ /* ================================================================================ __TEMPLATE_CLASS_NAME__. Docs: https://dev.notion.so/notion/Record-Framework-1f41f97e1cc746628a6af57ba75d75ad THIS IS FILE IS PARTIALLY GENERATED. ONLY EDIT THE EDITABLE REGION BELOW. __GENERATED_BY__ __GENERATED_FROM__ ================================================================================ */ import type { /*__VALUE_IMPORT__*/ BlockValue as __VALUE_TYPE__, /*__TABLE_IMPORT__*/ BlockTable as __TABLE__, } from /*__SCHEMA_IMPORT__*/ "../schemas/Block" import { Model } from "./Model" /*__REMOVE_LINE__*/ // : Value <Value> as __MODEL__<Value> | undefined <- Conform to template args in RecordStore. /** * This class is generated from {@link __VALUE_TYPE__}. * To customize, edit the section in {@link __TEMPLATE_CLASS_NAME__} below. */ abstract class Generated__TEMPLATE_CLASS_NAME__< Value extends __VALUE_TYPE__ = __VALUE_TYPE__ > extends Model<Value, __TABLE__> { /*__REMOVE_LINE__*/ __BODY__: undefined // Put the goods here. } /** * Read and interpret the data of a __RECORD__ record. */ export class __TEMPLATE_CLASS_NAME__< Value extends __VALUE_TYPE__ = __VALUE_TYPE__ > extends Generated__TEMPLATE_CLASS_NAME__<Value> { /*__USER_EDITABLE_SECTION__*/ }
  • wolfgang42 3y ago
    Cool, thanks! “Just use a template syntax that’s also valid code” is a neat trick; I was sort of headed in that direction, but having a concrete example is useful. What are the commented bits like /* __REMOVE_LINE__ / and / __SCHEMA_IMPORT__ */ used for?
    • jitl 3y ago
      The template replacer function takes a map from regex to replacement, and asserts that each regex must occur at least once which helps catch mistakes where you update the template but not the code calling it. __REMOVE_LINE__ is a common hack with our template system to remove lines in the template that are needed to make the template valid, but aren't wanted in the resulting code. We replace /^.__REMOVE_LINE__.$/gm with "". In this case we need to accept the __MODEL__ template var because a different template called with the same arguments needs it, but we don't care in this one. __SCHEMA_IMPORT__ is another marker we use to replace a whole line, here we replace it with `} from "${generateData.schemaImportPath}"`