4 ms·

The underlying problem is still highly relevant in relation to JavaScript build tooling. Let's say that you have a transpiler written in Rust and you want to wr

The underlying problem is still highly relevant in relation to JavaScript build tooling. Let's say that you have a transpiler written in Rust and you want to write a plugin that performs a custom AST transform. If you want to be able to write the plugin in JavaScript, you have to take the AST from Rust and convert it to a JavaScript data structure in order to pass it into the plugin and then you have to convert the output back into a Rust data structure on the other end. Or you have to provide a JavaScript API that can safely mutate a Rust data structure from JavaScript while converting primitive values each way on demand. This is exactly like the problem I described with my Markdown processor. There's a ton of overhead involved, and it can cancel out a depressing amount of the performance gain that you would otherwise get from moving things to Rust. Ultimately, these build tools need to have some degree of programmatic extensibility, and people want to get that without having to write their domain-specific logic in Rust and recompile the whole binary. There needs to be a better extensibility story and a cheaper (ideally, zero-copy) way to share data across the language boundary.

satvikpendem 4y ago
That's true that it needs more extensibility and zero-copy sharing of data, but in your example, are you serializing to something like JSON? Why does the end user need to specify the tags in JavaScript particularly and not something like JSON? Forgive me if I'm not understanding and you are indeed using JSON.
- segphault 4y ago
  Markdoc custom tag definitions can include arbitrary AST transforms on the child nodes, as in this example: https://markdoc.dev/docs/examples#tabs https://markdoc.dev/docs/examples#tabs In order to do this, you need some API, so you can't just define the tags in JSON.
  - satvikpendem 4y ago
    Ah I see. Then yeah, that's a little rough to be doing in both JS and Rust.

satvikpendem 4y ago

That's true that it needs more extensibility and zero-copy sharing of data, but in your example, are you serializing to something like JSON? Why does the end user need to specify the tags in JavaScript particularly and not something like JSON? Forgive me if I'm not understanding and you are indeed using JSON.

segphault 4y ago
Markdoc custom tag definitions can include arbitrary AST transforms on the child nodes, as in this example: https://markdoc.dev/docs/examples#tabs https://markdoc.dev/docs/examples#tabs In order to do this, you need some API, so you can't just define the tags in JSON.
- satvikpendem 4y ago
  Ah I see. Then yeah, that's a little rough to be doing in both JS and Rust.