5 ms·

Ask HN: Guidance on writing a source to source compiler (transpiler)

11 points by techn00 4y ago

I would like to learn to write a transpiler from scratch. Where would I get started? Preferably something that doesn't involve going though big projects to read the source code.

kunley 4y ago
Doesn't matter much if the output is also a source, you can't avoid lexing and parsing, so the best thing to get started seems to be... the Dragon Book, right?
- HelloNurse 4y ago
  In practice, using common programming languages as source should offer the important shortcut of adopting existing good, ready-made parsers and AST manipulation libraries because other compilers, linting tools etc. already developed them.
samsaga2 4y ago
If the languages are similar: 1. Tokenize 2. Build AST tree 3. Emit target code from the AST tree If they are very different: 1. Tokenize 2. Build AST tree 3. Create a simple intermediate language from the AST tree 4. Emit target code from the intermediate language It is the same as building a whole compiler, but without the optimization/register allocator passes.
ricklamers 4y ago
A good option would be to take a MooC on compilers https://www.edx.org/course/compilers https://www.edx.org/course/compilers
deleted 4y ago
[deleted]
quectophoton 4y ago
If you don't mind a video, Destroy All Software has a nice screencast on the topic: https://www.destroyallsoftware.com/screencasts/catalog/a-compiler-from-scratch https://www.destroyallsoftware.com/screencasts/catalog/a-com... Really easy to understand, even if Ruby is not your main language. The video is about compiling from a tiny Ruby-like language[1] into real JavaScript. [1]: It's only a single, simple function, but the concepts are well explained and it should be easy to build on top of that.
cdhdc 4y ago
http://www.txl.ca/txl-index.html http://www.txl.ca/txl-index.html
stevekemp 4y ago
You could start here: https://github.com/jamiebuilds/the-super-tiny-compiler https://github.com/jamiebuilds/the-super-tiny-compiler That converts from lisp-like to javascript. Really though this is a big field, and there are lots of resources out there. To get started look at your input language; you'll need to lex and parse that. Then massage the parsed structure into the appropriate output. You can see me convert brainfuck to C, or x86 assembly language here: https://github.com/skx/bfcc https://github.com/skx/bfcc

11 points by techn00 4y ago

I would like to learn to write a transpiler from scratch. Where would I get started? Preferably something that doesn't involve going though big projects to read the source code.

kunley 4y ago

Doesn't matter much if the output is also a source, you can't avoid lexing and parsing, so the best thing to get started seems to be... the Dragon Book, right?

HelloNurse 4y ago
In practice, using common programming languages as source should offer the important shortcut of adopting existing good, ready-made parsers and AST manipulation libraries because other compilers, linting tools etc. already developed them.

samsaga2 4y ago

If the languages are similar: 1. Tokenize 2. Build AST tree 3. Emit target code from the AST tree If they are very different: 1. Tokenize 2. Build AST tree 3. Create a simple intermediate language from the AST tree 4. Emit target code from the intermediate language It is the same as building a whole compiler, but without the optimization/register allocator passes.

ricklamers 4y ago

A good option would be to take a MooC on compilers https://www.edx.org/course/compilers https://www.edx.org/course/compilers

deleted 4y ago

[deleted]

quectophoton 4y ago

If you don't mind a video, Destroy All Software has a nice screencast on the topic: https://www.destroyallsoftware.com/screencasts/catalog/a-compiler-from-scratch https://www.destroyallsoftware.com/screencasts/catalog/a-com... Really easy to understand, even if Ruby is not your main language. The video is about compiling from a tiny Ruby-like language[1] into real JavaScript. [1]: It's only a single, simple function, but the concepts are well explained and it should be easy to build on top of that.

cdhdc 4y ago

http://www.txl.ca/txl-index.html http://www.txl.ca/txl-index.html

stevekemp 4y ago

You could start here: https://github.com/jamiebuilds/the-super-tiny-compiler https://github.com/jamiebuilds/the-super-tiny-compiler That converts from lisp-like to javascript. Really though this is a big field, and there are lots of resources out there. To get started look at your input language; you'll need to lex and parse that. Then massage the parsed structure into the appropriate output. You can see me convert brainfuck to C, or x86 assembly language here: https://github.com/skx/bfcc https://github.com/skx/bfcc