5 ms·

Meta question, do transpilers actually work well in practice? My intuition is that they would have all sorts of limitations, but I've never had to use one. Coul

Meta question, do transpilers actually work well in practice? My intuition is that they would have all sorts of limitations, but I've never had to use one. Could anyone point me to a resource describing what those limitations might be? In what circumstances would they work best?

dagw 11y ago
I use two transpilers in my day to day work, cython and coffeescript. In both cases they work really well and save me a bunch of time and effort. However in both cases the transpiler is written in parallel with the source language which always makes it a lot easier and the transpiler obviously supports 100% of the source language. I have also played with transpilers written for 'foreign' languages and then you do end up in a situation where you can only use a (often badly defined) subset of the source language, and debugging is a pain since the output is often very difficult to read. Still they're handy when you need them and can (sometimes) save you time compared to porting the code by hand.
Arnt 11y ago
Can't point to any resource... but my experience is that the main problems are a) parsing the input language and in the same way as other tools do and b) giving useful diagnostics and making the next-step compiler's and/or debugger's diagnostics useful. Far from trivial, but not insurmountable.
chii 11y ago
one of the most popular transpiler is http://haxe.org http://haxe.org - it compiles down to many different languages, and it works pretty well.
tomjen3 11y ago
Transpilers are just a new word for compilers - the major difference is that the target language is a high level one, but that just means that it is a lot easier to write the compiler (no optimizing register assignment!). I doubt a transpiler has any limitations that doesn't apply to compilers in general, though of course any particular one might.
jerf 11y ago
In the end, a "transpiler" is just a compiler and subject to the same limitations. There is no generic set of limitations that apply to all compilers. Some are very good because they do something relatively easy, like Coffeescript -> JavaScript. I don't mean that writing a great Coffeescript compiler [1] is necessarily "easy", but the Coffeescript language was tightly designed to map directly to Javascript constructs, so there's hardly any "limitations" at all; very nearly the entire target language is available to you, and to the extent that there may be JS you can't generate, it's probably JS the community generally agrees is a bad idea anyhow. Others, like gcc or LLVM, are actually bridging a fairly large gulf now between the source and target languages [2], but do a great job because of massive piles of effort by incredibly smart people. But if you did have to make generalizations, the two big ones are that generally you can have a performance problem if you're forced to compile something onto a target with a very different paradigm, and the target just doesn't let you have a low-enough level view of things to implement the source language's abstractions efficiently. For instance, consider the languages targeting the JVM before invokedynamic was added to the JVM... there was a fundamental paradigm mismatch that bled away a certain unavoidable amount of performance. The other problem you can get is that you may not get slick access to the low-level details of the target language; for instance, take a look at the piles of things that compile to JavaScript and compare how they allow you to use jQuery. It's anything from "What's the problem?" in CoffeeScript to being a full-on Foreign Function Interface-type call in GHCJS (even if you can get it pre-wrapped for you [3]), a Haskell->Javascript compiler. Both are really two aspects of the same "impedance mismatch" problem. [1]: Look, I'll be honest, I consider the term "transpiler" completely unnecessary. It's a compiler. Feel free to just keep using transpiler in your reply and I won't say anything again. I've had this argument, I bow to the trend, but this old fogey doesn't intend to change (and still considers it a serious impediment to understanding to believe they are two different things, despite using the exact same techniques, architecture, tools, mental models... grumble grumble). [2]: My recent previous comment goes into more detail on that claim: https://news.ycombinator.com/item?id=9800231 https://news.ycombinator.com/item?id=9800231 [3] https://github.com/ghcjs/ghcjs-jquery https://github.com/ghcjs/ghcjs-jquery
- munificent 11y ago
  > Look, I'll be honest, I consider the term "transpiler" > completely unnecessary. I'm part of the Dart team, so transpiling is my bread and butter. I do think it is important to make a distinction between a transpiler and a compiler. If you only consider the runtime behavior of the resulting code, sure, the two are roughly equivalent. But, that totally ignores the human factors involved. 1. You may need to inspect the ___piler output to find bugs in it. 2. A user may read the ___piler output to better understand the semantics of the source language. "Oh, ___ in CoffeeScript means ___ in JS!". 3. The user will need to debug their program. Unless you have very sophisticated debugging infrastructure (hint: source maps are not good enough), then that means they will be stepping through the ___piler output, not the original source. 4. A user may end up discarding the original source and hand-maintaining the ___piled output as the new source of truth. (This use case is exactly what the blog post is about.) Even if a user doesn't actually do this, they may want the psychological security of knowing that they could before they are willing to adopt the language. I consider a compiler's job to be to generate the most efficient representation of the source program's semantics in terms of the target language. "Efficient" here may mean "fast" or "small" (since download speed matters), or some combination of both. A transpiler's job is to generate the most similar representation of the source program's semantics in terms of the target. This means preserving function structure, control flow constructs, variable names, and comments whenever possible. The goal is to have as many pieces of the source program recognizably appear in the target. A transpiler is much better at addressing the human factors above. However, a good one can be more difficult to write than a compiler, and in many cases you lose some runtime efficiency. Also, knowing you want to transpile often constrains the design of your source language. (For example, this is one reason CoffeeScript is so similar to JS.)
patio11 11y ago
The chief difficulty is more social than technical: the description of a language is a Schelling Point (https://en.wikipedia.org/wiki/Focal_point_(game_theory) https://en.wikipedia.org/wiki/Focal_point_(game_theory)). Satisfying computers on the other end of the transformation doesn't necessarily help you satisfy people on both ends of the transformation. Are you in web development? If so, I rate it as "Extraordinarily likely" you will use transpilers in the future or have done so already. Common ones which people would generally agree with me "Yep, that's a transpiler" include Sass, CoffeeScript, and Babel (an ES6 to the-Javascript-modern-browsers-actually-run transpiler -- which is, by the way, quite nice). More broadly, and sidestepping a definitional debate, our day to day life is "Take a program state described in Language X then shoot it down the pipe in three other languages and pray the virtual machine knows what it is doing." Random note: you can totally write a compiler. Writing a really compelling one is a social and technological challenge, but writing a limited one for pedagogical or personal/business use is roughly "an undergrad lab assignment" in terms of difficulty level. I think many engineers get scared by words like "compiler" and "emulator" and "virtual machine" and "programming language" and "operating system" and forget that these are the kind of things that talented-but-very-much-mortal solo developers can ship in weeks for fun.
Slump 11y ago
I've been using been using type script (https://github.com/Microsoft/TypeScript https://github.com/Microsoft/TypeScript) quite extensively for enterprise projects and have been enjoying it immensely. Transpilers are becoming quite the norm in web development circles, I know very few people dealing with pure CSS these days. Nearly everyone I'm in contact with uses either LESS or SASS.

dagw 11y ago

I use two transpilers in my day to day work, cython and coffeescript. In both cases they work really well and save me a bunch of time and effort. However in both cases the transpiler is written in parallel with the source language which always makes it a lot easier and the transpiler obviously supports 100% of the source language. I have also played with transpilers written for 'foreign' languages and then you do end up in a situation where you can only use a (often badly defined) subset of the source language, and debugging is a pain since the output is often very difficult to read. Still they're handy when you need them and can (sometimes) save you time compared to porting the code by hand.

Arnt 11y ago

Can't point to any resource... but my experience is that the main problems are a) parsing the input language and in the same way as other tools do and b) giving useful diagnostics and making the next-step compiler's and/or debugger's diagnostics useful. Far from trivial, but not insurmountable.

chii 11y ago

one of the most popular transpiler is http://haxe.org http://haxe.org - it compiles down to many different languages, and it works pretty well.

tomjen3 11y ago

Transpilers are just a new word for compilers - the major difference is that the target language is a high level one, but that just means that it is a lot easier to write the compiler (no optimizing register assignment!). I doubt a transpiler has any limitations that doesn't apply to compilers in general, though of course any particular one might.

jerf 11y ago

In the end, a "transpiler" is just a compiler and subject to the same limitations. There is no generic set of limitations that apply to all compilers. Some are very good because they do something relatively easy, like Coffeescript -> JavaScript. I don't mean that writing a great Coffeescript compiler [1] is necessarily "easy", but the Coffeescript language was tightly designed to map directly to Javascript constructs, so there's hardly any "limitations" at all; very nearly the entire target language is available to you, and to the extent that there may be JS you can't generate, it's probably JS the community generally agrees is a bad idea anyhow. Others, like gcc or LLVM, are actually bridging a fairly large gulf now between the source and target languages [2], but do a great job because of massive piles of effort by incredibly smart people. But if you did have to make generalizations, the two big ones are that generally you can have a performance problem if you're forced to compile something onto a target with a very different paradigm, and the target just doesn't let you have a low-enough level view of things to implement the source language's abstractions efficiently. For instance, consider the languages targeting the JVM before invokedynamic was added to the JVM... there was a fundamental paradigm mismatch that bled away a certain unavoidable amount of performance. The other problem you can get is that you may not get slick access to the low-level details of the target language; for instance, take a look at the piles of things that compile to JavaScript and compare how they allow you to use jQuery. It's anything from "What's the problem?" in CoffeeScript to being a full-on Foreign Function Interface-type call in GHCJS (even if you can get it pre-wrapped for you [3]), a Haskell->Javascript compiler. Both are really two aspects of the same "impedance mismatch" problem. [1]: Look, I'll be honest, I consider the term "transpiler" completely unnecessary. It's a compiler. Feel free to just keep using transpiler in your reply and I won't say anything again. I've had this argument, I bow to the trend, but this old fogey doesn't intend to change (and still considers it a serious impediment to understanding to believe they are two different things, despite using the exact same techniques, architecture, tools, mental models... grumble grumble). [2]: My recent previous comment goes into more detail on that claim: https://news.ycombinator.com/item?id=9800231 https://news.ycombinator.com/item?id=9800231 [3] https://github.com/ghcjs/ghcjs-jquery https://github.com/ghcjs/ghcjs-jquery

munificent 11y ago
> Look, I'll be honest, I consider the term "transpiler" > completely unnecessary. I'm part of the Dart team, so transpiling is my bread and butter. I do think it is important to make a distinction between a transpiler and a compiler. If you only consider the runtime behavior of the resulting code, sure, the two are roughly equivalent. But, that totally ignores the human factors involved. 1. You may need to inspect the ___piler output to find bugs in it. 2. A user may read the ___piler output to better understand the semantics of the source language. "Oh, ___ in CoffeeScript means ___ in JS!". 3. The user will need to debug their program. Unless you have very sophisticated debugging infrastructure (hint: source maps are not good enough), then that means they will be stepping through the ___piler output, not the original source. 4. A user may end up discarding the original source and hand-maintaining the ___piled output as the new source of truth. (This use case is exactly what the blog post is about.) Even if a user doesn't actually do this, they may want the psychological security of knowing that they could before they are willing to adopt the language. I consider a compiler's job to be to generate the most efficient representation of the source program's semantics in terms of the target language. "Efficient" here may mean "fast" or "small" (since download speed matters), or some combination of both. A transpiler's job is to generate the most similar representation of the source program's semantics in terms of the target. This means preserving function structure, control flow constructs, variable names, and comments whenever possible. The goal is to have as many pieces of the source program recognizably appear in the target. A transpiler is much better at addressing the human factors above. However, a good one can be more difficult to write than a compiler, and in many cases you lose some runtime efficiency. Also, knowing you want to transpile often constrains the design of your source language. (For example, this is one reason CoffeeScript is so similar to JS.)

patio11 11y ago

The chief difficulty is more social than technical: the description of a language is a Schelling Point (https://en.wikipedia.org/wiki/Focal_point_(game_theory) https://en.wikipedia.org/wiki/Focal_point_(game_theory)). Satisfying computers on the other end of the transformation doesn't necessarily help you satisfy people on both ends of the transformation. Are you in web development? If so, I rate it as "Extraordinarily likely" you will use transpilers in the future or have done so already. Common ones which people would generally agree with me "Yep, that's a transpiler" include Sass, CoffeeScript, and Babel (an ES6 to the-Javascript-modern-browsers-actually-run transpiler -- which is, by the way, quite nice). More broadly, and sidestepping a definitional debate, our day to day life is "Take a program state described in Language X then shoot it down the pipe in three other languages and pray the virtual machine knows what it is doing." Random note: you can totally write a compiler. Writing a really compelling one is a social and technological challenge, but writing a limited one for pedagogical or personal/business use is roughly "an undergrad lab assignment" in terms of difficulty level. I think many engineers get scared by words like "compiler" and "emulator" and "virtual machine" and "programming language" and "operating system" and forget that these are the kind of things that talented-but-very-much-mortal solo developers can ship in weeks for fun.

Slump 11y ago

I've been using been using type script (https://github.com/Microsoft/TypeScript https://github.com/Microsoft/TypeScript) quite extensively for enterprise projects and have been enjoying it immensely. Transpilers are becoming quite the norm in web development circles, I know very few people dealing with pure CSS these days. Nearly everyone I'm in contact with uses either LESS or SASS.