6 ms·

Implementing a JIT Compiled Language with Haskell and LLVM

207 points by rwosync 13y ago

abc_lisper 13y ago
Very nice!
ericfode12 13y ago
So much awesome!!!
carterschonwald 13y ago
llvm-general is a great library, it is seriously the best high level api for LLVM usage currently on the planet. I'm very very grateful that Stephen Diehl took the time to port the Kaleidoscope tutorial to LLVM-General
pjmlp 13y ago
Nice work! This will be a nice addition to my compiler design documentation. Many thanks for the effort.
raphaelj 13y ago
I did a compiler for a subset of the C programming language (int & bool types, arrays, functions, loops, ...) for my post-graduate compiler course last year : https://github.com/RaphaelJ/Cours-passerelle/blob/master/Compilateur/ https://github.com/RaphaelJ/Cours-passerelle/blob/master/Com... (comments are in French) It was amazing : the full compiler was around 650 lines of code while only using Parsec as a third party library and was able to compile correctly at the first successful compilation. Compiling to LLVM (as text, I didn't used any library for code generation) using monads and Text-Lazy was also pretty pleasant. My only deception was to be unable to use GADTs : these are awesome but so hard to use with a parser.
- frik 13y ago
  Wow, cool. Thanks for sharing the link! (the code is in English, only the comments are in French) One rather minor question: Is writing comments or even code in the local language still common in France? Afaik, in most European countries the CS courses are taught in English.
  - pjmlp 13y ago
    > Afaik, in most European countries the CS courses are taught in English. Not really. You only write stuff in English if it intended for publication like papers and stuff.
  - thu 13y ago
    He is from Belgium, and it is not common. Maybe it was a requirement for his course.
  - Dewie 13y ago
    All the literature, programming libraries etc. have corrupted my brain to the point that I write my code in English when it's only for my own eyes, even though it isn't my native tongue.
    - pjmlp 13y ago
      I do the same. :)
  - raphaelj 13y ago
    I mostly write my assignments in French. It just feels "strange" to me to write comments (which are just real sentences) in English when anyone which will read the thing shares my native language. When I'm comment complex things (like some parts of this compiler), I feel it easier and more comprehensive to explain them in my mother tongue. Anyway, I write all of my "public code" in English. The audience is different.
- evincarofautumn 13y ago
  Re. GADTs, you can parse to an ADT then have a separate function to add the extra type information. The problem is that you want to add static information to dynamic input, so you need to handle the error cases.
  - deleted 13y ago
    [deleted]
  - raphaelj 13y ago
    Yep. I tought about it, but I was a few days from the deadline date, and this will add a significant amount of work. Anyway, I was dissatisfied of that. GADTs are awesome to add some "proofness" to a type checker.
TempleOSV2 13y ago
I made a complete AOT/JIT compiler from and back end in 20,000 lines of code. It can assemble, as well.
valtron 13y ago
Also worth reading: http://llvm.lyngvig.org/Articles/Mapping-High-Level-Constructs-to-LLVM-IR http://llvm.lyngvig.org/Articles/Mapping-High-Level-Construc...
strictfp 13y ago
Thank you very much for sharing!
egonschiele 13y ago
Holy cow. I tried using the llvm library a year ago to do something similar, and found it very hard to use. I can't believe someone else did it, and was nice enough to do a detailed writeup on it!
jheriko 13y ago
windows version? not everyone is a CS student or academic... making language tools in windows is actually quite challenging due to the poor quality of tools. its the only thing i've ever struggled with in that area. using haskell and llvm makes it a little easier than using C and llvm but not much... it would be nice to have a windows friendly set of instructions - setting up GHC, happy, alex etc. isn't hard, but LLVM is a pain for windows. it would be nice if it was in C rather than Haskell too. imo its highly likely nobody wants to touch Haskell if they have a choice. then the many C tools one might want to use like flex, bison - even win-flex and win-bison are a bit rubbish and difficult to configure for use in windows... still a very useful article - its great to see an example worked through like this, its one of the best forms of documentation imo and not enough of that around in this area...
- billrobertson42 13y ago
  Virtualbox runs just fine on Windows, and it's not difficult to set up a linux VM either.
- exDM69 13y ago
  > it would be nice if it was in C rather than Haskell too. imo its highly likely nobody wants to touch Haskell if they have a choice. This is roughly a "port" of the LLVM "Kaleidoscope" tutorial, which is originally written in C++. There's also an O'Caml version. Haskell is (not coincidentally) an excellent language for writing compilers. "nobody wants to touch Haskell" is just trolling that earned you a downvote.
  - jheriko 13y ago
    oh sure, there is a nice mapping between parser combinators and lbnf, however most programmers have little experience of fp and don't want to touch haskell for anything. this might be unpopular, but it has the unfortunate property of being a true statement. i'm happy to be downvoted for 'trolling' i had seen that kaleidoscope tutorial in the LLVM docs before but didn't remember it particularly or find it helpful - perhaps its the presentation but this site is much more polished than that part of the llvm docs. granted they make the explicit choice to not use lexer/parser generators but that is not as practical in c/c++ as it is with haskell and adds a lot of needless cruft imo.
exDM69 13y ago
Here's my toy programming language compiler written in Haskell using LLVM. It has classical Hindley-Milner type inference and a limited form of "lambda" closures using LLVM's trampoline intrinsic. It's not in a state where the language would actually be useful for useful, and it's only a learning project for me. I tried to suggest this project as the topic of my master's thesis but my prof thought that it wasn't a good subject. https://github.com/rikusalminen/funfun https://github.com/rikusalminen/funfun
mamcx 13y ago
Interesting. However, I still don't found good info in how do the rest of the work: GC, debugger, exception..

207 points by rwosync 13y ago

abc_lisper 13y ago

Very nice!

ericfode12 13y ago

So much awesome!!!

carterschonwald 13y ago

llvm-general is a great library, it is seriously the best high level api for LLVM usage currently on the planet. I'm very very grateful that Stephen Diehl took the time to port the Kaleidoscope tutorial to LLVM-General

pjmlp 13y ago

Nice work! This will be a nice addition to my compiler design documentation. Many thanks for the effort.

raphaelj 13y ago

I did a compiler for a subset of the C programming language (int & bool types, arrays, functions, loops, ...) for my post-graduate compiler course last year : https://github.com/RaphaelJ/Cours-passerelle/blob/master/Compilateur/ https://github.com/RaphaelJ/Cours-passerelle/blob/master/Com... (comments are in French) It was amazing : the full compiler was around 650 lines of code while only using Parsec as a third party library and was able to compile correctly at the first successful compilation. Compiling to LLVM (as text, I didn't used any library for code generation) using monads and Text-Lazy was also pretty pleasant. My only deception was to be unable to use GADTs : these are awesome but so hard to use with a parser.

frik 13y ago
Wow, cool. Thanks for sharing the link! (the code is in English, only the comments are in French) One rather minor question: Is writing comments or even code in the local language still common in France? Afaik, in most European countries the CS courses are taught in English.
- pjmlp 13y ago
  > Afaik, in most European countries the CS courses are taught in English. Not really. You only write stuff in English if it intended for publication like papers and stuff.
- thu 13y ago
  He is from Belgium, and it is not common. Maybe it was a requirement for his course.
- Dewie 13y ago
  All the literature, programming libraries etc. have corrupted my brain to the point that I write my code in English when it's only for my own eyes, even though it isn't my native tongue.
  - pjmlp 13y ago
    I do the same. :)
- raphaelj 13y ago
  I mostly write my assignments in French. It just feels "strange" to me to write comments (which are just real sentences) in English when anyone which will read the thing shares my native language. When I'm comment complex things (like some parts of this compiler), I feel it easier and more comprehensive to explain them in my mother tongue. Anyway, I write all of my "public code" in English. The audience is different.
evincarofautumn 13y ago
Re. GADTs, you can parse to an ADT then have a separate function to add the extra type information. The problem is that you want to add static information to dynamic input, so you need to handle the error cases.
- deleted 13y ago
  [deleted]
- raphaelj 13y ago
  Yep. I tought about it, but I was a few days from the deadline date, and this will add a significant amount of work. Anyway, I was dissatisfied of that. GADTs are awesome to add some "proofness" to a type checker.

TempleOSV2 13y ago

I made a complete AOT/JIT compiler from and back end in 20,000 lines of code. It can assemble, as well.

valtron 13y ago

Also worth reading: http://llvm.lyngvig.org/Articles/Mapping-High-Level-Constructs-to-LLVM-IR http://llvm.lyngvig.org/Articles/Mapping-High-Level-Construc...

strictfp 13y ago

Thank you very much for sharing!

egonschiele 13y ago

Holy cow. I tried using the llvm library a year ago to do something similar, and found it very hard to use. I can't believe someone else did it, and was nice enough to do a detailed writeup on it!

jheriko 13y ago

windows version? not everyone is a CS student or academic... making language tools in windows is actually quite challenging due to the poor quality of tools. its the only thing i've ever struggled with in that area. using haskell and llvm makes it a little easier than using C and llvm but not much... it would be nice to have a windows friendly set of instructions - setting up GHC, happy, alex etc. isn't hard, but LLVM is a pain for windows. it would be nice if it was in C rather than Haskell too. imo its highly likely nobody wants to touch Haskell if they have a choice. then the many C tools one might want to use like flex, bison - even win-flex and win-bison are a bit rubbish and difficult to configure for use in windows... still a very useful article - its great to see an example worked through like this, its one of the best forms of documentation imo and not enough of that around in this area...

billrobertson42 13y ago
Virtualbox runs just fine on Windows, and it's not difficult to set up a linux VM either.
exDM69 13y ago
> it would be nice if it was in C rather than Haskell too. imo its highly likely nobody wants to touch Haskell if they have a choice. This is roughly a "port" of the LLVM "Kaleidoscope" tutorial, which is originally written in C++. There's also an O'Caml version. Haskell is (not coincidentally) an excellent language for writing compilers. "nobody wants to touch Haskell" is just trolling that earned you a downvote.
- jheriko 13y ago
  oh sure, there is a nice mapping between parser combinators and lbnf, however most programmers have little experience of fp and don't want to touch haskell for anything. this might be unpopular, but it has the unfortunate property of being a true statement. i'm happy to be downvoted for 'trolling' i had seen that kaleidoscope tutorial in the LLVM docs before but didn't remember it particularly or find it helpful - perhaps its the presentation but this site is much more polished than that part of the llvm docs. granted they make the explicit choice to not use lexer/parser generators but that is not as practical in c/c++ as it is with haskell and adds a lot of needless cruft imo.

exDM69 13y ago

Here's my toy programming language compiler written in Haskell using LLVM. It has classical Hindley-Milner type inference and a limited form of "lambda" closures using LLVM's trampoline intrinsic. It's not in a state where the language would actually be useful for useful, and it's only a learning project for me. I tried to suggest this project as the topic of my master's thesis but my prof thought that it wasn't a good subject. https://github.com/rikusalminen/funfun https://github.com/rikusalminen/funfun

mamcx 13y ago

Interesting. However, I still don't found good info in how do the rest of the work: GC, debugger, exception..