4 ms·

Live coding a basic Go compiler with LLVM in 20 minutes

115 points by ngaut 8y ago

rawland 8y ago
Does a recording of this talk exist? Ran `go run .` and it ended with: [...] go: downloading github.com/mewmew/float v0.0.0-20181121163145-c0f786d7da73 go: downloading github.com/pkg/errors v0.8.0 go: downloading github.com/rickypai/natsort v0.0.0-20180124032556-f194e6bd5b0c @$const_str = global [16 x i8] c"Hello world %d\0A\00" declare void @printf(...) define void @main() { 0: %0 = add i64 30, 12 call void (...) @printf([16 x i8]* @$const_str, i64 %0) ret void } 0>
- richardjennings 8y ago
  The make file provides some additional utility https://github.com/pwaller/go2ll-talk/blob/master/Makefile https://github.com/pwaller/go2ll-talk/blob/master/Makefile
- peterwaller 8y ago
  Unfortunately there isn't a recording, sorry! What you're seeing is the LLVM IR for the program being printed. As another commenter noted, you can compile the program with make, which just runs `clang` on that IR.
- loeg 8y ago
  Minor nitpick: "%d" format string for i64 isn't quite right on most (all?) platforms.
  - pstuart 8y ago
    How so?
    - loeg 8y ago
      %d is for regular 'int' sized arguments. Most popular platforms have a 32-bit or smaller 'int'. i64 refers to a 64-bit int. For that you would use '%ld' or '%lld' depending on if long or long long was 64-bit. (Or '"%" PRId64' if you really like the inttypes.h syntax.)
  - peterwaller 8y ago
    Thanks for pointing this out, I fixed it.
awirth 8y ago
Note this uses golang.org/x/tools/go/packages and golang.org/x/tools/go/ssa to get to the go SSA AST and then generates some LLVM IR from that. It's not implementing a go parser or the transformation to SSA.
- peterwaller 8y ago
  Indeed, I had to take certain liberties in order to cram the talk into 20 minutes! I hope one of the takeaways was that you don't need to implement a parser to get started playing with this sort of thing.
  - UncleEntity 8y ago
    AFAICT parsing is the easy part, admittedly more than 20 minutes but... (Also AFAICT) Go was specifically designed for easy lexing/parsing as part of their core principals. After you get from text->AST is when all the magic happens. I've been reading up on this off and on for the last year or so and it's less magic than breaking the problem down into (not quite) simple steps that can be studied individually. It's actually a quite fascinating field to go diving down the rabbit hole -- there's a lot of "lost" ideas from 20-30 years ago that one could take from pseudo-code to a fairly reasonable compilation pipeline without too much trouble. True, it wouldn't win on any benchmark test against something like LLVM with their millions of person hours invested in the optimation phases but it would be reasonable none the less.
eternalban 8y ago
somewhat OT but: ended up in the 'go/constant' package's source from there somehow and there saw the comment on "Prevent external implementations" [1] regarding using a non-exported function in the definition of 'constant.Value' interface. Here is an external implementation of constant.Value: https://play.golang.org/p/3dVFCT2-kN8 https://play.golang.org/p/3dVFCT2-kN8 [1]: https://golang.org/src/go/constant/value.go?#L58 https://golang.org/src/go/constant/value.go?#L58

115 points by ngaut 8y ago

Does a recording of this talk exist? Ran `go run .` and it ended with: [...] go: downloading github.com/mewmew/float v0.0.0-20181121163145-c0f786d7da73 go: downloading github.com/pkg/errors v0.8.0 go: downloading github.com/rickypai/natsort v0.0.0-20180124032556-f194e6bd5b0c @$const_str = global [16 x i8] c"Hello world %d\0A\00" declare void @printf(...) define void @main() { 0: %0 = add i64 30, 12 call void (...) @printf([16 x i8]* @$const_str, i64 %0) ret void } 0>

richardjennings 8y ago
The make file provides some additional utility https://github.com/pwaller/go2ll-talk/blob/master/Makefile https://github.com/pwaller/go2ll-talk/blob/master/Makefile
peterwaller 8y ago
Unfortunately there isn't a recording, sorry! What you're seeing is the LLVM IR for the program being printed. As another commenter noted, you can compile the program with make, which just runs `clang` on that IR.
loeg 8y ago
Minor nitpick: "%d" format string for i64 isn't quite right on most (all?) platforms.
- pstuart 8y ago
  How so?
  - loeg 8y ago
    %d is for regular 'int' sized arguments. Most popular platforms have a 32-bit or smaller 'int'. i64 refers to a 64-bit int. For that you would use '%ld' or '%lld' depending on if long or long long was 64-bit. (Or '"%" PRId64' if you really like the inttypes.h syntax.)
- peterwaller 8y ago
  Thanks for pointing this out, I fixed it.

awirth 8y ago

Note this uses golang.org/x/tools/go/packages and golang.org/x/tools/go/ssa to get to the go SSA AST and then generates some LLVM IR from that. It's not implementing a go parser or the transformation to SSA.

peterwaller 8y ago
Indeed, I had to take certain liberties in order to cram the talk into 20 minutes! I hope one of the takeaways was that you don't need to implement a parser to get started playing with this sort of thing.
- UncleEntity 8y ago
  AFAICT parsing is the easy part, admittedly more than 20 minutes but... (Also AFAICT) Go was specifically designed for easy lexing/parsing as part of their core principals. After you get from text->AST is when all the magic happens. I've been reading up on this off and on for the last year or so and it's less magic than breaking the problem down into (not quite) simple steps that can be studied individually. It's actually a quite fascinating field to go diving down the rabbit hole -- there's a lot of "lost" ideas from 20-30 years ago that one could take from pseudo-code to a fairly reasonable compilation pipeline without too much trouble. True, it wouldn't win on any benchmark test against something like LLVM with their millions of person hours invested in the optimation phases but it would be reasonable none the less.

eternalban 8y ago

somewhat OT but: ended up in the 'go/constant' package's source from there somehow and there saw the comment on "Prevent external implementations" [1] regarding using a non-exported function in the definition of 'constant.Value' interface. Here is an external implementation of constant.Value: https://play.golang.org/p/3dVFCT2-kN8 https://play.golang.org/p/3dVFCT2-kN8 [1]: https://golang.org/src/go/constant/value.go?#L58 https://golang.org/src/go/constant/value.go?#L58