5 ms·

Runtime code generation and execution in Go

160 points by ingve 2y ago

a-french-anon 2y ago
[flagged]
- ncruces 2y ago
  So your point is that a Lisp that ships a JIT will help you implement a "copy-and-patch JIT" because "macros" and "code is data"? Let me point out that Go neither ships a JIT, nor has macros, so I really don't see how that's relevant insight for the Go ecosystem.
  - pjmlp 2y ago
    Go has poor man's macros via //go:generate, and having a JIT available that is an implementation detail, e.g. Yaegi could support a JIT if they feel like it.
    - ncruces 2y ago
      I know. But you're not running //go:generate at runtime, and for Yaegi to gain a JIT... well, they'd have to implement much of what's in the blog. That's the point. Mind you, I wouldn't mind having a something that translates Wasm to Go at compile time, and ship Go instead, for the cases where the Wasm doesn't change. But people also load Wasm at runtime, and for that to work, you need the (Wasm) compiler available at runtime, and there just aren't that many such compilers for the Go ecosystem.
    - jerf 2y ago
      go:generate is not macro support. It is just the ability to specify a command to run in the source code when you run "go generate". Having been around this loop multiple times now with people, here is the source code for go:generate: https://github.com/golang/go/blob/master/src/cmd/go/internal/generate/generate.go#L497 https://github.com/golang/go/blob/master/src/cmd/go/internal... I've highlighted the line where it executes the given command. This is not aimed at you specifically, pjmlp, but anyone in general who would like to vigorously assert go generate really is a macro facility is invited to point me at even a single line that involves such a thing in the source code. The only slightly useful thing it does is on line 366, where it will set some conceivably useful environment variables for the command it runs. But otherwise, this is "go generate": You add: //go:generate echo hello! and when you type "go generate" in the shell, go runs "echo hello!" for you. That's it. While you could conceivably use it to generate macros, you will be bringing 100% of the macro execution code to go generate; it provides no help. The closest thing Go has to built-in macro support is that it does ship with full AST parsing, some code built on top of that, and it can re-emit code based on your changes. (I don't target this at you pjmlp because I'm pretty sure from previous interactions you are not a Go programmer, and a disturbingly large portion of the Go community itself vigorously and emotionally believes that go generate is a macro generation facility, so how were you to know? People seem to have a very, very hard idea with a feature being labelled "generate" that in fact generates nothing.)
      - pjmlp 2y ago
        No issues, that is also why I stated "poor man's", so it is even poorer than I imagined, :)
        mseepgood 2y ago
        It's not "poor man's", it's simply a completely different thing and as such not comparable and irrelevant to this topic. It's not run at runtime, it's not run at build time, it's run at development time before commit. It's like saying a house is a poor man's apple.
        pjmlp 2y ago
        [flagged]
        jerf 2y ago
        I do not have a second account. I don't know what "it" is in that comment; I fear "it" is still some sort of macro capability, which doesn't exist, at dev time, compile time, run time, or otherwise. Like I said, the idea that "go generate" is some sort of macro language other than a thin wrapper around a command to run is very common. I suppose it would be interesting to try to get people to describe this macro language that supposedly exists. What's the macro look like to transparently wrap a string in a print statement for debugging?
        pjmlp 2y ago
        No worries, I knew that, it was more a jab to those kind of remarks that don't help anyone. Sorry if you got annoyed with your reference.
        ncruces 2y ago
        I guess the meta programming that most often gets used with //go:generate ends up being text/template. The Go sort algorithm (pdqsort) is implemented in this way so the same algorithm can be used to implement the various existing sort APIs. https://github.com/golang/go/blob/master/src/sort/gen_sort_variants.go https://github.com/golang/go/blob/master/src/sort/gen_sort_v...
  - a-french-anon 2y ago
    CL compilers aren't really JIT, they're AOT with user access to the compiler; yes, I know the distinction is blurry in this specific case (https://news.ycombinator.com/item?id=24387664 https://news.ycombinator.com/item?id=24387664). Macros and homoiconicity have absolutely nothing to do with my example. The fact is that go provides a compiler, so it should allow one to embed it in the final binary to use it at runtime.
deleted 2y ago
[deleted]
WhyNotHugo 2y ago
I found this article very approachable and well written. Thanks for sharing.
fl0ki 2y ago
I didn't know Go reserved some registers for its own use. For libraries that don't use cgo but do use assembly [1], does this mean they have to generate that assembly with a compiler flag that avoids reserved registers? [1] A common optimization technique, like in the superb https://github.com/klauspost/compress https://github.com/klauspost/compress
- jhoechtl 2y ago
  Pusha popa and never yield back in between should suffice?
  - fl0ki 2y ago
    Non-CGO goroutines are preemptible via signals so there may still be more to it than that. If anyone knows the precise mechanism used I'd be very keen to hear it.
- yencabulator 2y ago
  Generally the inline assembly is hand-written and must follow the platform rules, see "Architecture-specific details" in https://go.dev/doc/asm https://go.dev/doc/asm The few tools that generate that asm, such as https://github.com/mmcloughlin/avo https://github.com/mmcloughlin/avo , would also have follow the rules. You're not meant to take C source, crank it through GCC, and slurp that in as inline asm in Go. Background: https://www.youtube.com/watch?v=KINIAgRpkDA https://www.youtube.com/watch?v=KINIAgRpkDA
evacchi 2y ago
@ncruces and I are also contributors to wazero, so ask away if you have questions :)
neonsunset 2y ago
[flagged]
- ncruces 2y ago
  Comments extolling the virtues of LISP or the CLR in a Go thread are "not negative about Go"; they're simply off-topic. An on-topic, comment critical of Go would be: it's silly that cgo is considered so bad (why?) that people go to such great lengths to avoid it, when so many (otherwise great) Wasm runtimes already exist. But if I'm already using Go and need a Wasm runtime, my options are wazero or wasmtime-go (etc), not LISP and C#.
  - raggi 2y ago
    cgo is extremely slow, particularly in programs with many active goroutines. I regularly look sad at profiles of production cgo programs where findRunnable is in the top 10.
    - metadat 2y ago
      I thought Go could handle a million-plus active goroutines? Are you saying it'll work but be poor performance compared to something equivalent in C++, JavaScript, or Rust? A quick search reveals cgo is mainly slow due to overhead when invoking non-pure go functions (i.e. C-functions). https://stackoverflow.com/questions/28272285/why-cgos-performance-is-so-slow-is-there-something-wrong-with-my-testing-code https://stackoverflow.com/questions/28272285/why-cgos-perfor... How does this relate and interact with goroutines in a performance impacting way? As in, why is the case with goroutines any different? Edit: @neonsunset: Thanks <3. The thread you linked covers erl/BEAM and C#/dotnet, and makes salient points of skepticism about the practical need for millions of routines. I'm still curious what makes the Go story worse or different?
      - neonsunset 2y ago
        Related: https://news.ycombinator.com/item?id=40435220 https://news.ycombinator.com/item?id=40435220 (C# can handle millions of active tasks, and so can Elixir, though with greater overhead as it offers a different set of tradeoffs)
metadat 2y ago
Really cool article. This sort of stuff is mind expanding for me! I wonder what it would take to make a version that works on Windows. Will you indulge me?
- ncruces 2y ago
  wazero works fine on Windows. In fact the exact same assembly works across Windows, macOS, Linux and FreeBSD: the calling convention used by the compiler is entirely custom, and all "host calls" are provided by wazero, so assembly ports fine across OSes.
  - metadat 2y ago
    Perhaps I was unclear (or I'm confused now, haha): > The following is the tiny demo of the runtime code generation and execution in Go. I assume you are on a Unix-like system like Linux or macOS on an AArch64 machine. Is there a way to get tinydemo working on Windows?
    - ncruces 2y ago
      Oh, OK. You need to replace calls to syscall.Mmap with windows.VirtualAlloc [1], syscall.Mprotect with windows.VirtualProtect, etc. You don't even need to change the ASM preamble. [1]: https://pkg.go.dev/golang.org/x/sys/windows#VirtualAlloc https://pkg.go.dev/golang.org/x/sys/windows#VirtualAlloc
      - evacchi 2y ago
        and for reference see how it's done in wazero https://github.com/tetratelabs/wazero/blob/c397a402ad17e495a5a22fef709a1d4abef1f3ce/internal/platform/mmap_windows.go https://github.com/tetratelabs/wazero/blob/c397a402ad17e495a...
kgeist 2y ago
I wonder how safe the their (wazero's) approach is from the security point of view. According to the article (and I know it myself too after studying the Go runtime), the Go runtime is quite finicky when it comes to execution of foreign code because it can easily interfer with the goroutine scheduler, GC, etc. (unless you use CGO which deals with it, but they don't use it). As the article explains, it's easy to introduce random crashes by, say, innocently using a "wrong" register. Also, the Go runtime can change considerably from version to version (say, when they introduced preempted scheduling based on signals, or when they moved away from segmented stacks etc.), and so today Wazero may work OK but with a new Go version it may unexpectedly crash or corrupt memory in some subtle ways. Considering that the whole point of WASM is sandboxing, reading this article didn't make me feel very confident about the project.
- ncruces 2y ago
  If it gives you some confidence, wazero (the Go Wasm runtime) is used by the Go team (in CI) to test their own Go-to-Wasm compiler. So if they break wazero, they'll know pretty soon from failing tests of their own Wasm/WASI toolchain. https://go.dev/blog/wasi https://go.dev/blog/wasi https://github.com/golang/go/blob/go1.22.3/misc/wasm/go_wasip1_wasm_exec https://github.com/golang/go/blob/go1.22.3/misc/wasm/go_wasi... As for sandboxing, wazero tends to take sandboxing rather more seriously than other similar runtimes. Memory sandboxing is implemented through explicit bounds checks (rather than memory mapping and guard pages), which allows denser deployments and requires less runtime support. And, by default, the WASI implementation doesn't expose any capabilities (not even access to the system clock). You can also interject any relevant WASI host calls, and do your own filtering. And if the compiler is not your cup of tea, you can also use the (much slower) interpreter.
  - kgeist 2y ago
    That's good to hear that the Go team themselves test wazero. Thank you for clarifying.
syngrog66 2y ago
[flagged]
- zbentley 2y ago
  This is a compiler. It's a compiler that executes at a weird time (during program startup), but it's a compiler nevertheless. It's exactly as secure, or not, as its input code and the runtime in which that code runs once compiled. This isn't "code-gen" as in the xz debacle, where source code transformation was used to obfuscate malicious input. This is "code-gen" in a similar sense to what the JVM does at runtime, or what the Linux kernel does with BPF programs, or similar. While compiler toolchains aren't exempt from security considerations (especially this one, in which failing to appropriately reverse-engineer the Go runtime can result in compiled code behaving as scarily as an evil library dlopen'd at runtime in C), they're a totally different class of tool/vulnerability from what was used to exploit xz.
  - syngrog66 2y ago
    JIT is a form of code gen. My claim was about code gen in general, of which JIT is a special case. And I know how compilers work. No "well akshewally..." lecture needed or requested, thanks. And if you re-read carefully what I literally said I did not say Jia Tan used the exact same technique as the OA. You added that supposition and then, conveniently, attacked it. Straw man. FOSS (ie. rando stranger originating) software-based JIT is also a security "smell" at best. Tradeoff bewtween potentially improving perf in some cases vs increasing the amount of exploitable complexity surface for attackers. And always making it harder to reason about what code does at runtime. Less JIT and less code gen, in general, is wise in a world with growing threats from expert attackers backed by nation state level resources. Simpler is safer.
    - ncruces 2y ago
      Random stranger backed by company develops Wasm runtime in the open, that's used by Go authors themselves to test their own Go-to-Wasm compiler. I'm sure other Wasm runtimes are much better.

160 points by ingve 2y ago

a-french-anon 2y ago

[flagged]

ncruces 2y ago
So your point is that a Lisp that ships a JIT will help you implement a "copy-and-patch JIT" because "macros" and "code is data"? Let me point out that Go neither ships a JIT, nor has macros, so I really don't see how that's relevant insight for the Go ecosystem.
- pjmlp 2y ago
  Go has poor man's macros via //go:generate, and having a JIT available that is an implementation detail, e.g. Yaegi could support a JIT if they feel like it.
  - ncruces 2y ago
    I know. But you're not running //go:generate at runtime, and for Yaegi to gain a JIT... well, they'd have to implement much of what's in the blog. That's the point. Mind you, I wouldn't mind having a something that translates Wasm to Go at compile time, and ship Go instead, for the cases where the Wasm doesn't change. But people also load Wasm at runtime, and for that to work, you need the (Wasm) compiler available at runtime, and there just aren't that many such compilers for the Go ecosystem.
  - jerf 2y ago
    go:generate is not macro support. It is just the ability to specify a command to run in the source code when you run "go generate". Having been around this loop multiple times now with people, here is the source code for go:generate: https://github.com/golang/go/blob/master/src/cmd/go/internal/generate/generate.go#L497 https://github.com/golang/go/blob/master/src/cmd/go/internal... I've highlighted the line where it executes the given command. This is not aimed at you specifically, pjmlp, but anyone in general who would like to vigorously assert go generate really is a macro facility is invited to point me at even a single line that involves such a thing in the source code. The only slightly useful thing it does is on line 366, where it will set some conceivably useful environment variables for the command it runs. But otherwise, this is "go generate": You add: //go:generate echo hello! and when you type "go generate" in the shell, go runs "echo hello!" for you. That's it. While you could conceivably use it to generate macros, you will be bringing 100% of the macro execution code to go generate; it provides no help. The closest thing Go has to built-in macro support is that it does ship with full AST parsing, some code built on top of that, and it can re-emit code based on your changes. (I don't target this at you pjmlp because I'm pretty sure from previous interactions you are not a Go programmer, and a disturbingly large portion of the Go community itself vigorously and emotionally believes that go generate is a macro generation facility, so how were you to know? People seem to have a very, very hard idea with a feature being labelled "generate" that in fact generates nothing.)
    - pjmlp 2y ago
      No issues, that is also why I stated "poor man's", so it is even poorer than I imagined, :)
      - mseepgood 2y ago
        It's not "poor man's", it's simply a completely different thing and as such not comparable and irrelevant to this topic. It's not run at runtime, it's not run at build time, it's run at development time before commit. It's like saying a house is a poor man's apple.
        pjmlp 2y ago
        [flagged]
        jerf 2y ago
        I do not have a second account. I don't know what "it" is in that comment; I fear "it" is still some sort of macro capability, which doesn't exist, at dev time, compile time, run time, or otherwise. Like I said, the idea that "go generate" is some sort of macro language other than a thin wrapper around a command to run is very common. I suppose it would be interesting to try to get people to describe this macro language that supposedly exists. What's the macro look like to transparently wrap a string in a print statement for debugging?
        pjmlp 2y ago
        No worries, I knew that, it was more a jab to those kind of remarks that don't help anyone. Sorry if you got annoyed with your reference.
        ncruces 2y ago
        I guess the meta programming that most often gets used with //go:generate ends up being text/template. The Go sort algorithm (pdqsort) is implemented in this way so the same algorithm can be used to implement the various existing sort APIs. https://github.com/golang/go/blob/master/src/sort/gen_sort_variants.go https://github.com/golang/go/blob/master/src/sort/gen_sort_v...
- a-french-anon 2y ago
  CL compilers aren't really JIT, they're AOT with user access to the compiler; yes, I know the distinction is blurry in this specific case (https://news.ycombinator.com/item?id=24387664 https://news.ycombinator.com/item?id=24387664). Macros and homoiconicity have absolutely nothing to do with my example. The fact is that go provides a compiler, so it should allow one to embed it in the final binary to use it at runtime.

deleted 2y ago

[deleted]

WhyNotHugo 2y ago

I found this article very approachable and well written. Thanks for sharing.

fl0ki 2y ago

I didn't know Go reserved some registers for its own use. For libraries that don't use cgo but do use assembly [1], does this mean they have to generate that assembly with a compiler flag that avoids reserved registers? [1] A common optimization technique, like in the superb https://github.com/klauspost/compress https://github.com/klauspost/compress

jhoechtl 2y ago
Pusha popa and never yield back in between should suffice?
- fl0ki 2y ago
  Non-CGO goroutines are preemptible via signals so there may still be more to it than that. If anyone knows the precise mechanism used I'd be very keen to hear it.
yencabulator 2y ago
Generally the inline assembly is hand-written and must follow the platform rules, see "Architecture-specific details" in https://go.dev/doc/asm https://go.dev/doc/asm The few tools that generate that asm, such as https://github.com/mmcloughlin/avo https://github.com/mmcloughlin/avo , would also have follow the rules. You're not meant to take C source, crank it through GCC, and slurp that in as inline asm in Go. Background: https://www.youtube.com/watch?v=KINIAgRpkDA https://www.youtube.com/watch?v=KINIAgRpkDA

evacchi 2y ago

@ncruces and I are also contributors to wazero, so ask away if you have questions :)

neonsunset 2y ago

[flagged]

ncruces 2y ago
Comments extolling the virtues of LISP or the CLR in a Go thread are "not negative about Go"; they're simply off-topic. An on-topic, comment critical of Go would be: it's silly that cgo is considered so bad (why?) that people go to such great lengths to avoid it, when so many (otherwise great) Wasm runtimes already exist. But if I'm already using Go and need a Wasm runtime, my options are wazero or wasmtime-go (etc), not LISP and C#.
- raggi 2y ago
  cgo is extremely slow, particularly in programs with many active goroutines. I regularly look sad at profiles of production cgo programs where findRunnable is in the top 10.
  - metadat 2y ago
    I thought Go could handle a million-plus active goroutines? Are you saying it'll work but be poor performance compared to something equivalent in C++, JavaScript, or Rust? A quick search reveals cgo is mainly slow due to overhead when invoking non-pure go functions (i.e. C-functions). https://stackoverflow.com/questions/28272285/why-cgos-performance-is-so-slow-is-there-something-wrong-with-my-testing-code https://stackoverflow.com/questions/28272285/why-cgos-perfor... How does this relate and interact with goroutines in a performance impacting way? As in, why is the case with goroutines any different? Edit: @neonsunset: Thanks <3. The thread you linked covers erl/BEAM and C#/dotnet, and makes salient points of skepticism about the practical need for millions of routines. I'm still curious what makes the Go story worse or different?
    - neonsunset 2y ago
      Related: https://news.ycombinator.com/item?id=40435220 https://news.ycombinator.com/item?id=40435220 (C# can handle millions of active tasks, and so can Elixir, though with greater overhead as it offers a different set of tradeoffs)

metadat 2y ago

Really cool article. This sort of stuff is mind expanding for me! I wonder what it would take to make a version that works on Windows. Will you indulge me?

ncruces 2y ago
wazero works fine on Windows. In fact the exact same assembly works across Windows, macOS, Linux and FreeBSD: the calling convention used by the compiler is entirely custom, and all "host calls" are provided by wazero, so assembly ports fine across OSes.
- metadat 2y ago
  Perhaps I was unclear (or I'm confused now, haha): > The following is the tiny demo of the runtime code generation and execution in Go. I assume you are on a Unix-like system like Linux or macOS on an AArch64 machine. Is there a way to get tinydemo working on Windows?
  - ncruces 2y ago
    Oh, OK. You need to replace calls to syscall.Mmap with windows.VirtualAlloc [1], syscall.Mprotect with windows.VirtualProtect, etc. You don't even need to change the ASM preamble. [1]: https://pkg.go.dev/golang.org/x/sys/windows#VirtualAlloc https://pkg.go.dev/golang.org/x/sys/windows#VirtualAlloc
    - evacchi 2y ago
      and for reference see how it's done in wazero https://github.com/tetratelabs/wazero/blob/c397a402ad17e495a5a22fef709a1d4abef1f3ce/internal/platform/mmap_windows.go https://github.com/tetratelabs/wazero/blob/c397a402ad17e495a...

kgeist 2y ago

I wonder how safe the their (wazero's) approach is from the security point of view. According to the article (and I know it myself too after studying the Go runtime), the Go runtime is quite finicky when it comes to execution of foreign code because it can easily interfer with the goroutine scheduler, GC, etc. (unless you use CGO which deals with it, but they don't use it). As the article explains, it's easy to introduce random crashes by, say, innocently using a "wrong" register. Also, the Go runtime can change considerably from version to version (say, when they introduced preempted scheduling based on signals, or when they moved away from segmented stacks etc.), and so today Wazero may work OK but with a new Go version it may unexpectedly crash or corrupt memory in some subtle ways. Considering that the whole point of WASM is sandboxing, reading this article didn't make me feel very confident about the project.

ncruces 2y ago
If it gives you some confidence, wazero (the Go Wasm runtime) is used by the Go team (in CI) to test their own Go-to-Wasm compiler. So if they break wazero, they'll know pretty soon from failing tests of their own Wasm/WASI toolchain. https://go.dev/blog/wasi https://go.dev/blog/wasi https://github.com/golang/go/blob/go1.22.3/misc/wasm/go_wasip1_wasm_exec https://github.com/golang/go/blob/go1.22.3/misc/wasm/go_wasi... As for sandboxing, wazero tends to take sandboxing rather more seriously than other similar runtimes. Memory sandboxing is implemented through explicit bounds checks (rather than memory mapping and guard pages), which allows denser deployments and requires less runtime support. And, by default, the WASI implementation doesn't expose any capabilities (not even access to the system clock). You can also interject any relevant WASI host calls, and do your own filtering. And if the compiler is not your cup of tea, you can also use the (much slower) interpreter.
- kgeist 2y ago
  That's good to hear that the Go team themselves test wazero. Thank you for clarifying.

syngrog66 2y ago

[flagged]

zbentley 2y ago
This is a compiler. It's a compiler that executes at a weird time (during program startup), but it's a compiler nevertheless. It's exactly as secure, or not, as its input code and the runtime in which that code runs once compiled. This isn't "code-gen" as in the xz debacle, where source code transformation was used to obfuscate malicious input. This is "code-gen" in a similar sense to what the JVM does at runtime, or what the Linux kernel does with BPF programs, or similar. While compiler toolchains aren't exempt from security considerations (especially this one, in which failing to appropriately reverse-engineer the Go runtime can result in compiled code behaving as scarily as an evil library dlopen'd at runtime in C), they're a totally different class of tool/vulnerability from what was used to exploit xz.
- syngrog66 2y ago
  JIT is a form of code gen. My claim was about code gen in general, of which JIT is a special case. And I know how compilers work. No "well akshewally..." lecture needed or requested, thanks. And if you re-read carefully what I literally said I did not say Jia Tan used the exact same technique as the OA. You added that supposition and then, conveniently, attacked it. Straw man. FOSS (ie. rando stranger originating) software-based JIT is also a security "smell" at best. Tradeoff bewtween potentially improving perf in some cases vs increasing the amount of exploitable complexity surface for attackers. And always making it harder to reason about what code does at runtime. Less JIT and less code gen, in general, is wise in a world with growing threats from expert attackers backed by nation state level resources. Simpler is safer.
  - ncruces 2y ago
    Random stranger backed by company develops Wasm runtime in the open, that's used by Go authors themselves to test their own Go-to-Wasm compiler. I'm sure other Wasm runtimes are much better.