6 ms·

Go, the language for emulators

69 points by geetarista 14y ago

vor_ 14y ago
"It turns out that Go is a great language for writing retro-computing emulators." Why?
cwzwarich 14y ago
If you're writing a good emulator, you're going to want it to generate machine code. Doesn't that interact poorly with Go?
- daeken 14y ago
  Actually, if you can compile Go modules at runtime, you could JIT that way. My IronBabel emulator "decompiled" the guest code to C# and then compiled that, for impressively high performance; you could do the same with Go, without the horrid compile times that I had. Combine that with some nice caching, and you're golden. But I don't know that Go has such a programatic compiler API.
  - swdunlop 14y ago
    You can, but Go doesn't really have a good way to link in the product at runtime. Maybe something like a Go version of GNU Lightning would be a fun project -- http://www.gnu.org/software/lightning/ http://www.gnu.org/software/lightning/ You can't wrap Lightning itself, otherwise any possible gains you could expect are lost at the CGO transition.
    - lcampbell 14y ago
      > You can't wrap Lightning itself What prevents wrapping lightning? (I'm not at all familiar with Lightning) > any possible gains you could expect are lost at the CGO transition I was under the impression that the CGO overhead was due to marshaling data -- could that not be avoided by representing the JIT'd code as unsafe.Pointer's and type annotations (akin to how reflect invokes reflect.Values)? Or is there a whole realm of stuff I'm missing? EDIT: Did a bit of digging; reflect.Value.Call internally uses reflect·call (defined in runtime[1]), which obviously uses the Go ABI. Since any JIT'd code we want to load will expect to be called with the C ABI, presumably, we're stuck with runtime·asmcgocall[2]. Combined with runtime·cgocall[3], it certainly is longer than a vanilla preamble in C, so there's definitely some significant overhead not related to, e.g., cgo.GoString. I don't have enough prowess to determine the actual performance impact, though :( Could be a fun weekend project. -- [1] http://code.google.com/p/go/source/browse/src/pkg/runtime/asm_386.s#229 http://code.google.com/p/go/source/browse/src/pkg/runtime/as... [2] http://code.google.com/p/go/source/browse/src/pkg/runtime/asm_386.s#437 http://code.google.com/p/go/source/browse/src/pkg/runtime/as... [3] http://code.google.com/p/go/source/browse/src/pkg/runtime/cgocall.c#100 http://code.google.com/p/go/source/browse/src/pkg/runtime/cg...
      - swdunlop 14y ago
        Interesting. It appears that cgocall has improved from the last time I went down this rabbit hole. Previously, it used a pthread_create to construct a C stack and context on each runtime.cgocall and then waited on a mutex for it to return. The overhead was enough to make me favor image/png Encode over libpng for smaller images. Time to go profiling again! :)
- ajross 14y ago
  Not necessarily. Computers are so fast these days that real time simulation of a lot of these older systems in interpreted environments (a naive C instruction interpreter, or even Javascript, etc...) is perfectly feasible. The Apple ][ had a 1MHz clock, a CPU that took 2-10 cycles per instruction, and an 8kb framebuffer that refreshed at 60Hz. That's not a lot of work to emulate.
  - mithaler 14y ago
    That depends on what your standard of accuracy is. http://arstechnica.com/gaming/2011/08/accuracy-takes-power-one-mans-3ghz-quest-to-build-a-perfect-snes-emulator/ http://arstechnica.com/gaming/2011/08/accuracy-takes-power-o...
    - ajross 14y ago
      That's a fair argument, but "accuracy" isn't really the right word. Because then you'd have to deal with the fact that the color gamut from a 1982 NTSC TV is different from your LCD. Similarly I've never seen a good simulation of interlace flicker (not an issue on the NES though -- it didn't interlace) on a LCD either. The impedance mismatch between a NES controller and a modern keyboard alone is much higher than that beteween a "naive" simulation and the one in that article. And if you want to go down the "elaborate" road, there are even niftier projects out there like the Javascript transistor-level simulation of a 6502. There's always room to spend cycles. But simulating a 1980's computer to the level of precision required by "correct user experience" is a straightforward engineering task these days that doesn't require anything more than "just plain code."
- Simucal 14y ago
  How does that interact poorly with Go? I'm just curious.
- gillianseed 14y ago
  Assuming that by 'good' you mean emulating a system which has pretty high demands on hardware when emulating (I guess ps2,gamecube,dreamcast and onwards) then yes it's very likely you will want to use dynamic recompiling (jit) to convert the emulated code from foreign machine code to native machine code and execute the native code for increased performance. I assume this is possible with Go as it has the ability to call unsafe code?
- strmpnk 14y ago
  Lots of optimized code in Go's standard library has fast paths in assembly which is linked into the final binary on a platform by platform basis. I this sense, it's probably easier than C to drop into assembly. Granted, Go's memory model is not going to enforce a whole lot so you better know what you're doing at this point.
  - azakai 14y ago
    Eh? Go doesn't have any capability to include inline assembly, > "There is no facility in the Go programming language to support in-line assembler language code, and there are no plans to do so" http://stackoverflow.com/questions/2951028/is-it-possible-to-include-inline-assembly-in-google-go-code http://stackoverflow.com/questions/2951028/is-it-possible-to... Of course the C parts of Go's standard library have optimized assembly - using C's capability to do. So Go is not easier in any way for dropping into assembly - in fact it's much harder, you can't do it in the language, at best you might be lucky if Go's library does some stuff like that for you.
    - strmpnk 14y ago
      I didn't say inline. I said linked. It's pretty easy to add.
      - azakai 14y ago
        You said "easier than C". How is linking to assembly code from Go any easier than linking to assembly code from C?
        burntsushi 14y ago
        Just go see how it's done. Look for example at how `math.Exp` is declared: http://golang.org/src/pkg/math/exp.go?s=368:395#L4 http://golang.org/src/pkg/math/exp.go?s=368:395#L4 You'll notice that 'exp' is defined below, using Go. However, 'Exp' is also defined in Assembly. For example, amd64: http://golang.org/src/pkg/math/exp_amd64.s http://golang.org/src/pkg/math/exp_amd64.s This is all compatible with the `go` tool. You just drop in your Assembly, run `go build`, and the appropriate Assembly version is chosen based on your target architecture (or the Go code is used as a fallback if no Assembly for the target architecture exists). I don't have much experience with inlining Assembly in C, but this is pretty damn easy.
shardling 14y ago
So, I don't see that the article mentions why the language is good for emulators.
- StavrosK 14y ago
  Because it's a good opportunity to ride the Go bandwagon and get some free traffic with a list of five software projects, of course...
  - drivebyacct2 14y ago
    Pffft. Dave is hardly riding the "Go bandwagon". (I guess this comment doesn't mean much unless you know what Dave does in the community or read the ML or have had help from him in the past. It's not as if he just picked this up to be "in", though I don't want to speak out of turn for him.)
    - deleted 14y ago
      [deleted]
  - 4ad 14y ago
    Dave is not riding the bandwagon, he is one of the Go developers working mostly on the SSH package, the ARM platform and compiler optimizations. He is also core developer of Juju, software written in Go.
    - StavrosK 14y ago
      Oh, I didn't know that. It's probably a weird choice of words, then.
- swdunlop 14y ago
  Off the top of my head, I can think of a few why Go is better than Python or other high level languages for writing an emulator: - Go makes it easy to write a process for your emulator, instead of a state machine. - Go's slice types makes passing blocks of memory around a zero copy operation. - Pass by value lets you reduce the number of objects GC must trace. - Go is explicit about integer size and sign, makes conversion errors more visible. It's not better than C, but it's certainly easier to work with. I'm routinely surprised at how C-friendly Go's produced code is -- take a spin through profiling some of the image library primitives and compare them against your favorite C library.
killpg 14y ago
Yawn. Here's a Space Invaders emulator from 2006 in Factor. https://github.com/slavapestov/factor/blob/master/extra/space-invaders/space-invaders.factor https://github.com/slavapestov/factor/blob/master/extra/spac...
frozenport 14y ago
The systems emulated look quite primitive and, hence, give off the opposite impression: it feels like Go is not able to make a serious emulator. I suspect there are more complex emulators in javascript.
- btrask 14y ago
  I'm not going to enter a language flame war, but I was quite impressed by n64js. http://n64js.blogspot.co.uk/ http://n64js.blogspot.co.uk/

69 points by geetarista 14y ago

vor_ 14y ago

"It turns out that Go is a great language for writing retro-computing emulators." Why?

If you're writing a good emulator, you're going to want it to generate machine code. Doesn't that interact poorly with Go?

daeken 14y ago
Actually, if you can compile Go modules at runtime, you could JIT that way. My IronBabel emulator "decompiled" the guest code to C# and then compiled that, for impressively high performance; you could do the same with Go, without the horrid compile times that I had. Combine that with some nice caching, and you're golden. But I don't know that Go has such a programatic compiler API.
- swdunlop 14y ago
  You can, but Go doesn't really have a good way to link in the product at runtime. Maybe something like a Go version of GNU Lightning would be a fun project -- http://www.gnu.org/software/lightning/ http://www.gnu.org/software/lightning/ You can't wrap Lightning itself, otherwise any possible gains you could expect are lost at the CGO transition.
  - lcampbell 14y ago
    > You can't wrap Lightning itself What prevents wrapping lightning? (I'm not at all familiar with Lightning) > any possible gains you could expect are lost at the CGO transition I was under the impression that the CGO overhead was due to marshaling data -- could that not be avoided by representing the JIT'd code as unsafe.Pointer's and type annotations (akin to how reflect invokes reflect.Values)? Or is there a whole realm of stuff I'm missing? EDIT: Did a bit of digging; reflect.Value.Call internally uses reflect·call (defined in runtime[1]), which obviously uses the Go ABI. Since any JIT'd code we want to load will expect to be called with the C ABI, presumably, we're stuck with runtime·asmcgocall[2]. Combined with runtime·cgocall[3], it certainly is longer than a vanilla preamble in C, so there's definitely some significant overhead not related to, e.g., cgo.GoString. I don't have enough prowess to determine the actual performance impact, though :( Could be a fun weekend project. -- [1] http://code.google.com/p/go/source/browse/src/pkg/runtime/asm_386.s#229 http://code.google.com/p/go/source/browse/src/pkg/runtime/as... [2] http://code.google.com/p/go/source/browse/src/pkg/runtime/asm_386.s#437 http://code.google.com/p/go/source/browse/src/pkg/runtime/as... [3] http://code.google.com/p/go/source/browse/src/pkg/runtime/cgocall.c#100 http://code.google.com/p/go/source/browse/src/pkg/runtime/cg...
    - swdunlop 14y ago
      Interesting. It appears that cgocall has improved from the last time I went down this rabbit hole. Previously, it used a pthread_create to construct a C stack and context on each runtime.cgocall and then waited on a mutex for it to return. The overhead was enough to make me favor image/png Encode over libpng for smaller images. Time to go profiling again! :)
ajross 14y ago
Not necessarily. Computers are so fast these days that real time simulation of a lot of these older systems in interpreted environments (a naive C instruction interpreter, or even Javascript, etc...) is perfectly feasible. The Apple ][ had a 1MHz clock, a CPU that took 2-10 cycles per instruction, and an 8kb framebuffer that refreshed at 60Hz. That's not a lot of work to emulate.
- mithaler 14y ago
  That depends on what your standard of accuracy is. http://arstechnica.com/gaming/2011/08/accuracy-takes-power-one-mans-3ghz-quest-to-build-a-perfect-snes-emulator/ http://arstechnica.com/gaming/2011/08/accuracy-takes-power-o...
  - ajross 14y ago
    That's a fair argument, but "accuracy" isn't really the right word. Because then you'd have to deal with the fact that the color gamut from a 1982 NTSC TV is different from your LCD. Similarly I've never seen a good simulation of interlace flicker (not an issue on the NES though -- it didn't interlace) on a LCD either. The impedance mismatch between a NES controller and a modern keyboard alone is much higher than that beteween a "naive" simulation and the one in that article. And if you want to go down the "elaborate" road, there are even niftier projects out there like the Javascript transistor-level simulation of a 6502. There's always room to spend cycles. But simulating a 1980's computer to the level of precision required by "correct user experience" is a straightforward engineering task these days that doesn't require anything more than "just plain code."
Simucal 14y ago
How does that interact poorly with Go? I'm just curious.
gillianseed 14y ago
Assuming that by 'good' you mean emulating a system which has pretty high demands on hardware when emulating (I guess ps2,gamecube,dreamcast and onwards) then yes it's very likely you will want to use dynamic recompiling (jit) to convert the emulated code from foreign machine code to native machine code and execute the native code for increased performance. I assume this is possible with Go as it has the ability to call unsafe code?
strmpnk 14y ago
Lots of optimized code in Go's standard library has fast paths in assembly which is linked into the final binary on a platform by platform basis. I this sense, it's probably easier than C to drop into assembly. Granted, Go's memory model is not going to enforce a whole lot so you better know what you're doing at this point.
- azakai 14y ago
  Eh? Go doesn't have any capability to include inline assembly, > "There is no facility in the Go programming language to support in-line assembler language code, and there are no plans to do so" http://stackoverflow.com/questions/2951028/is-it-possible-to-include-inline-assembly-in-google-go-code http://stackoverflow.com/questions/2951028/is-it-possible-to... Of course the C parts of Go's standard library have optimized assembly - using C's capability to do. So Go is not easier in any way for dropping into assembly - in fact it's much harder, you can't do it in the language, at best you might be lucky if Go's library does some stuff like that for you.
  - strmpnk 14y ago
    I didn't say inline. I said linked. It's pretty easy to add.
    - azakai 14y ago
      You said "easier than C". How is linking to assembly code from Go any easier than linking to assembly code from C?
      - burntsushi 14y ago
        Just go see how it's done. Look for example at how `math.Exp` is declared: http://golang.org/src/pkg/math/exp.go?s=368:395#L4 http://golang.org/src/pkg/math/exp.go?s=368:395#L4 You'll notice that 'exp' is defined below, using Go. However, 'Exp' is also defined in Assembly. For example, amd64: http://golang.org/src/pkg/math/exp_amd64.s http://golang.org/src/pkg/math/exp_amd64.s This is all compatible with the `go` tool. You just drop in your Assembly, run `go build`, and the appropriate Assembly version is chosen based on your target architecture (or the Go code is used as a fallback if no Assembly for the target architecture exists). I don't have much experience with inlining Assembly in C, but this is pretty damn easy.

shardling 14y ago

So, I don't see that the article mentions why the language is good for emulators.

StavrosK 14y ago
Because it's a good opportunity to ride the Go bandwagon and get some free traffic with a list of five software projects, of course...
- drivebyacct2 14y ago
  Pffft. Dave is hardly riding the "Go bandwagon". (I guess this comment doesn't mean much unless you know what Dave does in the community or read the ML or have had help from him in the past. It's not as if he just picked this up to be "in", though I don't want to speak out of turn for him.)
  - deleted 14y ago
    [deleted]
- 4ad 14y ago
  Dave is not riding the bandwagon, he is one of the Go developers working mostly on the SSH package, the ARM platform and compiler optimizations. He is also core developer of Juju, software written in Go.
  - StavrosK 14y ago
    Oh, I didn't know that. It's probably a weird choice of words, then.
swdunlop 14y ago
Off the top of my head, I can think of a few why Go is better than Python or other high level languages for writing an emulator: - Go makes it easy to write a process for your emulator, instead of a state machine. - Go's slice types makes passing blocks of memory around a zero copy operation. - Pass by value lets you reduce the number of objects GC must trace. - Go is explicit about integer size and sign, makes conversion errors more visible. It's not better than C, but it's certainly easier to work with. I'm routinely surprised at how C-friendly Go's produced code is -- take a spin through profiling some of the image library primitives and compare them against your favorite C library.

killpg 14y ago

Yawn. Here's a Space Invaders emulator from 2006 in Factor. https://github.com/slavapestov/factor/blob/master/extra/space-invaders/space-invaders.factor https://github.com/slavapestov/factor/blob/master/extra/spac...

frozenport 14y ago

The systems emulated look quite primitive and, hence, give off the opposite impression: it feels like Go is not able to make a serious emulator. I suspect there are more complex emulators in javascript.

btrask 14y ago
I'm not going to enter a language flame war, but I was quite impressed by n64js. http://n64js.blogspot.co.uk/ http://n64js.blogspot.co.uk/