7 ms·

Another very nice feature of Go is, that since 1.5, the whole runtime, including the GC is written in Go itself. So every Go developer can look at the implement

by _ph_ 10y ago

Another very nice feature of Go is, that since 1.5, the whole runtime, including the GC is written in Go itself. So every Go developer can look at the implementation. The GC itself is surprisingly small amount of code.

knorker 10y ago
It's not yet idiomatic or good Go code now, though, is it? IIRC the initial state (1.5) was mostly machine translated code from C to Go.
- _ph_ 10y ago
  The main thing is, that they machine translated the Go compiler itself from C to Go, so the compiler had not so great Go code. The rest of the runtime probably was already human-written in Go before that. And of course, anything which has been under development since then is human written, like the new SSA-based compiler.
  - EdiX 10y ago
    The rest of the runtime probably was already human-written in Go find -name '*.X' | xargs wc -l in src/runtime says Go 1.4: 21701 .c 33348 .go 19160 .s Go 1.7b2: 3340 .c 77597 .go 33827 .s The assembly growth seems to be attributable to the increased number of supported platforms
    - 4ad 10y ago
      You should exclude runtime/cgo, since it's not relevant.
    - giovannibajo1 10y ago
      There's also an increased number of optimized code-paths, like for instance some crypto code in the TLS stacks. The natural way to optimize Go code, for instance to benefit from advanced CPU instructions in modern architectures, is to fallback to assembly.
- 4ad 10y ago
  The runtime is written by people, not machines. The compilers and low-level toolchain libraries that encode machine instructions have been machine-translated. The runtime is an example of very good low-level Go code that breaks safety rules. However, it is not a good example of user-level Go code. Note that even though the Go code in the runtime heavily uses unsafe, the runtime is much safer and more stable (we find less bugs) then when it was written in C.
chrisseaton 10y ago
I never understand this argument. Are there people who know enough about GC to understand potential issues and have an informed opinion on how to improve it, but can't read C?
- _ph_ 10y ago
  "read C" is a very loose term. Yes, I can read C. But, as I am not using in daily practice, I am rusty. While as a Go programmer, I am quite trained in reading Go code. You also do need to know the behavior of one compiler (the Go one) vs. also have a good understanding what the C compiler does. Also, Go is a much clearer and strongly typed language, so Go certainly is a much nicer implementation language than C. (If I thought C was better suited, I would be using C in the first place...)
  - chrisseaton 10y ago
    Right, but are you also skilled in writing garbage collectors? (You may be, I'm not assuming you aren't. I'm not.) Otherwise the fact that you can read Go more easily than C doesn't make a difference, as what would you do after you've read the code if you don't understand what any of the algorithms or why they are designed that way? I'm sure it's very good that as much of the runtime is written in Go as possible, but I think people are being too optimistic when they hope it will empower people who aren't already skilled in compilers or garbage collection to contribute.
    - _ph_ 10y ago
      Of course, being a Go expert does not make you a GC expert automatically. You need to be both. But why should you also be a C expert? Adding another whole field of expertise to the requirements does not sound like an improvement. And everything written in Go also means you are dealing with just one compiler and not two, if you mix Go and C code.
      - chrisseaton 10y ago
        I don't think you should also have to be a C expert, but I am suggesting that, in practice, there is nobody in the world who is proficient in garbage collection who does not also know C. I think if you learned everything needed to understand GC, but were never exposed to C, you would already know enough to pick the language up in a couple of hours. Nobody is going to turn up in the Go IRC room saying that they have a great idea how to reduce pause times by improving the work-stealing between concurrent markers by using a better lock-free queue algorithm, except do'h they don't know C. I get your point about just one compiler though - less moving parts is good.
colordrops 10y ago
Excuse my obvious ignorance, but how is the GC written in Go? The GC in Go is not optional, right? Does the GC use GC? Turtles all the way down?
- andrewvc 10y ago
  I can't speak for go specifically but in some languages you can use a manually allocated subset of the language to write the gc.
- 4ad 10y ago
  In general, memory allocated by the Go runtime is garbage collected, yes. When the garbage collector runs, there isn't really any difference between user and runtime goroutines anymore. But the garbage collector itself doesn't created garbage implicitly. Go is a language that does not expose stack vs. heap allocation as a primitive to the programmer. Memory is allocated in the best place possible, preferably on the stack, but if that is not possible it is allocated on the heap. But in the runtime we need to control the generation of garbage, so runtime code is compiled with a switch that forbids implicit heap allocations (code won't compile if it requires transparent heap allocations). Heap memory is allocated by calling a function like runtime.mallocgc. However, this memory is garbage collected just like everything else (e.g. there is no free).
- mark-r 10y ago
  One of the signs of maturity in a language is when it reaches the point that it can implement itself. C wasn't written in C originally either.
- uluyol 10y ago
  How does one implement malloc in C? (Edit: ignore this. As pointed out in the responses, this actually different since GC calls are inserted by the compiler) Unsafe code, direct syscalls, using only a subset of language features, and coupling to the compiler (both for generating data used by the GC as well as inserting calls into the runtime in appropriate places). None of this would really be any different if implemented in C. C is clearly an unsafe language, the syscalls would still be there, as would the coupling to the compiler. The difference is that you have to have a fast way to call from Go into C. With Go this is unnecessary. Of course if you're really curious, you can always check out the source :)
vorg 10y ago
> since 1.5, the whole runtime is written in Go itself The parser was written in Yacc (with C code generated) until version 1.6. I'm wondering if there's any other parts of Go yet to be converted to Go.

by _ph_ 10y ago

knorker 10y ago

It's not yet idiomatic or good Go code now, though, is it? IIRC the initial state (1.5) was mostly machine translated code from C to Go.

_ph_ 10y ago
The main thing is, that they machine translated the Go compiler itself from C to Go, so the compiler had not so great Go code. The rest of the runtime probably was already human-written in Go before that. And of course, anything which has been under development since then is human written, like the new SSA-based compiler.
- EdiX 10y ago
  The rest of the runtime probably was already human-written in Go find -name '*.X' | xargs wc -l in src/runtime says Go 1.4: 21701 .c 33348 .go 19160 .s Go 1.7b2: 3340 .c 77597 .go 33827 .s The assembly growth seems to be attributable to the increased number of supported platforms
  - 4ad 10y ago
    You should exclude runtime/cgo, since it's not relevant.
  - giovannibajo1 10y ago
    There's also an increased number of optimized code-paths, like for instance some crypto code in the TLS stacks. The natural way to optimize Go code, for instance to benefit from advanced CPU instructions in modern architectures, is to fallback to assembly.
4ad 10y ago
The runtime is written by people, not machines. The compilers and low-level toolchain libraries that encode machine instructions have been machine-translated. The runtime is an example of very good low-level Go code that breaks safety rules. However, it is not a good example of user-level Go code. Note that even though the Go code in the runtime heavily uses unsafe, the runtime is much safer and more stable (we find less bugs) then when it was written in C.

chrisseaton 10y ago

I never understand this argument. Are there people who know enough about GC to understand potential issues and have an informed opinion on how to improve it, but can't read C?

_ph_ 10y ago
"read C" is a very loose term. Yes, I can read C. But, as I am not using in daily practice, I am rusty. While as a Go programmer, I am quite trained in reading Go code. You also do need to know the behavior of one compiler (the Go one) vs. also have a good understanding what the C compiler does. Also, Go is a much clearer and strongly typed language, so Go certainly is a much nicer implementation language than C. (If I thought C was better suited, I would be using C in the first place...)
- chrisseaton 10y ago
  Right, but are you also skilled in writing garbage collectors? (You may be, I'm not assuming you aren't. I'm not.) Otherwise the fact that you can read Go more easily than C doesn't make a difference, as what would you do after you've read the code if you don't understand what any of the algorithms or why they are designed that way? I'm sure it's very good that as much of the runtime is written in Go as possible, but I think people are being too optimistic when they hope it will empower people who aren't already skilled in compilers or garbage collection to contribute.
  - _ph_ 10y ago
    Of course, being a Go expert does not make you a GC expert automatically. You need to be both. But why should you also be a C expert? Adding another whole field of expertise to the requirements does not sound like an improvement. And everything written in Go also means you are dealing with just one compiler and not two, if you mix Go and C code.
    - chrisseaton 10y ago
      I don't think you should also have to be a C expert, but I am suggesting that, in practice, there is nobody in the world who is proficient in garbage collection who does not also know C. I think if you learned everything needed to understand GC, but were never exposed to C, you would already know enough to pick the language up in a couple of hours. Nobody is going to turn up in the Go IRC room saying that they have a great idea how to reduce pause times by improving the work-stealing between concurrent markers by using a better lock-free queue algorithm, except do'h they don't know C. I get your point about just one compiler though - less moving parts is good.

colordrops 10y ago

Excuse my obvious ignorance, but how is the GC written in Go? The GC in Go is not optional, right? Does the GC use GC? Turtles all the way down?

andrewvc 10y ago
I can't speak for go specifically but in some languages you can use a manually allocated subset of the language to write the gc.
4ad 10y ago
In general, memory allocated by the Go runtime is garbage collected, yes. When the garbage collector runs, there isn't really any difference between user and runtime goroutines anymore. But the garbage collector itself doesn't created garbage implicitly. Go is a language that does not expose stack vs. heap allocation as a primitive to the programmer. Memory is allocated in the best place possible, preferably on the stack, but if that is not possible it is allocated on the heap. But in the runtime we need to control the generation of garbage, so runtime code is compiled with a switch that forbids implicit heap allocations (code won't compile if it requires transparent heap allocations). Heap memory is allocated by calling a function like runtime.mallocgc. However, this memory is garbage collected just like everything else (e.g. there is no free).
mark-r 10y ago
One of the signs of maturity in a language is when it reaches the point that it can implement itself. C wasn't written in C originally either.
uluyol 10y ago
How does one implement malloc in C? (Edit: ignore this. As pointed out in the responses, this actually different since GC calls are inserted by the compiler) Unsafe code, direct syscalls, using only a subset of language features, and coupling to the compiler (both for generating data used by the GC as well as inserting calls into the runtime in appropriate places). None of this would really be any different if implemented in C. C is clearly an unsafe language, the syscalls would still be there, as would the coupling to the compiler. The difference is that you have to have a fast way to call from Go into C. With Go this is unnecessary. Of course if you're really curious, you can always check out the source :)

vorg 10y ago

> since 1.5, the whole runtime is written in Go itself The parser was written in Yacc (with C code generated) until version 1.6. I'm wondering if there's any other parts of Go yet to be converted to Go.