5 ms·

C and Go without CGO

67 points by shanemhansen 12y ago

conroy 12y ago
The article doesn't mention any of the downsides to this approach, the biggest being style of C code. From Dave Cheney[0] - Using C code is inherently unsafe, not just because it unholsters all the C footguns, but because you can address any symbol in the runtime. With great power comes great responsibility. - The Go 1 compatibility guarantee does not extend to C code. - C functions cannot be inlined. - Escape analysis cannot follow values passed into C functions. - Code coverage does not extend to C functions. - The C compilers (5c, 6c, 8c) are not as optimised as their companion Go compilers, you may find that the code generated is not as efficient as the same code in Go. - You are writing plan 9 style C code, which is a rough analogue of C89. [0]: http://dave.cheney.net/2013/09/07/how-to-include-c-code-in-your-go-package http://dave.cheney.net/2013/09/07/how-to-include-c-code-in-y...
- akavel 12y ago
  Also some highly non-obvious gotchas and notes, straight from the trenches: - VERY IMPORTANT: you'll be actually writing for a pre-ANSI C compiler! Anything you think you know about C may prove false! Example real-world discrepancy found when porting Lua (an extremely high-quality ANSI C codebase) to Go's cc: http://golang.org/issue/7027 http://golang.org/issue/7027 - The ternary operator (?:) is not supported on ARM (see: https://github.com/akavel/goluago/issues/8#issuecomment-41448868 https://github.com/akavel/goluago/issues/8#issuecomment-4144...) - You don't have #if, #elif - You must be super careful, and really know what you're doing, when trying to pass pointers through the C<->Go boundary, as you're entering GC (garbage collector)'s carefully tended house of cards - You don't have access to the C standard library (although, see https://github.com/akavel/gostdc https://github.com/akavel/gostdc) - Using varargs functions is non-trivial - You must ensure you include one special header in each and every C file (#include "runtime.h") because of special "extern register" variables; although, I'm not sure if this requirement was not removed in Go 1.3. Hm, that's all I could remember from the top of my head. If you have any questions, feel free to ask me by email (czapkofan@gmail.com) or, obviously, anyone on the golang-nuts mailing list. That said, it's sure fun! :D actually, maybe in part because of all that :)
  - 4ad 12y ago
    > you'll be actually writing for a pre-ANSI C compiler! I would not say the compiler is pre-ANSI. For example it even has some C99 features. Rather, it chose to ignore ANSI, so yes, you need to really careful with what your doing. The integer promotion rules are different, for example -1 < u16int(42) is false, unlike in ANSI C. > You must be super careful, and really know what you're doing, when trying to pass pointers through the C<->Go boundary Yes, however that is true with cgo as well. Especially now with the new precise garbage collector. > because of special "extern register" variables; although, I'm not sure if this requirement was not removed in Go 1.3. Nothing changed in 1.3, however big changes are coming into Go 1.4 and Go 1.5. Eventually all the C code in the runtime will go away. Until then, the plan is to move all remaining C code onto the scheduler stack instead of having common Go and C stacks, as that interferes with the copying stacks and the new precise garbage collector. In other words, using the Plan 9 C compiler in your project uses to work by accident. Not it appears as it would still work, but it actually has a subtle net negative effect that's not obvious to see at first glance. You should avoid it.
- 4ad 12y ago
  Even worse, the C parts of the Go runtime are being rewritten in Go. The Plan 9 C compiler is going away along with all the other C bits as the compilers themselves are being translated to Go. In short, this will go away very soon. Possibly even Go 1.4. Definitely Go 1.5.
  - dualogy 12y ago
    The way he describes linking/including .c files is identical to how you can link-in/include assembly code via .s files. I don't see the latter going away (the standard `math` package uses this extensively), so why should the former---even if Go 1.4 or later itself doesn't use it..
    - rsc 12y ago
      The main Go distribution will always have assembly source files: there are always going to be special instructions or routines, such as low-level startup, that must be written at that level. The same is not true of (non-gcc) C source files. Once the main Go distribution has no non-gcc C source files, we _will_ delete the Plan 9 C compilers. There is no reason to maintain them if we are not using them.
voltagex_ 12y ago
Excellent article, but this post needs a better title. C and Go without CGO might be okay for here.
voltagex_ 12y ago
I'd love to know how this works behind the scenes. In the article, a file called chello.c and a file called gohello.go are created - do the filenames have any significance? How does the go compiler and linker know what to do here?
- shanemhansen 12y ago
  Run strace -f -eexecve go build to see whats going on under the covers. The filenames are because 6c and 6g both produced an object file named hello.6. 6g,6c, and 6a produce object files on amd64. 6l is the linker which combines them. Honestly I had a little trouble running the commands individually so I just used go build.
  - 4ad 12y ago
    Run go build -x There's no need for strace.
pekk 12y ago
"as fast as physically possible on your CPU architecture" is a ludicrous phrase full of misunderstanding.
- voltagex_ 12y ago
  Can you suggest a better one?
  - danieltillett 12y ago
    I think what pekk is complaining about is the “as fast as physically possible” - unless your CPU is on a rocket heading to the moon it is not physically moving as fast as physically possible. As for a suggestion how about just “as fast as possible”.
    - kibibu 12y ago
      As fast as physically possible means "as fast as allowed by physics". It's all physics!
marcell 12y ago
Can anyone explain the use of the "·" character?
- hornetblack 12y ago
  In go you use '.' for namespace accesses. Eg: os.NewFile. But in C '.' means access struct member. So the · is used instead so to do f = os.NewFile(3, "/dev/zero") # Go you would use f = os·NewFile(3, "/dev/zero"); # C
- markuskobler 12y ago
  So Russ Cox goes into more detail in this post https://groups.google.com/d/msg/golang-nuts/MdYlJbW4SAo/TrAEoCjAkMwJ https://groups.google.com/d/msg/golang-nuts/MdYlJbW4SAo/TrAE... Although Go does not compile via C, occasionally C code does need to refer to Go identifiers. Since we chose not to restrict the mangled Go names to the space of valid C names, we must add some mechanism to refer to Go names from C. That mechanism is: 1. In the assemblers and C compilers, which already accepted all non-ASCII Unicode code points in identifiers, the Unicode characters · and / rewrite to ordinary . and / in the object files. 2. A symbol with a leading . has its import path inserted before the . when being linked: inside encoding/json.a, a reference to ".Marshal" is equivalent to "encoding/json.Marshal". Because of 2, we went a long time without needing a special character for slash. Recently the introduction of race detection has made it convenient for package runtime to be able to refer to a few symbols in runtime/race, hence the new slash lookalike."
  - akavel 12y ago
    For some supplementary information, see also this answer on stackoverflow: http://stackoverflow.com/a/13490787/98528 http://stackoverflow.com/a/13490787/98528
  - frou_dh 12y ago
    When did statements like this sprout a So on the front? It's everywhere now!
- 4ad 12y ago

67 points by shanemhansen 12y ago

The article doesn't mention any of the downsides to this approach, the biggest being style of C code. From Dave Cheney[0] - Using C code is inherently unsafe, not just because it unholsters all the C footguns, but because you can address any symbol in the runtime. With great power comes great responsibility. - The Go 1 compatibility guarantee does not extend to C code. - C functions cannot be inlined. - Escape analysis cannot follow values passed into C functions. - Code coverage does not extend to C functions. - The C compilers (5c, 6c, 8c) are not as optimised as their companion Go compilers, you may find that the code generated is not as efficient as the same code in Go. - You are writing plan 9 style C code, which is a rough analogue of C89. [0]: http://dave.cheney.net/2013/09/07/how-to-include-c-code-in-your-go-package http://dave.cheney.net/2013/09/07/how-to-include-c-code-in-y...

akavel 12y ago
Also some highly non-obvious gotchas and notes, straight from the trenches: - VERY IMPORTANT: you'll be actually writing for a pre-ANSI C compiler! Anything you think you know about C may prove false! Example real-world discrepancy found when porting Lua (an extremely high-quality ANSI C codebase) to Go's cc: http://golang.org/issue/7027 http://golang.org/issue/7027 - The ternary operator (?:) is not supported on ARM (see: https://github.com/akavel/goluago/issues/8#issuecomment-41448868 https://github.com/akavel/goluago/issues/8#issuecomment-4144...) - You don't have #if, #elif - You must be super careful, and really know what you're doing, when trying to pass pointers through the C<->Go boundary, as you're entering GC (garbage collector)'s carefully tended house of cards - You don't have access to the C standard library (although, see https://github.com/akavel/gostdc https://github.com/akavel/gostdc) - Using varargs functions is non-trivial - You must ensure you include one special header in each and every C file (#include "runtime.h") because of special "extern register" variables; although, I'm not sure if this requirement was not removed in Go 1.3. Hm, that's all I could remember from the top of my head. If you have any questions, feel free to ask me by email (czapkofan@gmail.com) or, obviously, anyone on the golang-nuts mailing list. That said, it's sure fun! :D actually, maybe in part because of all that :)
- 4ad 12y ago
  > you'll be actually writing for a pre-ANSI C compiler! I would not say the compiler is pre-ANSI. For example it even has some C99 features. Rather, it chose to ignore ANSI, so yes, you need to really careful with what your doing. The integer promotion rules are different, for example -1 < u16int(42) is false, unlike in ANSI C. > You must be super careful, and really know what you're doing, when trying to pass pointers through the C<->Go boundary Yes, however that is true with cgo as well. Especially now with the new precise garbage collector. > because of special "extern register" variables; although, I'm not sure if this requirement was not removed in Go 1.3. Nothing changed in 1.3, however big changes are coming into Go 1.4 and Go 1.5. Eventually all the C code in the runtime will go away. Until then, the plan is to move all remaining C code onto the scheduler stack instead of having common Go and C stacks, as that interferes with the copying stacks and the new precise garbage collector. In other words, using the Plan 9 C compiler in your project uses to work by accident. Not it appears as it would still work, but it actually has a subtle net negative effect that's not obvious to see at first glance. You should avoid it.
4ad 12y ago
Even worse, the C parts of the Go runtime are being rewritten in Go. The Plan 9 C compiler is going away along with all the other C bits as the compilers themselves are being translated to Go. In short, this will go away very soon. Possibly even Go 1.4. Definitely Go 1.5.
- dualogy 12y ago
  The way he describes linking/including .c files is identical to how you can link-in/include assembly code via .s files. I don't see the latter going away (the standard `math` package uses this extensively), so why should the former---even if Go 1.4 or later itself doesn't use it..
  - rsc 12y ago
    The main Go distribution will always have assembly source files: there are always going to be special instructions or routines, such as low-level startup, that must be written at that level. The same is not true of (non-gcc) C source files. Once the main Go distribution has no non-gcc C source files, we _will_ delete the Plan 9 C compilers. There is no reason to maintain them if we are not using them.

voltagex_ 12y ago

Excellent article, but this post needs a better title. C and Go without CGO might be okay for here.

voltagex_ 12y ago

I'd love to know how this works behind the scenes. In the article, a file called chello.c and a file called gohello.go are created - do the filenames have any significance? How does the go compiler and linker know what to do here?

shanemhansen 12y ago
Run strace -f -eexecve go build to see whats going on under the covers. The filenames are because 6c and 6g both produced an object file named hello.6. 6g,6c, and 6a produce object files on amd64. 6l is the linker which combines them. Honestly I had a little trouble running the commands individually so I just used go build.
- 4ad 12y ago
  Run go build -x There's no need for strace.

pekk 12y ago

"as fast as physically possible on your CPU architecture" is a ludicrous phrase full of misunderstanding.

voltagex_ 12y ago
Can you suggest a better one?
- danieltillett 12y ago
  I think what pekk is complaining about is the “as fast as physically possible” - unless your CPU is on a rocket heading to the moon it is not physically moving as fast as physically possible. As for a suggestion how about just “as fast as possible”.
  - kibibu 12y ago
    As fast as physically possible means "as fast as allowed by physics". It's all physics!

marcell 12y ago

Can anyone explain the use of the "·" character?

hornetblack 12y ago
In go you use '.' for namespace accesses. Eg: os.NewFile. But in C '.' means access struct member. So the · is used instead so to do f = os.NewFile(3, "/dev/zero") # Go you would use f = os·NewFile(3, "/dev/zero"); # C
markuskobler 12y ago
So Russ Cox goes into more detail in this post https://groups.google.com/d/msg/golang-nuts/MdYlJbW4SAo/TrAEoCjAkMwJ https://groups.google.com/d/msg/golang-nuts/MdYlJbW4SAo/TrAE... Although Go does not compile via C, occasionally C code does need to refer to Go identifiers. Since we chose not to restrict the mangled Go names to the space of valid C names, we must add some mechanism to refer to Go names from C. That mechanism is: 1. In the assemblers and C compilers, which already accepted all non-ASCII Unicode code points in identifiers, the Unicode characters · and / rewrite to ordinary . and / in the object files. 2. A symbol with a leading . has its import path inserted before the . when being linked: inside encoding/json.a, a reference to ".Marshal" is equivalent to "encoding/json.Marshal". Because of 2, we went a long time without needing a special character for slash. Recently the introduction of race detection has made it convenient for package runtime to be able to refer to a few symbols in runtime/race, hence the new slash lookalike."
- akavel 12y ago
  For some supplementary information, see also this answer on stackoverflow: http://stackoverflow.com/a/13490787/98528 http://stackoverflow.com/a/13490787/98528
- frou_dh 12y ago
  When did statements like this sprout a So on the front? It's everywhere now!
4ad 12y ago