5 ms·

My complaint about Go centers around its mechanics of code reuse: static linking for all Go code; making anything Really Useful requires using other libraries,

by delinka 14y ago

My complaint about Go centers around its mechanics of code reuse: static linking for all Go code; making anything Really Useful requires using other libraries, which includes those written in C, which require use of a tool to help you write your wrapper ... If we could get dynamic linking and something less cumbersome for interfacing with existing libs, I could use Go for Serious Work.

shadowmint 14y ago
I dont see why this got downvoted. It's the biggest issue I have with go as well. It's a royal pain writing go that talks to C code, compared to say, lua or python, and there just _isnt_ a way to make other languages pickup go libraries and run the symbols from them afaik...
- jkn 14y ago
  According to my limited experience (small tools such as a process monitor linking to libproc) it is astonishingly easy and convenient to call C code from Go: no need to write bindings, as Cgo allows to access C symbols directly from Go. I would be interested in what are the difficulties when dealing with larger projects. For illustration, here's how I call the openproc() function from libproc: import ( /* #cgo LDFLAGS: -lproc #include <proc/readproc.h> PROCTAB* my_openproc(int flags) { return openproc(flags); } */ "C" ) func parse_procs() { proc := C.my_openproc(C.PROC_FILLUSR | C.PROC_FILLCOM | C.PROC_FILLSTAT) ... } This illustrates one issue I found with Cgo: calling functions with variable number of arguments is not supported. I had to define a new C function "my_openproc" with a fixed number of arguments, which you can see in the metadata of the import statement. It also includes the compiler and Cgo directives that make editing a Makefile unnecessary. The code for the whole tool is contained in one Go file.
- luriel 14y ago
  > It's a royal pain writing go that talks to C code, compared to say, lua or python This is plain wrong, you can pretty much call C code directly, while in Python and the like you really have to write a wrapper. Of course in Go you will write a wrapper anyway to give the library a more Go-like API, but I don't see how this could be any worse than in any other language. > there just _isnt_ a way to make other languages pickup go libraries and run the symbols from them afaik... To get non-Go code to call Go code is trickier (also Go really wants to run your main()), but can be done, and there are even libraries to write Python extensions in Go, see: http://gopy.qur.me/extensions/ http://gopy.qur.me/extensions/
  - xyproto 14y ago
    Also, there's assembly. http://stackoverflow.com/a/6535590/131264 http://stackoverflow.com/a/6535590/131264
- tedunangst 14y ago
  There's no single definition for Serious Work. It's just an insult to belittle people whose work is different than your own.
  - delinka 14y ago
    Let me help you out so you don't feel belittled or insulted. Serious Work is anything that will ever leave the confines of my experimental system to be used, viewed, or edited by anyone else. Doesn't matter if it's a trivial script-like tool or if it's a complex app.
thraxil 14y ago
My understanding is that the static linking is a temporary measure to keep things simple and let them focus on getting the language stable. Dynamic linking will come later.
luriel 14y ago
The C and C++ code Google runs on their servers is always statically linked, and I would call that Serious Work. In the end, static linking is almost invariably superior to dynamic linking: simpler toolchain, simpler runtime, better reliability, better performance and better security, see: http://harmful.cat-v.org/software/dynamic-linking/ http://harmful.cat-v.org/software/dynamic-linking/ That said, I think you can do dynamic linking with gccgo, and when linking to C libs with cgo, you do dynamic linking.
- comex 14y ago
  > (Remember that the real impetus for adding them to Unix was X11 and its big and badly-factored libraries, which most of us aren't blessed with.) Really? Today we are very much blessed with quite a few large libraries - not just X but Qt on top of that and KDE on top of that. WebKit. ICU. These come out to dozens of megabytes. Static linking would add somewhat less by eliminating unused symbols, but you'd still end up with massive bloat for your GUI processes; maybe with today's disk and memory sizes and SSDs that's not the end of the world, but it would be a rather pointless performance drag. That's not counting the difference between Go and Plan 9: I count 4,000 odd binaries in my PATH; if they each had the 1 MB overhead of hello world in Go, I'd be out 4 GB, more than I pay for the entire set of libraries on my system (with some redundancy). (And I hope you don't mind waiting for your programs to compile; despite the Go compiler's vaunted speed, it actually takes over 4 times as long to build hello world as clang for the same in C, the vast majority of which is spent in the linker.) Security? Today's security environment requires frequent patches to library code. If you have a system to automatically re-link all the programs that link against libraries with security updates, great (make sure you don't miss any, such as manually compiled or copied binaries); otherwise, your system is impossible to keep secure.
  - luriel 14y ago
    > Today we are very much blessed with quite a few large libraries And with magnitude orders more RAM. And just because there are large libraries doesn't mean they are either necessary or desirable. > not just X but Qt on top of that and KDE on top of that. WebKit. ICU. These come out to dozens of megabytes. Interestingly most apps that depend on Qt, WebKit or ICU include their own copies of this libraries. > Static linking would add somewhat less by eliminating unused symbols Thanks for reminding me of another benefit of static linking which I forgot to mention. > I count 4,000 odd binaries in my PATH I'm a fan of many small tools working well together but I can't help but feel that when you have systems with 4,000 binaries in your PATH, something has gone terribly wrong. Also, Go's current binaries are large not because they are statically linked but because there has been basically zero optimization of generated binary size. There is no reason why Go binaries couldn't be much smaller, other than so far it has not been a problem for anyone building systems in Go. > Today's security environment requires frequent patches to library code. Which is not helped by dynamic linking for reasons both of complexity (see http://harmful.cat-v.org/software/dynamic-linking/versioned-symbols http://harmful.cat-v.org/software/dynamic-linking/versioned-... ) and because most programs anyway use their own version of such 'shared' libs (update your system's ffmpeg and chrome will keep using its own copy). Dll-hell has security implications too. In practice people end up doing things that either nullify the alleged benefits of dynamic linking, or simply using static linking (Google deploys statically linked binaries, sometimes multiple Gb in size to their servers).
0xABADC0DA 14y ago
Somebody should write a Google Go to C transpiler. Mostly Google Go features are limitations or different styles in order to 'improve' C, so it should produce fairly readable C code. Then you could use any compiler, linking type, combine with any language, etc. You could probably even use LD_PRELOAD to replace coroutines with pthreads. For instance Russ Cox did coroutines and channels in his C libtask in a few thousand lines of code. It's not pretty but it works. Include something like it behind an API as 'libgrt.o' with a few other features and the rest of the code transpiles cleanly. ...but say this existed. What would be the point of writing Google Go code? A more compiler-friendly syntax, less flexibility to make simple memory/pointer mistakes? Would that be enough or would you, with perfect C linkage, just end up writing most of the code in C and Lua? It certainly seem the effect of creating their own compiler (worse than gcc), an opaque "go" tool that does everything, project layout and hosting rules, complicated FFI, etc is to make a 'toolchain island' locking you into writing everything in Google Go, where you might not otherwise.
- xyproto 14y ago
  You write "worse than gcc", but Go has been included in GCC since version 4.6: http://gcc.gnu.org/gcc-4.6/changes.html http://gcc.gnu.org/gcc-4.6/changes.html
  - 0xABADC0DA 14y ago
    I guess you aren't aware that Google Go has their own compilers based on Plan 9 C compiler called "6g" or "8g" (depends on what architecture). These compilers produce much less efficient code than gcc does. Yes, gcc now also compiles the language, but they still pimp their own compilers... iirc the "go" tool uses them by default (or maybe exclusively). Why create a new compiler? Why a project management tool that does everything except work with other build systems? Why isolate the language by making it difficult to use with others? I feel those are good questions to ask.

by delinka 14y ago

shadowmint 14y ago

I dont see why this got downvoted. It's the biggest issue I have with go as well. It's a royal pain writing go that talks to C code, compared to say, lua or python, and there just _isnt_ a way to make other languages pickup go libraries and run the symbols from them afaik...

jkn 14y ago
According to my limited experience (small tools such as a process monitor linking to libproc) it is astonishingly easy and convenient to call C code from Go: no need to write bindings, as Cgo allows to access C symbols directly from Go. I would be interested in what are the difficulties when dealing with larger projects. For illustration, here's how I call the openproc() function from libproc: import ( /* #cgo LDFLAGS: -lproc #include <proc/readproc.h> PROCTAB* my_openproc(int flags) { return openproc(flags); } */ "C" ) func parse_procs() { proc := C.my_openproc(C.PROC_FILLUSR | C.PROC_FILLCOM | C.PROC_FILLSTAT) ... } This illustrates one issue I found with Cgo: calling functions with variable number of arguments is not supported. I had to define a new C function "my_openproc" with a fixed number of arguments, which you can see in the metadata of the import statement. It also includes the compiler and Cgo directives that make editing a Makefile unnecessary. The code for the whole tool is contained in one Go file.
luriel 14y ago
> It's a royal pain writing go that talks to C code, compared to say, lua or python This is plain wrong, you can pretty much call C code directly, while in Python and the like you really have to write a wrapper. Of course in Go you will write a wrapper anyway to give the library a more Go-like API, but I don't see how this could be any worse than in any other language. > there just _isnt_ a way to make other languages pickup go libraries and run the symbols from them afaik... To get non-Go code to call Go code is trickier (also Go really wants to run your main()), but can be done, and there are even libraries to write Python extensions in Go, see: http://gopy.qur.me/extensions/ http://gopy.qur.me/extensions/
- xyproto 14y ago
  Also, there's assembly. http://stackoverflow.com/a/6535590/131264 http://stackoverflow.com/a/6535590/131264
tedunangst 14y ago
There's no single definition for Serious Work. It's just an insult to belittle people whose work is different than your own.
- delinka 14y ago
  Let me help you out so you don't feel belittled or insulted. Serious Work is anything that will ever leave the confines of my experimental system to be used, viewed, or edited by anyone else. Doesn't matter if it's a trivial script-like tool or if it's a complex app.

thraxil 14y ago

My understanding is that the static linking is a temporary measure to keep things simple and let them focus on getting the language stable. Dynamic linking will come later.

luriel 14y ago

The C and C++ code Google runs on their servers is always statically linked, and I would call that Serious Work. In the end, static linking is almost invariably superior to dynamic linking: simpler toolchain, simpler runtime, better reliability, better performance and better security, see: http://harmful.cat-v.org/software/dynamic-linking/ http://harmful.cat-v.org/software/dynamic-linking/ That said, I think you can do dynamic linking with gccgo, and when linking to C libs with cgo, you do dynamic linking.

comex 14y ago
> (Remember that the real impetus for adding them to Unix was X11 and its big and badly-factored libraries, which most of us aren't blessed with.) Really? Today we are very much blessed with quite a few large libraries - not just X but Qt on top of that and KDE on top of that. WebKit. ICU. These come out to dozens of megabytes. Static linking would add somewhat less by eliminating unused symbols, but you'd still end up with massive bloat for your GUI processes; maybe with today's disk and memory sizes and SSDs that's not the end of the world, but it would be a rather pointless performance drag. That's not counting the difference between Go and Plan 9: I count 4,000 odd binaries in my PATH; if they each had the 1 MB overhead of hello world in Go, I'd be out 4 GB, more than I pay for the entire set of libraries on my system (with some redundancy). (And I hope you don't mind waiting for your programs to compile; despite the Go compiler's vaunted speed, it actually takes over 4 times as long to build hello world as clang for the same in C, the vast majority of which is spent in the linker.) Security? Today's security environment requires frequent patches to library code. If you have a system to automatically re-link all the programs that link against libraries with security updates, great (make sure you don't miss any, such as manually compiled or copied binaries); otherwise, your system is impossible to keep secure.
- luriel 14y ago
  > Today we are very much blessed with quite a few large libraries And with magnitude orders more RAM. And just because there are large libraries doesn't mean they are either necessary or desirable. > not just X but Qt on top of that and KDE on top of that. WebKit. ICU. These come out to dozens of megabytes. Interestingly most apps that depend on Qt, WebKit or ICU include their own copies of this libraries. > Static linking would add somewhat less by eliminating unused symbols Thanks for reminding me of another benefit of static linking which I forgot to mention. > I count 4,000 odd binaries in my PATH I'm a fan of many small tools working well together but I can't help but feel that when you have systems with 4,000 binaries in your PATH, something has gone terribly wrong. Also, Go's current binaries are large not because they are statically linked but because there has been basically zero optimization of generated binary size. There is no reason why Go binaries couldn't be much smaller, other than so far it has not been a problem for anyone building systems in Go. > Today's security environment requires frequent patches to library code. Which is not helped by dynamic linking for reasons both of complexity (see http://harmful.cat-v.org/software/dynamic-linking/versioned-symbols http://harmful.cat-v.org/software/dynamic-linking/versioned-... ) and because most programs anyway use their own version of such 'shared' libs (update your system's ffmpeg and chrome will keep using its own copy). Dll-hell has security implications too. In practice people end up doing things that either nullify the alleged benefits of dynamic linking, or simply using static linking (Google deploys statically linked binaries, sometimes multiple Gb in size to their servers).

0xABADC0DA 14y ago

Somebody should write a Google Go to C transpiler. Mostly Google Go features are limitations or different styles in order to 'improve' C, so it should produce fairly readable C code. Then you could use any compiler, linking type, combine with any language, etc. You could probably even use LD_PRELOAD to replace coroutines with pthreads. For instance Russ Cox did coroutines and channels in his C libtask in a few thousand lines of code. It's not pretty but it works. Include something like it behind an API as 'libgrt.o' with a few other features and the rest of the code transpiles cleanly. ...but say this existed. What would be the point of writing Google Go code? A more compiler-friendly syntax, less flexibility to make simple memory/pointer mistakes? Would that be enough or would you, with perfect C linkage, just end up writing most of the code in C and Lua? It certainly seem the effect of creating their own compiler (worse than gcc), an opaque "go" tool that does everything, project layout and hosting rules, complicated FFI, etc is to make a 'toolchain island' locking you into writing everything in Google Go, where you might not otherwise.

xyproto 14y ago
You write "worse than gcc", but Go has been included in GCC since version 4.6: http://gcc.gnu.org/gcc-4.6/changes.html http://gcc.gnu.org/gcc-4.6/changes.html
- 0xABADC0DA 14y ago
  I guess you aren't aware that Google Go has their own compilers based on Plan 9 C compiler called "6g" or "8g" (depends on what architecture). These compilers produce much less efficient code than gcc does. Yes, gcc now also compiles the language, but they still pimp their own compilers... iirc the "go" tool uses them by default (or maybe exclusively). Why create a new compiler? Why a project management tool that does everything except work with other build systems? Why isolate the language by making it difficult to use with others? I feel those are good questions to ask.