3 ms·

I don't buy the 'cgo is not go' mantra. The original argument was that reimplementing everything in Go is better and avoids a bunch of trade-offs, mainly: more

I don't buy the 'cgo is not go' mantra. The original argument was that reimplementing everything in Go is better and avoids a bunch of trade-offs, mainly: more complicated and slower builds, loss of cross-compilation, tooling. But it completely ignores the real cost of maintaining software, even if 'just a translation'. Having the same underlying libraries interoperating with Go, Node, Java, etc is a massive advantage, gives you predictable performance, memory usage, and known reliability regardless of the host language. Who would ever port and maintain a beast such as a libvips or IM port and all the 25 other image libraries they depend on? What really should be addressed is precisely that pain, so users of modules using CGO don't have to worry about it. Fix it, not abandon it.

rob74 4y ago
I agree with everything you write, but the opposition to "too much cgo" has another reason too: what's the point of using a memory safe language if you tie it to a large body of code that is not memory safe? Rust applications using C libraries have this issue too, although the performance penalty of calling C from Rust is less than for Go. Of course, there are still valid reasons for using cgo. But if you are building a library, you should ask yourself the "cgo or not cgo" question even more seriously, because you will be forcing everybody who uses your library to use cgo too...
- srer 4y ago
  Is it memory safe? As I understood it you lose that as soon as you start using the namesake keyword, "go".
  - matthewmacleod 4y ago
    Yes – Go is generally memory safe, with the exception of the `unsafe` package which is (obviously) not. Outside of this, there is no access to raw pointers or pointer arithmetic. The use of goroutines does not affect memory safety. None of this means that you can't make an absolute mess of concurrency, but that's not a memory safety concern.
    - srer 4y ago
      https://research.swtch.com/gorace https://research.swtch.com/gorace describes a loss of memory safety through data races, but I note it says "In the current Go implementations" and was written in 2010. I never heard of news that the situation had changed, but if it has I'm most interested in when it did! :)
      - jrockway 4y ago
        It really depends on where you draw the line. This is an obviously-incorrect program (`++` isn't atomic): func main() { var x, y int64 doneCh := make(chan struct{}) inc := func() { for i := 0; i < 2<<20; i++ { x++ // line 13 in test/main.go atomic.AddInt64(&y, 1) } doneCh <- struct{}{} return } go inc() go inc() <-doneCh <-doneCh fmt.Printf("x, y = %v, %v\n", x, y) } This prints something like: $ go run main.go x, y = 3482626, 4194304 If you run it with the race detector the problem is clear: $ go run -race main.go ================== WARNING: DATA RACE Read at 0x00c0001bc008 by goroutine 7: main.main.func1() /home/jrockway/test/main.go:13 +0x50 Previous write at 0x00c0001bc008 by goroutine 8: main.main.func1() /home/jrockway/test/main.go:13 +0x64 Goroutine 7 (running) created at: main.main() /home/jrockway/test/main.go:19 +0x176 Goroutine 8 (running) created at: main.main() /home/jrockway/test/main.go:20 +0x184 ================== x, y = 4189962, 4194304 Found 1 data race(s) exit status 66 This is not strictly a memory safety problem (this can't crash the runtime), but a program that returns the wrong answer is pretty useless, so there is that. Obviously if x were going to be used as a pointer through the `unsafe` package, you could have problems. (Though I think that x will always be less than y, so if you have proved that memory[base+y] is safe to read, then memory[base+x] is safe to read. But it's easy to imagine a scenario where you overcount instead of undercount.
        samhw 4y ago
        I'm very confused by this comment. Memory safety doesn't mean "can't crash the runtime"[0]. That would exclude the most basic, canonical examples of memory unsafety: use after free, buffer overwrite/overread, double free, race conditions, etc. The erroneous Go exemplar you posted is literally one of the first examples of memory unsafety on Wikipedia: https://en.wikipedia.org/wiki/Memory_safety https://en.wikipedia.org/wiki/Memory_safety. Go is not memory safe by any stretch of the imagination. It doesn't even claim to be. At most it incidentally prevents a narrow subset of unsafe memory uses, compared with C (not a high standard), in the sense of providing an array type (and concomitant string type) rather than just pointer arithmetic.
- _ph_ 4y ago
  Yes, memory safety certainly would be an argument for rewriting a library in Go. But we are talking here about an extremely well tested and robust C program which is actively maintained. In most cases, it is more likely to have correct programs by calling the C version than attempting a port, which can introduced bugs of its own.
  - rob74 4y ago
    Yes, in this particular case, SQLite has a stellar reputation for stability, test coverage etc. But for other C codebases, even if they have been around for a long time and are stable and well maintained, exploits have still been found...
davidjfelix 4y ago
I think the underlying mantra is actually "if I lose compilation speed and simplicity, go is no longer the correct language choice for this problem" which is why cgo can be a non-starter for many.

rob74 4y ago

I agree with everything you write, but the opposition to "too much cgo" has another reason too: what's the point of using a memory safe language if you tie it to a large body of code that is not memory safe? Rust applications using C libraries have this issue too, although the performance penalty of calling C from Rust is less than for Go. Of course, there are still valid reasons for using cgo. But if you are building a library, you should ask yourself the "cgo or not cgo" question even more seriously, because you will be forcing everybody who uses your library to use cgo too...

srer 4y ago
Is it memory safe? As I understood it you lose that as soon as you start using the namesake keyword, "go".
- matthewmacleod 4y ago
  Yes – Go is generally memory safe, with the exception of the `unsafe` package which is (obviously) not. Outside of this, there is no access to raw pointers or pointer arithmetic. The use of goroutines does not affect memory safety. None of this means that you can't make an absolute mess of concurrency, but that's not a memory safety concern.
  - srer 4y ago
    https://research.swtch.com/gorace https://research.swtch.com/gorace describes a loss of memory safety through data races, but I note it says "In the current Go implementations" and was written in 2010. I never heard of news that the situation had changed, but if it has I'm most interested in when it did! :)
    - jrockway 4y ago
      It really depends on where you draw the line. This is an obviously-incorrect program (`++` isn't atomic): func main() { var x, y int64 doneCh := make(chan struct{}) inc := func() { for i := 0; i < 2<<20; i++ { x++ // line 13 in test/main.go atomic.AddInt64(&y, 1) } doneCh <- struct{}{} return } go inc() go inc() <-doneCh <-doneCh fmt.Printf("x, y = %v, %v\n", x, y) } This prints something like: $ go run main.go x, y = 3482626, 4194304 If you run it with the race detector the problem is clear: $ go run -race main.go ================== WARNING: DATA RACE Read at 0x00c0001bc008 by goroutine 7: main.main.func1() /home/jrockway/test/main.go:13 +0x50 Previous write at 0x00c0001bc008 by goroutine 8: main.main.func1() /home/jrockway/test/main.go:13 +0x64 Goroutine 7 (running) created at: main.main() /home/jrockway/test/main.go:19 +0x176 Goroutine 8 (running) created at: main.main() /home/jrockway/test/main.go:20 +0x184 ================== x, y = 4189962, 4194304 Found 1 data race(s) exit status 66 This is not strictly a memory safety problem (this can't crash the runtime), but a program that returns the wrong answer is pretty useless, so there is that. Obviously if x were going to be used as a pointer through the `unsafe` package, you could have problems. (Though I think that x will always be less than y, so if you have proved that memory[base+y] is safe to read, then memory[base+x] is safe to read. But it's easy to imagine a scenario where you overcount instead of undercount.
      - samhw 4y ago
        I'm very confused by this comment. Memory safety doesn't mean "can't crash the runtime"[0]. That would exclude the most basic, canonical examples of memory unsafety: use after free, buffer overwrite/overread, double free, race conditions, etc. The erroneous Go exemplar you posted is literally one of the first examples of memory unsafety on Wikipedia: https://en.wikipedia.org/wiki/Memory_safety https://en.wikipedia.org/wiki/Memory_safety. Go is not memory safe by any stretch of the imagination. It doesn't even claim to be. At most it incidentally prevents a narrow subset of unsafe memory uses, compared with C (not a high standard), in the sense of providing an array type (and concomitant string type) rather than just pointer arithmetic.
_ph_ 4y ago
Yes, memory safety certainly would be an argument for rewriting a library in Go. But we are talking here about an extremely well tested and robust C program which is actively maintained. In most cases, it is more likely to have correct programs by calling the C version than attempting a port, which can introduced bugs of its own.
- rob74 4y ago
  Yes, in this particular case, SQLite has a stellar reputation for stability, test coverage etc. But for other C codebases, even if they have been around for a long time and are stable and well maintained, exploits have still been found...

davidjfelix 4y ago

I think the underlying mantra is actually "if I lose compilation speed and simplicity, go is no longer the correct language choice for this problem" which is why cgo can be a non-starter for many.