6 ms·

I am curious why they were advised not to use Go. Probably not a safety concern. Edit: cgo != Go. Thanks for the responses. I have done a bit of Go, but just p

I am curious why they were advised not to use Go. Probably not a safety concern. Edit: cgo != Go. Thanks for the responses. I have done a bit of Go, but just pure Go.

nickpsecurity 10y ago
Here's what I'd have told them: https://news.ycombinator.com/item?id=14013617 https://news.ycombinator.com/item?id=14013617
m-j-fox 10y ago
They were advised against cgo. In my not so recent experience, it is a huge PITA.
- pjmlp 10y ago
  I never understood the idea behind cgo instead of having a proper FFI like Delphi, .NET, Eiffel and so many other languages.
  - stepik777 10y ago
    That's probably because of coroutines.
    - pjmlp 10y ago
      The runtime could take care of that, just like it happens in another languages.
hannob 10y ago
Go needs a garbage collector that needs to be set up. Using Go code within C code is possible, but it creates additional hurdles. Therefore a slow transition of rewriting parts of the code in a safer language and having the core still in C is much less feasible with Go. With Rust you can easier just compile some object files and link them into your application.
masklinn 10y ago
They were not advised against Go but against cgo. Part of what they want is incremental conversion and cgo is at the same time not-go[0], costly[1] and complex[2], and then you still need to manage the Go runtime (GC & al) from within your C system. That makes integrating the two difficult, especially when you want to replace the existing system piecemeal. A pure-Go rewrite might be an option (in fact Tor seems pretty firmly in Go's use cases), but that's not what the Tor team is trying to do. [0] https://dave.cheney.net/2016/01/18/cgo-is-not-go https://dave.cheney.net/2016/01/18/cgo-is-not-go [1] a cgo->c call is ~100 times more expensive than a go->go call, and ~400 times more expensive than a c->c or rust->c call https://www.reddit.com/r/golang/comments/3oztwi/from_python_to_go_and_back_again_mozilla_dev/cw2jg76/ https://www.reddit.com/r/golang/comments/3oztwi/from_python_... [2] https://www.cockroachlabs.com/blog/the-cost-and-complexity-of-cgo/ https://www.cockroachlabs.com/blog/the-cost-and-complexity-o...
- steveklabnik 10y ago
  go -> C calls have gotten way way cheaper in newer versions of Go. There's still overhead but it's not as bad as it used to be.
  - hckr1292 10y ago
    I love that the Rust core team is defending Go in a thread ostensibly about Rust and Tor.
  - riking 10y ago
    It's still absolutely terrible in terms of ergonomics. You're forced to perform manual memory management, etc. I've done it a few times and I absolutely don't recommend it.
    - steveklabnik 10y ago
      I'm not making any value judgements here; just saying that the times have decreased significantly since the links that were posted.
    - 10y ago
geofft 10y ago
Go and Rust are very different languages. Rust is, by design, well-suited to Tor's use case, where they have a large C or C++ program and they need to incrementally rewrite parts of it (and maybe never all of it!) in a better language. It turns out (or so I hear) that Google statically links everything in production, and has been using C++ as a language to implement HTTP endpoints for a long time. So Go is a better C++ for what they want out of a better C++; for the rest of us, it looks more like a compiled language along the lines of Python/Ruby/etc. with a nice deployment story. If you want that out of your better C++, Go is great. If you want to reimplement all of Tor from scratch, Go certainly seems like a reasonable choice. But as a result of these priorities, Go basically doesn't have interoperability with the platform ABI as a goal. (For some combination of historical reasons and the lack of complicated features in C, the platform ABI on just about every platform these days is a C ABI.) Rust does; it uses a standard compiler toolchain (LLVM) instead of what's basically a custom one (Plan 9), and the standard toolchain knows how to generate calls that follow the C ABI. Rust doesn't have a runtime of its own, and it's safe to directly call into a Rust program from some arbitrary point in a C program. Rust's allocator doesn't care if you do stupid things with pointers it allocates, as long as you give them back eventually. Rust doesn't create threads on its own unless you ask. Rust functions use the normal stack. Rust on UNIX uses the platform libc. And so forth. It's possible to call C code from Go and vice versa, just as it's possible to call C code from Python and vice versa. But Go is not best tool for this particular job.
- socmag 10y ago
  Totally agree, and Rust isn't a sane choice either even though the points you make are valid. Rust has some interesting features, but being C isn't one of them.
  - geofft 10y ago
    Can you expand on why you think Rust isn't a sane choice here?
    - socmag 10y ago
      Because again you are deviating from the industry standard completely portable lingua-franca language that is designed explicitly for precisely these types of problem spaces, and is perfectly in tune with the OS and existing standard library. What would be the advantage, just improved memory safety guarantees for people working on the project? If that's the case start again from scratch and the first thing you do is build a handle based memory management system that everything must go through Safexxxx() versions of everything and then explicitly enforce no other memory access patterns. If you want to do concurrency, build simple threads with input queues that would just be like channels in Go. You can use a completely functional actor model in C. Is all this object oriented dangling pointer stuff that is scary, but there really isn't any need for that. Software pipelines with slab allocation or ring buffers virtually guarantee no leaks and are very scalable and cache friendly. That would be my personal recommendation.

I am curious why they were advised not to use Go. Probably not a safety concern. Edit: cgo != Go. Thanks for the responses. I have done a bit of Go, but just pure Go.

nickpsecurity 10y ago

Here's what I'd have told them: https://news.ycombinator.com/item?id=14013617 https://news.ycombinator.com/item?id=14013617

m-j-fox 10y ago

They were advised against cgo. In my not so recent experience, it is a huge PITA.

pjmlp 10y ago
I never understood the idea behind cgo instead of having a proper FFI like Delphi, .NET, Eiffel and so many other languages.
- stepik777 10y ago
  That's probably because of coroutines.
  - pjmlp 10y ago
    The runtime could take care of that, just like it happens in another languages.

hannob 10y ago

Go needs a garbage collector that needs to be set up. Using Go code within C code is possible, but it creates additional hurdles. Therefore a slow transition of rewriting parts of the code in a safer language and having the core still in C is much less feasible with Go. With Rust you can easier just compile some object files and link them into your application.

masklinn 10y ago

They were not advised against Go but against cgo. Part of what they want is incremental conversion and cgo is at the same time not-go[0], costly[1] and complex[2], and then you still need to manage the Go runtime (GC & al) from within your C system. That makes integrating the two difficult, especially when you want to replace the existing system piecemeal. A pure-Go rewrite might be an option (in fact Tor seems pretty firmly in Go's use cases), but that's not what the Tor team is trying to do. [0] https://dave.cheney.net/2016/01/18/cgo-is-not-go https://dave.cheney.net/2016/01/18/cgo-is-not-go [1] a cgo->c call is ~100 times more expensive than a go->go call, and ~400 times more expensive than a c->c or rust->c call https://www.reddit.com/r/golang/comments/3oztwi/from_python_to_go_and_back_again_mozilla_dev/cw2jg76/ https://www.reddit.com/r/golang/comments/3oztwi/from_python_... [2] https://www.cockroachlabs.com/blog/the-cost-and-complexity-of-cgo/ https://www.cockroachlabs.com/blog/the-cost-and-complexity-o...

steveklabnik 10y ago
go -> C calls have gotten way way cheaper in newer versions of Go. There's still overhead but it's not as bad as it used to be.
- hckr1292 10y ago
  I love that the Rust core team is defending Go in a thread ostensibly about Rust and Tor.
- riking 10y ago
  It's still absolutely terrible in terms of ergonomics. You're forced to perform manual memory management, etc. I've done it a few times and I absolutely don't recommend it.
  - steveklabnik 10y ago
    I'm not making any value judgements here; just saying that the times have decreased significantly since the links that were posted.
  - 10y ago

geofft 10y ago

Go and Rust are very different languages. Rust is, by design, well-suited to Tor's use case, where they have a large C or C++ program and they need to incrementally rewrite parts of it (and maybe never all of it!) in a better language. It turns out (or so I hear) that Google statically links everything in production, and has been using C++ as a language to implement HTTP endpoints for a long time. So Go is a better C++ for what they want out of a better C++; for the rest of us, it looks more like a compiled language along the lines of Python/Ruby/etc. with a nice deployment story. If you want that out of your better C++, Go is great. If you want to reimplement all of Tor from scratch, Go certainly seems like a reasonable choice. But as a result of these priorities, Go basically doesn't have interoperability with the platform ABI as a goal. (For some combination of historical reasons and the lack of complicated features in C, the platform ABI on just about every platform these days is a C ABI.) Rust does; it uses a standard compiler toolchain (LLVM) instead of what's basically a custom one (Plan 9), and the standard toolchain knows how to generate calls that follow the C ABI. Rust doesn't have a runtime of its own, and it's safe to directly call into a Rust program from some arbitrary point in a C program. Rust's allocator doesn't care if you do stupid things with pointers it allocates, as long as you give them back eventually. Rust doesn't create threads on its own unless you ask. Rust functions use the normal stack. Rust on UNIX uses the platform libc. And so forth. It's possible to call C code from Go and vice versa, just as it's possible to call C code from Python and vice versa. But Go is not best tool for this particular job.

socmag 10y ago
Totally agree, and Rust isn't a sane choice either even though the points you make are valid. Rust has some interesting features, but being C isn't one of them.
- geofft 10y ago
  Can you expand on why you think Rust isn't a sane choice here?
  - socmag 10y ago
    Because again you are deviating from the industry standard completely portable lingua-franca language that is designed explicitly for precisely these types of problem spaces, and is perfectly in tune with the OS and existing standard library. What would be the advantage, just improved memory safety guarantees for people working on the project? If that's the case start again from scratch and the first thing you do is build a handle based memory management system that everything must go through Safexxxx() versions of everything and then explicitly enforce no other memory access patterns. If you want to do concurrency, build simple threads with input queues that would just be like channels in Go. You can use a completely functional actor model in C. Is all this object oriented dangling pointer stuff that is scary, but there really isn't any need for that. Software pipelines with slab allocation or ring buffers virtually guarantee no leaks and are very scalable and cache friendly. That would be my personal recommendation.