10 ms·

Mistakes C/C++ Devs make writing Go

202 points by beliu 8y ago

shoo 8y ago
re: "# channels < # goroutines" func doSomethingTwice() error { // Issue occurs below errc := make(chan error) go func() { defer fmt.Println("done wth a") errc <- doSomething("a") }() go func() { defer fmt.Println("done with b") errc <- doSomething("b") }() err := <-errc return err } > When one routine writes to the channel, the program exits and the other goroutine is lost, building up memory use as a results I'm not very familiar with go, but my guess would be that since the two goroutines send to an unbuffered channel that is read at most once, the "slower" of the two goroutines will sit there blocking attempting to send to a channel that will never be read. So this would "leak" a goroutine that would consume resources while the process was still running, but it doesn't have anything to do with the program exiting. https://golang.org/ref/spec#Channel_types https://golang.org/ref/spec#Channel_types https://golang.org/ref/spec#Send_statements https://golang.org/ref/spec#Send_statements > How do we fix this? We simply increase the number of channels to 2 The fix is okay but the language is a bit hazy, there's still only one channel, but now it's a buffered channel with capacity to hold up to 2 messages, so the two goroutines don't need to block waiting for a receiver to be ready to synchronously receive the message they are sending.
- SamReidHughes 8y ago
  Far better to just read twice from the channel, too. It's generally a smell to leave stuff on a channel. And that's a good example why -- you're losing track of a goroutine with no way to wait for completion or interrupt it.
  - shoo 8y ago
    i see what you mean. it'd be pretty strange to wait and see if you got an error from the first goroutine to complete. i'd imagine usually you'd want to wait for the first non-error result to arrive, say, or for all results. maybe there's a scenario where this kind of thing is useful, but it's probably not common
    - seabee 8y ago
      The two most common scenarios in my 3 years of experience are fan-in and first-error (executing stuff for their side effects only). It’s easy to mess up the latter, but golang.org/x/sync/errgroup is usually what you want.
- geocar 8y ago
  > The fix is okay but the language is a bit hazy, there's still only one channel, but now it's a buffered channel with capacity to hold up to 2 messages, so the two goroutines don't need to block waiting for a receiver to be ready to synchronously receive the message they are sending. I was very disappointed with this explanation as well. The documentation is less than direct on this point, and suggests goroutines "execute independently", so my only conclusion is that the author doesn't understand channels very well, and was perhaps led to not understand them well. 01 func doSomethingTwice() error { 02 // Issue occurs below 03 errc := make(chan error) 04 05 go func() { 06 defer fmt.Println("done wth a") 07 errc <- doSomething("a") 08 }() 09 go func() { 10 defer fmt.Println("done with b") 11 errc <- doSomething("b") 12 }() 13 err := <-errc 14 return err 15 } If the programmer understands the control flow is something like: 03, 06, 07, 10, 13, 14 then they're not going to be confused, and they're certainly not going to "fix it" by increasing the buffer, they'll fix it by reading from the channel twice, and guarding against this by closing the channel. The real question is how to explain to the programmer what the semantics of channels is: A goroutine blocks on read or write of an unbuffered channel (what we're seeing here). The garbage collector is for simulating an infinite memory machine only. It is not for "cleaning up" after you. The issue comes up when go programmers learn about runtime.GOMAXPROCS much too early, so they learn goroutines as threads, and they guess that the thread will be cleaned up when "garbage collected" because that's how Python works (or some other "garbage collected" language they might be familiar with). They're further confused because golang values have a "finalizer" so they might just assume that the thread finalizer (somehow) kills the thread, or the channel finalizer (somehow) closes the channel. Perhaps if they noticed that writing to a closed channel causes a run-time panic, they might think these semantics are unlikely. The point about buffering is to be able to write twice without blocking. Yes, you get 03, 06, 07, 10, 11, 13, 14 but how important was that really? If you did it just because you don't want to "leak memory", then you still don't know what's going on.
  - cdoxsey 8y ago
    Increasing the channel size to 2 is the correct fix. You don't want to read twice because then you would block waiting for both messages. When the channel size is two the two inner writing goroutines can succeed on their write to the channel even if nothing reads from it. The channel has three references, the outer receiver, and the two inner senders. The receiver finishes as soon as its able to receive the first message. So what happens is one of the messages is sent and received, the other is sent and just sits in the channel. Then the channel has no more references and is garbage collected, so there's no memory leak. Also goroutines are scheduled onto threads. So these orphaned goroutines that can't complete eat up memory (their call stack + the channel which can't be completed) as well as put pressure on the scheduler. (though the scheduler is efficient and can handle millions of goroutines) That's why it makes sense to refer to this as a memory leak.
- nilsocket 8y ago
  A better solution would be a struct as return value of channel which contains both value and error as fields. So just casually one would handle the case: if ret.err != nil{ //Handle error. }
- nemothekid 8y ago
  I don't like this example and solution, because while it doesn't leaks (the channel goes out of scope, and the goroutine ends) it still is 1. error prone you now have to have a buffered channel, 2. sort of wasteful, setting the buffer to 1 instead of 2, also wouldn't leak and 3. not clear. If you don't care about the results of the goroutine (here you explicitly don't care about the return value of at least one of the routines), a WaitGroup is much better. import "sync" func doSomethingTwice() error { // Issue occurs below var wg sync.WaitGroup() wg.Add(1) go func() { defer fmt.Println("done wth a") wg.Done() }() wg.Add(1) go func() { defer fmt.Println("done with b") wg.Done() }() wg.Wait() return nil } The more advanced concept than this, which isn't in the stdlib is errgroup (https://godoc.org/golang.org/x/sync/errgroup#Group.Wait https://godoc.org/golang.org/x/sync/errgroup#Group.Wait), where instead the accompayning `Wait()` function can also return an error. In general I consider a code smell if you aren't reading every value off the channel.
dis-sys 8y ago
for goroutine leaks, I have been using the leaktest package made by fortytw2 [1] in all my tests, pretty useful for my day-to-day dev work. the article is pretty cool, especially when the author actually has a family name Check. :) [1] https://github.com/fortytw2/leaktest https://github.com/fortytw2/leaktest
brian-armstrong 8y ago
Surely the biggest mistake is using Go
alexott 8y ago
Can’t leave comment in blog, but the first two listings are broken - it looks like there is no empty line after ``` in listing line, and before the same in listing two. So enumeration isn’t rendered correctly
deleted 8y ago
[deleted]
raverbashing 8y ago
Good tips, unfortunately, the explanations are not very clear and it's written in poor English, this gets very distracting.
- SheinhardtWigCo 8y ago
  Which part are you looking at? It all reads quite clearly to this native English speaker.
  - deleted 8y ago
    [deleted]
- sdinsn 8y ago
  It reads fine to me
tomohawk 8y ago
Regarding the defer statements in loops. It is better to just not ever do that. Instead, move the defer outside of any loops, and have the defer check the state of the variable to decide action. For example, in the case of an open file, the loop should close the file and nil out the reference. In the defer, if the reference is not nil, it closes the file.
- masklinn 8y ago
  The issue outlined in the article is not defer in a loop, it's defer in an infinite loop.
deleted 8y ago
[deleted]
CleanShirt 8y ago
Fixed title. Mistakes C/C++ Devs make: writing Go.
ktpsns 8y ago
Here's a newbie's bit off topic question: As a C++ dev who wants to write "slightly less low-level code", should I go for RUST or GO? My intuition says Rust is more like a safer version of C++ while Go is more half way to Python/Julia. Do you agree with this sloppy assessment? (Please, don't start a religious war on programming languages. We are all grown ups)
- cshenton 8y ago
  Yeah that's about right. Go wont fit the bill if you can't trust the garbage collector (often true for scientific computing), or if you're not running your code on a full OS (like if you were writing an OS / embedded code). Since Go is easier to learn, it probably couldn't hurt to learn it, then turn to Rust if it's not powerful enough for your use case.
  - _ph_ 8y ago
    Why wouldn't you be able to trust the garbage collector? Especially with scientific computing, by which you probably mean numerics, your program shouldn't produce much garbage in the first place.
    - cshenton 8y ago
      Sure it will, scientific computing is rife with potentially very large intermediate matrices/tensors of data that need to be deallocated. In C++ I can explicitly free that memory, and in higher level frameworks, like tensorflow, dependency properties of the computational graph as used to figure out the earliest point in time a tensor can be deallocated.
- dbcurtis 8y ago
  I have not studied Go. I am learning Rust. I am not current with C++, but once was fluent. My take on Rust (as a Pythonista) is that while C++ is tedious and error-prone, Rust is merely tedious. Which IS progress. But... if you want to up-level, then I think the goal you seek depends on the state of the Rust crate ecosystem to get you out of the trenches.
agallego 8y ago
This has no c++ content. only C. the mistakes highlighted are just not representative of a team of C devs. the resource leak is specially a bad example because there better examples where go safety is a real benefit.
- pjmlp 8y ago
  A reflection of lots of "C++" code I see in the enterprise, what I call "C compiled with C++ compiler", because usually it doesn't even make use of C with Classes kind of features. Also why the C++ community has started to focus on teaching C++ the right way.
nialv7 8y ago
Sounds more like design mistakes of Go.
ncmncm 8y ago
1.There is no such language as C/C++. 2. There are no developers of C/C++. 3. C and C++ are distinct languages. 4. Anybody advertising for a C/C++ developer is insufficiently aware of software requirements to provide meaningful employment. 5. C++ programmers and C programmers have different responses to unfamiliar languages. 6. Some C programmers would like to be thought of as skilled "C/C++" programmers. There are none. Thus, they are not. 7. Many C++ programmers are capable of altering C code. They, also, are not "C/C++" programmers. When altering C code, they are programming in C, not C++. 8. C++ programmers and C programmers make different kinds of mistakes. 9. Anyone titling an article about "C/C++ programmers" make is making a fundamental category error, and has self-identified as lacking insight. 10. Anyone titling an article about mistakes "C/C++ programmers make" refers to an empty set of programmers. 11. People new to Go make mistakes. (One such mistake might be using Go at all; that is not decided.) 12. There will be some intersection between mistakes made by any two groups of programmers. 13. The intersection between mistakes made by Java programmers and Javascript programmers does not define a "Java/Javascript" language, nor a set of "Java/Javascript programmers", despite any similarity of names or surface syntax between the two.
- SamReidHughes 8y ago
  Wrong. I'm a C/C++ developer.
  - Ws32ok 8y ago
    I too develop software using this mythical language known as C. I haven’t seen any unicorns yet but I do seem to have found a quantity of gold. But no rainbow.
- cmrdporcupine 8y ago
  I guess you're being downvoted for tone, but I agree with you. I program professionally in C++. Waltzing into a non-C++ C code base requires a strong mental shift. I work with embedded programmers new to our team who have a strong history of C programming, and for them, C++ is a very steep hill to climb to become proficient. They are two languages with interoperability and semantic similarities that share a compiler infrastructure and a runtime and memory model. But modern C++ programming differs in drastic ways from C such that I don't think you can call them a unified language and brush over it with 'C/C++ programmers'
axilmar 8y ago
I wouldn't want to use a language that I cannot predict how memory allocation would work. I'd like to have every inch of performance in my disposal for the tasks I am interested in (video games, simulations, data mining).

202 points by beliu 8y ago

shoo 8y ago

re: "# channels < # goroutines" func doSomethingTwice() error { // Issue occurs below errc := make(chan error) go func() { defer fmt.Println("done wth a") errc <- doSomething("a") }() go func() { defer fmt.Println("done with b") errc <- doSomething("b") }() err := <-errc return err } > When one routine writes to the channel, the program exits and the other goroutine is lost, building up memory use as a results I'm not very familiar with go, but my guess would be that since the two goroutines send to an unbuffered channel that is read at most once, the "slower" of the two goroutines will sit there blocking attempting to send to a channel that will never be read. So this would "leak" a goroutine that would consume resources while the process was still running, but it doesn't have anything to do with the program exiting. https://golang.org/ref/spec#Channel_types https://golang.org/ref/spec#Channel_types https://golang.org/ref/spec#Send_statements https://golang.org/ref/spec#Send_statements > How do we fix this? We simply increase the number of channels to 2 The fix is okay but the language is a bit hazy, there's still only one channel, but now it's a buffered channel with capacity to hold up to 2 messages, so the two goroutines don't need to block waiting for a receiver to be ready to synchronously receive the message they are sending.

SamReidHughes 8y ago
Far better to just read twice from the channel, too. It's generally a smell to leave stuff on a channel. And that's a good example why -- you're losing track of a goroutine with no way to wait for completion or interrupt it.
- shoo 8y ago
  i see what you mean. it'd be pretty strange to wait and see if you got an error from the first goroutine to complete. i'd imagine usually you'd want to wait for the first non-error result to arrive, say, or for all results. maybe there's a scenario where this kind of thing is useful, but it's probably not common
  - seabee 8y ago
    The two most common scenarios in my 3 years of experience are fan-in and first-error (executing stuff for their side effects only). It’s easy to mess up the latter, but golang.org/x/sync/errgroup is usually what you want.
geocar 8y ago
> The fix is okay but the language is a bit hazy, there's still only one channel, but now it's a buffered channel with capacity to hold up to 2 messages, so the two goroutines don't need to block waiting for a receiver to be ready to synchronously receive the message they are sending. I was very disappointed with this explanation as well. The documentation is less than direct on this point, and suggests goroutines "execute independently", so my only conclusion is that the author doesn't understand channels very well, and was perhaps led to not understand them well. 01 func doSomethingTwice() error { 02 // Issue occurs below 03 errc := make(chan error) 04 05 go func() { 06 defer fmt.Println("done wth a") 07 errc <- doSomething("a") 08 }() 09 go func() { 10 defer fmt.Println("done with b") 11 errc <- doSomething("b") 12 }() 13 err := <-errc 14 return err 15 } If the programmer understands the control flow is something like: 03, 06, 07, 10, 13, 14 then they're not going to be confused, and they're certainly not going to "fix it" by increasing the buffer, they'll fix it by reading from the channel twice, and guarding against this by closing the channel. The real question is how to explain to the programmer what the semantics of channels is: A goroutine blocks on read or write of an unbuffered channel (what we're seeing here). The garbage collector is for simulating an infinite memory machine only. It is not for "cleaning up" after you. The issue comes up when go programmers learn about runtime.GOMAXPROCS much too early, so they learn goroutines as threads, and they guess that the thread will be cleaned up when "garbage collected" because that's how Python works (or some other "garbage collected" language they might be familiar with). They're further confused because golang values have a "finalizer" so they might just assume that the thread finalizer (somehow) kills the thread, or the channel finalizer (somehow) closes the channel. Perhaps if they noticed that writing to a closed channel causes a run-time panic, they might think these semantics are unlikely. The point about buffering is to be able to write twice without blocking. Yes, you get 03, 06, 07, 10, 11, 13, 14 but how important was that really? If you did it just because you don't want to "leak memory", then you still don't know what's going on.
- cdoxsey 8y ago
  Increasing the channel size to 2 is the correct fix. You don't want to read twice because then you would block waiting for both messages. When the channel size is two the two inner writing goroutines can succeed on their write to the channel even if nothing reads from it. The channel has three references, the outer receiver, and the two inner senders. The receiver finishes as soon as its able to receive the first message. So what happens is one of the messages is sent and received, the other is sent and just sits in the channel. Then the channel has no more references and is garbage collected, so there's no memory leak. Also goroutines are scheduled onto threads. So these orphaned goroutines that can't complete eat up memory (their call stack + the channel which can't be completed) as well as put pressure on the scheduler. (though the scheduler is efficient and can handle millions of goroutines) That's why it makes sense to refer to this as a memory leak.
nilsocket 8y ago
A better solution would be a struct as return value of channel which contains both value and error as fields. So just casually one would handle the case: if ret.err != nil{ //Handle error. }
nemothekid 8y ago
I don't like this example and solution, because while it doesn't leaks (the channel goes out of scope, and the goroutine ends) it still is 1. error prone you now have to have a buffered channel, 2. sort of wasteful, setting the buffer to 1 instead of 2, also wouldn't leak and 3. not clear. If you don't care about the results of the goroutine (here you explicitly don't care about the return value of at least one of the routines), a WaitGroup is much better. import "sync" func doSomethingTwice() error { // Issue occurs below var wg sync.WaitGroup() wg.Add(1) go func() { defer fmt.Println("done wth a") wg.Done() }() wg.Add(1) go func() { defer fmt.Println("done with b") wg.Done() }() wg.Wait() return nil } The more advanced concept than this, which isn't in the stdlib is errgroup (https://godoc.org/golang.org/x/sync/errgroup#Group.Wait https://godoc.org/golang.org/x/sync/errgroup#Group.Wait), where instead the accompayning `Wait()` function can also return an error. In general I consider a code smell if you aren't reading every value off the channel.

dis-sys 8y ago

for goroutine leaks, I have been using the leaktest package made by fortytw2 [1] in all my tests, pretty useful for my day-to-day dev work. the article is pretty cool, especially when the author actually has a family name Check. :) [1] https://github.com/fortytw2/leaktest https://github.com/fortytw2/leaktest

brian-armstrong 8y ago

Surely the biggest mistake is using Go

alexott 8y ago

Can’t leave comment in blog, but the first two listings are broken - it looks like there is no empty line after ``` in listing line, and before the same in listing two. So enumeration isn’t rendered correctly

deleted 8y ago

[deleted]

raverbashing 8y ago

Good tips, unfortunately, the explanations are not very clear and it's written in poor English, this gets very distracting.

SheinhardtWigCo 8y ago
Which part are you looking at? It all reads quite clearly to this native English speaker.
- deleted 8y ago
  [deleted]
sdinsn 8y ago
It reads fine to me

tomohawk 8y ago

Regarding the defer statements in loops. It is better to just not ever do that. Instead, move the defer outside of any loops, and have the defer check the state of the variable to decide action. For example, in the case of an open file, the loop should close the file and nil out the reference. In the defer, if the reference is not nil, it closes the file.

masklinn 8y ago
The issue outlined in the article is not defer in a loop, it's defer in an infinite loop.

deleted 8y ago

[deleted]

CleanShirt 8y ago

Fixed title. Mistakes C/C++ Devs make: writing Go.

ktpsns 8y ago

Here's a newbie's bit off topic question: As a C++ dev who wants to write "slightly less low-level code", should I go for RUST or GO? My intuition says Rust is more like a safer version of C++ while Go is more half way to Python/Julia. Do you agree with this sloppy assessment? (Please, don't start a religious war on programming languages. We are all grown ups)

cshenton 8y ago
Yeah that's about right. Go wont fit the bill if you can't trust the garbage collector (often true for scientific computing), or if you're not running your code on a full OS (like if you were writing an OS / embedded code). Since Go is easier to learn, it probably couldn't hurt to learn it, then turn to Rust if it's not powerful enough for your use case.
- _ph_ 8y ago
  Why wouldn't you be able to trust the garbage collector? Especially with scientific computing, by which you probably mean numerics, your program shouldn't produce much garbage in the first place.
  - cshenton 8y ago
    Sure it will, scientific computing is rife with potentially very large intermediate matrices/tensors of data that need to be deallocated. In C++ I can explicitly free that memory, and in higher level frameworks, like tensorflow, dependency properties of the computational graph as used to figure out the earliest point in time a tensor can be deallocated.
dbcurtis 8y ago
I have not studied Go. I am learning Rust. I am not current with C++, but once was fluent. My take on Rust (as a Pythonista) is that while C++ is tedious and error-prone, Rust is merely tedious. Which IS progress. But... if you want to up-level, then I think the goal you seek depends on the state of the Rust crate ecosystem to get you out of the trenches.

agallego 8y ago

This has no c++ content. only C. the mistakes highlighted are just not representative of a team of C devs. the resource leak is specially a bad example because there better examples where go safety is a real benefit.

pjmlp 8y ago
A reflection of lots of "C++" code I see in the enterprise, what I call "C compiled with C++ compiler", because usually it doesn't even make use of C with Classes kind of features. Also why the C++ community has started to focus on teaching C++ the right way.

nialv7 8y ago

Sounds more like design mistakes of Go.

ncmncm 8y ago

1.There is no such language as C/C++. 2. There are no developers of C/C++. 3. C and C++ are distinct languages. 4. Anybody advertising for a C/C++ developer is insufficiently aware of software requirements to provide meaningful employment. 5. C++ programmers and C programmers have different responses to unfamiliar languages. 6. Some C programmers would like to be thought of as skilled "C/C++" programmers. There are none. Thus, they are not. 7. Many C++ programmers are capable of altering C code. They, also, are not "C/C++" programmers. When altering C code, they are programming in C, not C++. 8. C++ programmers and C programmers make different kinds of mistakes. 9. Anyone titling an article about "C/C++ programmers" make is making a fundamental category error, and has self-identified as lacking insight. 10. Anyone titling an article about mistakes "C/C++ programmers make" refers to an empty set of programmers. 11. People new to Go make mistakes. (One such mistake might be using Go at all; that is not decided.) 12. There will be some intersection between mistakes made by any two groups of programmers. 13. The intersection between mistakes made by Java programmers and Javascript programmers does not define a "Java/Javascript" language, nor a set of "Java/Javascript programmers", despite any similarity of names or surface syntax between the two.

SamReidHughes 8y ago
Wrong. I'm a C/C++ developer.
- Ws32ok 8y ago
  I too develop software using this mythical language known as C. I haven’t seen any unicorns yet but I do seem to have found a quantity of gold. But no rainbow.
cmrdporcupine 8y ago
I guess you're being downvoted for tone, but I agree with you. I program professionally in C++. Waltzing into a non-C++ C code base requires a strong mental shift. I work with embedded programmers new to our team who have a strong history of C programming, and for them, C++ is a very steep hill to climb to become proficient. They are two languages with interoperability and semantic similarities that share a compiler infrastructure and a runtime and memory model. But modern C++ programming differs in drastic ways from C such that I don't think you can call them a unified language and brush over it with 'C/C++ programmers'

axilmar 8y ago

I wouldn't want to use a language that I cannot predict how memory allocation would work. I'd like to have every inch of performance in my disposal for the tasks I am interested in (video games, simulations, data mining).