4 ms·

I want to see an “Effective Go” about unit testing functions that call other functions, without devolving into meaningless mocking and error propagation boilerp

I want to see an “Effective Go” about unit testing functions that call other functions, without devolving into meaningless mocking and error propagation boilerplate.

morelisp 5y ago
For effective testing in Go: - Don't unit test so much (or expand your definition of "unit", or whatever semantic difference you prefer). Test functionality. (Unit tests can still be appropriate for large classes of complicated pure functions, e.g. parsers, but these don't require mocks.) - Don't mock so much; rather, stub (or mock if absolutely necessary) at lower levels. Use the httptest server; use the sqlmock or sqlite drivers; use a net.Conn with canned data; etc.
- closeparen 5y ago
  I would appreciate any resources you could point me towards to help make this argument against the Staff+ engineering leaders at my company who are pushing standards that say exactly the opposite.
  - morelisp 5y ago
    This sounds a lot more like company politics than a technical issue, but I would probably start with Mitchell Hashimoto's talk "Advanced Testing With Go" - along with the just, like, reading the tests / testing tools in stdlib. They didn't include httptest so you could spend time mocking away http.Client usage behind an interface! (I should add that this is explicitly contra to e.g sethammons's suggestion above, which seems to be relatively common in the part of the Go community that come from PHP. I inherited a couple large projects that did this. Today they use sqlite instead, and both the program and test code is ~50% the size it used to be.) For us, stub injection points come naturally out of 12-factor-style application design; the program can already configure the address of the 2-3 other things it needs to talk to or files it needs to output, etc, just out of our need for manual testing or staging vs. production environments. If you have technical leadership encouraging Spring-but-in-Go, you'll probably hit a wall here too though. It's also possible you're simply writing too many functions that can return errors. Over-complex code makes over-complex tests; always think about whether you're handling an error or a programming mistake - if the latter, panic instead of returning.
    - closeparen 5y ago
      Thanks for the suggestion. I watched the talk and found some new information, as well as confirmation of some things I had been starting to adopt. I don't find the stdlib very informative about my problem, since most stdlib packages are "leaf nodes" - not layers that call out to lower layers. I'll check out more of Hashicorp's tests as I suspect their code might be more similar to the kind of code I work on. From a quick glance, in all of Consul I see only a handful of Mockery mocks, suggesting they are doing something very differently.
  - sagichmal 5y ago
    Maintainable software projects are modeled as a dependency graph of components that encapsulate implementation details and depend on other components. func main foo, err := NewFoo() handle err bar, err := NewBar(foo) handle err Given a single component, each external dependency should be injected as an interface. type Fooer interface{ Foo() int } func NewBar(f Fooer) (*Bar, error) { ... } Test components in isolation by providing mock (stub, fake, whatever, it's all meaningless) implementations of its dependencies. func TestBar(t *testing.T) { f := &mockFoo{...} b, err := NewBar(f) ...
    - closeparen 5y ago
      You're giving a baby-level introduction to mocking in a thread about how this approach leads to low-quality, meaningless tests in some cases, and right beneath concrete suggestions about how to make tests better by deviating from this pattern.
      - sagichmal 5y ago
        I'm sorry if you've had bad experiences with this approach in the past, but it emphatically does not lead to low quality and/or meaningless tests. It's the essential foundation of well-abstracted and maintainable software.
        closeparen 5y ago
        Here's a test I wrote recently, in the style expected at my company. Tell me what exactly you think this is contributing to maintainability, or what you think could be done better. I spent an hour on this and found it pure drudgery. I half suspect I could have written a code generator for it in that hour instead. I had no idea whether the code really worked until I ran it against the real upstream. The unit under test is a gateway to a configuration service. https://dpaste.com/FZGC8R66K https://dpaste.com/FZGC8R66K
        sagichmal 5y ago
deanCommie 5y ago
From what I understand (from making similar complaints to Gophers), if you're complaining about boilerplate, you don't get it. The boilerplate propagation is the point in Go. Go isn't DRY. Go would rather copy and paste a bunch of code than introduce the "complexity" of inheritance.
- closeparen 5y ago
  I am okay with the level of abstraction available in production code. Where it gets ridiculous is the tests. Unit testing the simplest, most obvious Go code is a huge chore. Each error-returning function call costs 15 seconds to type but 15 minutes to work into the tests.
  - ed25519FUUU 5y ago
    There is unnecessary friction, which doesn’t really seem to fit with the Go ethos. I would have assumed from afar that testing gets a lot of consideration in a language like Go.
    - alecthomas 5y ago
      It does. It sounds like the parent has had an unfortunate interaction with a bad codebase, because that is not accurate at all.
      - closeparen 5y ago
        I am not being sarcastic when I say I want an "Effective Go" for this. If you have examples of high quality tests in an MVC-style service codebase, I would love to see them!
        sagichmal 5y ago
        "MVC" is not an idiomatic pattern in Go.
sethammons 5y ago
I love testing in Go because I avoid meaningless mocking. Test structs that match interface signatures are meaningfully used to validate any state handling and/or error generation, and ensures error paths are properly exercised. In unit tests, we validate logs and metrics when appropriate in addition to return values. However, if you are mocking db, http or other net packages, you are likely doing it wrong. You want to know you handle an error from the db, you don't have to mock a db: you have a struct whose interface has 'GetUser(username) (*User, error)', and your test version returns an error when needed. The fact the real code will use a db is an implementation detail. You should be able to refactor the real implementation to change from a db call to an http api call and still have valid, useful tests. Elsewise, your unit tests are too coupled and hinder refactoring. Anyway, I love testing in Go; it is one of my favorite parts of working with it.
- closeparen 5y ago
  Mocks are simply auto-generated and consistent test structs. And you still need to test the code which implements GetUser based on DB or HTTP or whatever else.
- eyelidlessness 5y ago
  Disclaimer: I’ve never worked with Go, there may be some nuance here I’m missing. This sounds like dependency injection of test behavior as a substitute for the “real” IO implementations injected in whatever glue code accesses the unit under test. If that’s a correct understanding, it sounds like mocking by another name? Not that I think there’s anything necessarily wrong with that. And I think it that kind of inversion of control can often produce more robust designs/systems/tests. But I think it’s a good idea to recognize that’s what it is, and that it has similar limitations to other mocking techniques.
  - randomswede 5y ago
    There's basically two ways I have seen this work. In one case, you have a "IO thing" stored somewhere in a struct. type example struct { backend db.DB // Never actually used the database library, so this MAY not be the right type name ... } Then your code uses that to call the methods on var.backend, and you can replace it with a test instance. This feels approximately like dependency injection, maybe? Or maybe just encapsulation? And the other is that you pass your "IO" as a parameter. A pretty typical example would be fmt.Fprintf. func Fprintf(out io.Writer, format string, args ...interface{}) (n int, e error) It is of course limited, but all testing is limited. But, most "mocking" I have seen replaces the original type while the test is ongoing, and that specifically is pretty hard in Go. It is trivial to pass in a parameter, or set a struct slot, to something that fulfils a specific interface. And it is often quite useful. And, again, of course something that you may have to write tests for, to ensure that it does what you intend it to (especially if it is complex and needs to hold some state, which unfortunately sometimes happens).
mxz3000 5y ago
What do you mean? If you compose functions (and objects) together then you have to mock things out one way or another, inject them in, and assert that your code interacts correctly with its dependencies. This doesn't seem like a go specific issue but an engineering one.
- closeparen 5y ago
  In Java or Python you do not normally need write a test case for “if this dependency fails, it will be propagated to the caller.” In Go you have to do this or your line/branch coverage will be abysmal. Java and Python mocking is also a lot more lightweight; you don’t have to generate mocks ahead of using them, regenerate them when the interface changes or work generation into your build process, etc. Richer reflection APIs make it a pretty casual handful of characters to mock something out.
  - AlexCoventry 5y ago
    > In Go you have to do this or your line/branch coverage will be abysmal. If reliability is critical, having explicit error handling and coverage tools which expose error conditions which haven't been tested is very helpful.
    - closeparen 5y ago
      I make a sequence of calls during the service of a request. If any one of them fails I want to stop and return the error to the caller. Having to check this in the particular case of every call at every later underlying every handler, does not make my software more reliable in than when it is simply guaranteed in general.
      - AlexCoventry 5y ago
        Explicit error handling can improve reliability in some circumstances, because it highlights corner cases which are easy to ignore otherwise.
  - 5y ago
oxplot 5y ago
Re error propagation: look at standard library’s JSON encoding source code. They use panic internally to bubble up errors from recursive calls and have a recover at the API boundary which turns it into an error value. I thought it was a neat trick, until error handling matures further.

I want to see an “Effective Go” about unit testing functions that call other functions, without devolving into meaningless mocking and error propagation boilerplate.

morelisp 5y ago

For effective testing in Go: - Don't unit test so much (or expand your definition of "unit", or whatever semantic difference you prefer). Test functionality. (Unit tests can still be appropriate for large classes of complicated pure functions, e.g. parsers, but these don't require mocks.) - Don't mock so much; rather, stub (or mock if absolutely necessary) at lower levels. Use the httptest server; use the sqlmock or sqlite drivers; use a net.Conn with canned data; etc.

closeparen 5y ago
I would appreciate any resources you could point me towards to help make this argument against the Staff+ engineering leaders at my company who are pushing standards that say exactly the opposite.
- morelisp 5y ago
  This sounds a lot more like company politics than a technical issue, but I would probably start with Mitchell Hashimoto's talk "Advanced Testing With Go" - along with the just, like, reading the tests / testing tools in stdlib. They didn't include httptest so you could spend time mocking away http.Client usage behind an interface! (I should add that this is explicitly contra to e.g sethammons's suggestion above, which seems to be relatively common in the part of the Go community that come from PHP. I inherited a couple large projects that did this. Today they use sqlite instead, and both the program and test code is ~50% the size it used to be.) For us, stub injection points come naturally out of 12-factor-style application design; the program can already configure the address of the 2-3 other things it needs to talk to or files it needs to output, etc, just out of our need for manual testing or staging vs. production environments. If you have technical leadership encouraging Spring-but-in-Go, you'll probably hit a wall here too though. It's also possible you're simply writing too many functions that can return errors. Over-complex code makes over-complex tests; always think about whether you're handling an error or a programming mistake - if the latter, panic instead of returning.
  - closeparen 5y ago
    Thanks for the suggestion. I watched the talk and found some new information, as well as confirmation of some things I had been starting to adopt. I don't find the stdlib very informative about my problem, since most stdlib packages are "leaf nodes" - not layers that call out to lower layers. I'll check out more of Hashicorp's tests as I suspect their code might be more similar to the kind of code I work on. From a quick glance, in all of Consul I see only a handful of Mockery mocks, suggesting they are doing something very differently.
- sagichmal 5y ago
  Maintainable software projects are modeled as a dependency graph of components that encapsulate implementation details and depend on other components. func main foo, err := NewFoo() handle err bar, err := NewBar(foo) handle err Given a single component, each external dependency should be injected as an interface. type Fooer interface{ Foo() int } func NewBar(f Fooer) (*Bar, error) { ... } Test components in isolation by providing mock (stub, fake, whatever, it's all meaningless) implementations of its dependencies. func TestBar(t *testing.T) { f := &mockFoo{...} b, err := NewBar(f) ...
  - closeparen 5y ago
    You're giving a baby-level introduction to mocking in a thread about how this approach leads to low-quality, meaningless tests in some cases, and right beneath concrete suggestions about how to make tests better by deviating from this pattern.
    - sagichmal 5y ago
      I'm sorry if you've had bad experiences with this approach in the past, but it emphatically does not lead to low quality and/or meaningless tests. It's the essential foundation of well-abstracted and maintainable software.
      - closeparen 5y ago
        Here's a test I wrote recently, in the style expected at my company. Tell me what exactly you think this is contributing to maintainability, or what you think could be done better. I spent an hour on this and found it pure drudgery. I half suspect I could have written a code generator for it in that hour instead. I had no idea whether the code really worked until I ran it against the real upstream. The unit under test is a gateway to a configuration service. https://dpaste.com/FZGC8R66K https://dpaste.com/FZGC8R66K
        sagichmal 5y ago

deanCommie 5y ago

From what I understand (from making similar complaints to Gophers), if you're complaining about boilerplate, you don't get it. The boilerplate propagation is the point in Go. Go isn't DRY. Go would rather copy and paste a bunch of code than introduce the "complexity" of inheritance.

closeparen 5y ago
I am okay with the level of abstraction available in production code. Where it gets ridiculous is the tests. Unit testing the simplest, most obvious Go code is a huge chore. Each error-returning function call costs 15 seconds to type but 15 minutes to work into the tests.
- ed25519FUUU 5y ago
  There is unnecessary friction, which doesn’t really seem to fit with the Go ethos. I would have assumed from afar that testing gets a lot of consideration in a language like Go.
  - alecthomas 5y ago
    It does. It sounds like the parent has had an unfortunate interaction with a bad codebase, because that is not accurate at all.
    - closeparen 5y ago
      I am not being sarcastic when I say I want an "Effective Go" for this. If you have examples of high quality tests in an MVC-style service codebase, I would love to see them!
      - sagichmal 5y ago
        "MVC" is not an idiomatic pattern in Go.

sethammons 5y ago

I love testing in Go because I avoid meaningless mocking. Test structs that match interface signatures are meaningfully used to validate any state handling and/or error generation, and ensures error paths are properly exercised. In unit tests, we validate logs and metrics when appropriate in addition to return values. However, if you are mocking db, http or other net packages, you are likely doing it wrong. You want to know you handle an error from the db, you don't have to mock a db: you have a struct whose interface has 'GetUser(username) (*User, error)', and your test version returns an error when needed. The fact the real code will use a db is an implementation detail. You should be able to refactor the real implementation to change from a db call to an http api call and still have valid, useful tests. Elsewise, your unit tests are too coupled and hinder refactoring. Anyway, I love testing in Go; it is one of my favorite parts of working with it.

closeparen 5y ago
Mocks are simply auto-generated and consistent test structs. And you still need to test the code which implements GetUser based on DB or HTTP or whatever else.
eyelidlessness 5y ago
Disclaimer: I’ve never worked with Go, there may be some nuance here I’m missing. This sounds like dependency injection of test behavior as a substitute for the “real” IO implementations injected in whatever glue code accesses the unit under test. If that’s a correct understanding, it sounds like mocking by another name? Not that I think there’s anything necessarily wrong with that. And I think it that kind of inversion of control can often produce more robust designs/systems/tests. But I think it’s a good idea to recognize that’s what it is, and that it has similar limitations to other mocking techniques.
- randomswede 5y ago
  There's basically two ways I have seen this work. In one case, you have a "IO thing" stored somewhere in a struct. type example struct { backend db.DB // Never actually used the database library, so this MAY not be the right type name ... } Then your code uses that to call the methods on var.backend, and you can replace it with a test instance. This feels approximately like dependency injection, maybe? Or maybe just encapsulation? And the other is that you pass your "IO" as a parameter. A pretty typical example would be fmt.Fprintf. func Fprintf(out io.Writer, format string, args ...interface{}) (n int, e error) It is of course limited, but all testing is limited. But, most "mocking" I have seen replaces the original type while the test is ongoing, and that specifically is pretty hard in Go. It is trivial to pass in a parameter, or set a struct slot, to something that fulfils a specific interface. And it is often quite useful. And, again, of course something that you may have to write tests for, to ensure that it does what you intend it to (especially if it is complex and needs to hold some state, which unfortunately sometimes happens).

mxz3000 5y ago

What do you mean? If you compose functions (and objects) together then you have to mock things out one way or another, inject them in, and assert that your code interacts correctly with its dependencies. This doesn't seem like a go specific issue but an engineering one.

closeparen 5y ago
In Java or Python you do not normally need write a test case for “if this dependency fails, it will be propagated to the caller.” In Go you have to do this or your line/branch coverage will be abysmal. Java and Python mocking is also a lot more lightweight; you don’t have to generate mocks ahead of using them, regenerate them when the interface changes or work generation into your build process, etc. Richer reflection APIs make it a pretty casual handful of characters to mock something out.
- AlexCoventry 5y ago
  > In Go you have to do this or your line/branch coverage will be abysmal. If reliability is critical, having explicit error handling and coverage tools which expose error conditions which haven't been tested is very helpful.
  - closeparen 5y ago
    I make a sequence of calls during the service of a request. If any one of them fails I want to stop and return the error to the caller. Having to check this in the particular case of every call at every later underlying every handler, does not make my software more reliable in than when it is simply guaranteed in general.
    - AlexCoventry 5y ago
      Explicit error handling can improve reliability in some circumstances, because it highlights corner cases which are easy to ignore otherwise.
- 5y ago

oxplot 5y ago

Re error propagation: look at standard library’s JSON encoding source code. They use panic internally to bubble up errors from recursive calls and have a recover at the API boundary which turns it into an error value. I thought it was a neat trick, until error handling matures further.