6 ms·
Testing Without Mocks
- kshay 4y ago> The factory should create a “Nulled” instance that disables all external communication, but behaves normally in every other respect.... >...For example, calling LoginClient.createNull().getUserInfo(...) should return a default response without actually talking to the third-party login service. So... a mock.
- BiteCode_dev 4y agoExactly. All those talks are always going to tell you the same things: - limit side effects to your program boundaries - abstract boundaries with utility functions - mock program boundaries for integration tests in those utility functions And then they pretend they didn't see the mock part.
- kaczordon 4y agoNo that’s not a mock that’s a stub. See this book from software engineers at Google, it talks about various test doubles: https://abseil.io/resources/swe-book/html/ch13.html https://abseil.io/resources/swe-book/html/ch13.html
- jmholla 4y agoThey use the word mock several times to explain stubbing. I think they mean stubbing is a type of mocking not an independent approach. They even use a mocking library to create stubs in the Google article.
- BiteCode_dev 4y agoIndeed, like people stating something is a deferred or a future instead of promise, and then pointing out a minor difference in their approach that justifies a whole new taxonomy.
- Izkata 4y ago"Mock" can be both a noun and a verb. As a noun, it's an alternative to stubs, fakes, and other such things. As a verb, it refers to the "monkey patching" method of putting those into place, in a way that undoing it is automatic and won't leak into other tests. The title seems to be referring to this second version, showing us ways to avoid patching with things like dependency injection.
- inadequatespace 4y agoAlso https://www.martinfowler.com/articles/mocksArentStubs.html https://www.martinfowler.com/articles/mocksArentStubs.html
- Aeolun 4y agoMocks are for objects, stubs are for responses and ‘fakes’ are basically the same thing as mocks.
- Cyph0n 4y agoFakes are not mocks. A fake is an actual (simplified) implementation of the dependency. For example, if your dependency is a distributed key-value store, a fake would expose the same API but using an in-memory hashmap under the hood.
- Aeolun 4y agoYeah no, that’s a mock. At least, that’s been a mock ever since I started prgramming 15 years ago. Only later did libraries that automatically mock dependencies appear (and I presume people suddenly felt a need for a new name for not-automatically mocked dependencies?
- lnenad 4y agoI think the issue is "a mock" is a generalized term that implies different things depending on the usage. More junior developers think that it only means when you have an external entity that you have control over that "mocks" the external service that you don't (shameless plug, I built https://mockadillo.com https://mockadillo.com for this). But yeah, these are obviously mocks and the article kind of doesn't make sense.
- jotaen 4y agoI’m not sure I manage to follow the author correctly, but to me it seems that by “mock” they strictly refer to external mocking libraries, in the sense of JS’s `testdouble` [1] or Java’s `Mockito` [2]. The static fake object as returned by `LoginClient.createNull().getUserInfo(...)` appears to be a “Nullable” for them. [1] https://www.npmjs.com/package/testdouble https://www.npmjs.com/package/testdouble [2] https://site.mockito.org/ https://site.mockito.org/
- drewcoo 4y ago> Folks in the know use mocks and spies (I say “mocks” for short in this article)
- jrockway 4y agoI think we call those "fakes", and reserve mocks for those tests that count the number of times a function was called, yielding "change detector tests". Personally, I find it best to not be too concerned about the terminology, and make sure your test doubles get the data that the code under test needs to the place that needs it. The more real code you can involve in integration tests, the better; only look at replacing real code with test doubles if there is a performance or reliability downside. Databases have never hit that performance/reliability downside for me; network services do.
- kaczordon 4y agoNo, fakes are specifically designed to emulate the underlying dependency somehow so that you can assert on it’s state later. Stubs don’t try to do that at all and are mostly for returning a default response.
- dkarl 4y ago> Personally, I find it best to not be too concerned about the terminology I second this. My experience is that some people know this terminology if they're actively learning or teaching the techniques, but most of the people I've worked with who know the techniques and apply them successfully don't know the terminology and use the word "mock" universally, so it usually creates more confusion than clarity to try to distinguish the different kinds of test doubles by name.
- akkartik 4y agoThe nulled instance continues to be available in production, which has different pros and cons than a conventional mock class which is only available when running tests. Seems worth a different name so we can compare and contrast the two.
- tengbretson 4y agoI get that in other software stacks you might want to avoid mocks and stubs due to friction, but in JavaScript what's the big deal? Mocks are easy– just do it.
- kaczordon 4y agoMocks and stubs provide lower test fidelity than the real thing. Also makes your tests more brittle as you need to update them whenever the real implementation changes.
- tengbretson 4y agoOf course test the real thing if you can. The introduction of mocks and stubs to the conversation to me has already implied that is not possible.
- gspencley 4y agoMocks, especially spies, couple your tests to the implementation of the system under test. This leads to a very common code smell in tests: Fragile Test. The problem is that instead of being concerned only with behaviour, the test are knowing something about the implementation details of the SUT (dependencies are an implementation detail that consumers ought to be unaware of). One of the major benefits of automated tests is that they can provide a safety net that allows you to refactor with less risk of breaking behaviour. If tests get in your way of refactoring, because changing the implementation details of your SUT cause your tests to start failing, then the tests have failed at being a tool to facilitate refactoring.
- revskill 4y agoIf you make all your code as pure function, then testing is easy. First thing to do: Stop using class. Ask yourself first: Should i use a class here and there ? Wait, i miss something here: Writing pure functions is hard ?
- BiteCode_dev 4y agoAlright, so you get numbers from a json file in a ftp servers, you use scipy.newton to perform root finding on it, but needs the number of steps to be outputted as well (which newton() doesn't return), and you will save the result in a postgres table. All this should be send to a task queue, triggered by a call to your REST API. Good luck with purity.
- pancrufty 4y agoYou’re describing a bad API to begin with. Those are too many steps to test at once too.
- nordsieck 4y ago> Good luck with purity. Most of your counter example needs integration tests to test properly anyhow - not really a good point. If you can give up encapsulation (which IMO isn't actually that useful in practice) and in return, your program becomes 95% functional, that seems like a really good tradeoff.
- nemetroid 4y ago1. code that listens to a REST API and puts a request on a task queue 2. code that listens to the task queue for new jobs 3. code that fetches a file from the FTP server 4. code that converts the JSON file to a format appropriate for the root finding 5. code that performs the root finding 6. code that converts the result of (3) to a format appropriate for insertion into a table 7. code that inserts the result into a Postgres table Steps 4, 5 and 6 seem easy enough to implement in a pure fashion.
- Izkata 4y ago
- britch 4y agoA "fake" (as opposed to a mock) is a pretty well understood concept. Is there a difference between a fake and a "nullable"? As far as I can tell this is a pattern for merging your fakes and reals which seems cumbersome and error prone
- jerf 4y agoThis seems close to what I do, except I dependency inject the alternate implementations rather than modifying production ones to do the job, which seems weird. Plus I may want more than one implementation. At least at the scale of code I've been working with, it has been excellent. People keep threatening me with dire promises of how this or that testing practice will result in fragile, highly-coupled tests, and all I can say in response is that in 10+ years of continuously writing tests you'd think it would have happened to me by now. Instead I tend to make very reasonable changes to my tests as I go, and the only times I've ever ended up rewriting them top to bottom is when the underlying code under test changed so much that it was inevitable anyhow. I tend to have a test module that sets up an entire test environment with all the stub implementations, which works fairly well in that middle ground between full integration and unit tests.
- bbq 4y agoLots of people in the comments here stuck on the boring semantic definition debate of mocks vs stubs vs nulls vs whatever. This is not really by what the article is about IMHO. It’s a great overview of how to grow and organize your application with some opinionated patterns that give some favorable properties to your system. The nullable stuff just helps achieve those properties. If you want to use a mocking library to write nullables, and call them mocks, fine. Not important.
- willio58 4y agoAgreed and I find this happening a lot in discussion of testing. It seems everyone/every company has "their way" of doing testing, which at times even includes new words for certain concepts that they find useful (e.g. stubs) and then the conversation about testing gets muddled by people doing things/naming things differently. I only recently got into testing my code and while I love writing tests, getting started was kind of difficult because there was just so much noise. I eventually had to find one person to follow in my space (Kent C. Dodds) and just kind of go all in on his methodology.
- GrumpyNl 4y agoI don't always test my code But when I do, I do it in production.
- jotaen 4y agoI think this might be a problem of expectation management that could stem from the title. A headline along the lines of “Patterns to organize your code for testability” might reflect the general character of the article more clearly. But as it specifically says “testing without mocks”, I was intuitively expecting something different, and even was a bit confused when the author suggested techniques that I would refer to as “mocking”.
- kshay 4y agoYes, exactly. “Bake your mocks into your third-party library implementations” is an interesting idea and might be a useful thing to do, but it’s certainly not “testing without mocks.”
- stuaxo 4y agoObviously he's keeping the interesting bit about how this works for the actual course, but I already only mock bits right on the edge, e.g. the point where the system would connect to the outside world, and that seems to work. What I'd really like, is some way of acknowledging redundancy - lots of tests ultimately go through the same code underneath, it would be good if that could be tested only once, and then the data replayed from those points but done in such a way that everything is still nice and readable.
- jdlshore 4y ago> Obviously he's keeping the interesting bit about how this works for the actual course No. Absolutely everything in the course is in the article. The benefit of the course is that you get to interact with me directly. You can ask for clarification, conduct exercises involving real-world code, and bring your own code for my feedback. It's a guided tour, but you don't need it. Everything is in the article.
- planede 4y agoMy introduction to mocking was through C++ and googlemock, where you have to dependency inject the mock object to do any mocking. This always made sense to me, and could often make APIs nicer and more generic with this approach. The API for dependency injection just ends up an other public interface that you need to test, and one natural way to test it is with mock objects. Then I learned about pytest, magicmock, and that the norm for other testing frameworks in other (mostly dynamic) languages is that you can replace whatever member or free function with a mock without this being part of the public interface of the original class or function. This feels insane to me, as this just tests implementation details. Is there terminology to distinguish between these two kinds of mocking?
- Deadron 4y agoPoorly vs well structured code.
- heywhatupboys 4y agothis is what happens when you make an OO language without interfaces... Oh how I miss Java et al. in situations like reading this blog
- jdlshore 4y agoHi everyone, author here. This article tends to raise a lot of questions. I'm happy to answer them. (The most common question: isn't it the same as a mock? The answer is no, mocks are used for isolated, interaction-based testing, and this is for sociable, state-based testing. These are polar opposite approaches to testing with distinct tradeoffs. If you're not familiar with those terms, they're defined in the article in the "Foundational Patterns" section.)
- brtkdotse 4y agoSeems like a lot of work just to avoid mocks. Don’t know if this is language specific but I find it’s really easy to create mocks and/or in-memory substitutions in .NET using something like Moq.
- dang 4y agoReposts are fine after a year or so (https://news.ycombinator.com/newsfaq.html https://news.ycombinator.com/newsfaq.html) but this had a major thread just a month ago, so it counts as a dupe: Testing Without Mocks: A Pattern Language - https://news.ycombinator.com/item?id=34293631 https://news.ycombinator.com/item?id=34293631 - Jan 2023 (76 comments) Previously: Testing Without Mocks: A Pattern Language (2018) - https://news.ycombinator.com/item?id=30565918 https://news.ycombinator.com/item?id=30565918 - March 2022 (28 comments) Testing Without Mocks: A Pattern Language - https://news.ycombinator.com/item?id=16943876 https://news.ycombinator.com/item?id=16943876 - April 2018 (12 comments)