10 ms·
How to unit test code that depends on time
- xmprt 3y agoIt's a shame that the recommended solution is using a C++ trick of templates that doesn't generalize well to other languages.
- keithnz 3y agowell, in the latest .NET 8, for any of the .NET languages there is https://learn.microsoft.com/en-us/dotnet/core/whats-new/dotnet-8#time-abstraction https://learn.microsoft.com/en-us/dotnet/core/whats-new/dotn... Though most people were rolling their own version of it before this.
- jweir 3y agoWhen possible pass the time as an argument and create pure functions or methods. Reserve Time.now or its equavlent to jobs, controllers, or what not at the edge of your application. Watch out when using NOW() in your db. Not that you shouldn’t but it might be surprising when you do.
- zer00eyz 3y ago>> Watch out when using NOW() in your db. Would you be so kind as to elaborate on this? It was always my thought that NOW() was the more sensible approach as it would capture time in the execution order from a more centralized source (the db server). Where having the origin of a transaction setting at time would get me when, on the origin server the query was initiated.
- macintux 3y agoNot the person to whom you're replying, but I suspect the problem isn't that the database is using NOW(), but that your code being tested will be using an entirely different time.
- jweir 3y agoBingo. There is nothing wrong with using NOW(), in of itself, and it maybe the best thing to do. But it may lead to surprising results when say you have an application that has mocked out time for a specific time, but you get db results for the current time. It introduces another clock into your system. I have found most of the time we were using NOW() we could pass the time into the query and create a more understandable chain of code.
- theamk 3y agoYou mostly want to be consistent. For example if you do a job scheduler which uses NOW() to create job, but a local clock to execute them, then any clock difference between machines will be causing hard-to-debug problems.
- omeid2 3y agoIf you can't test your code because you use `NOW()` or another form of automatic timestamping in your records, it is on your language or framework. You shouldn't forgo such niceties for limitations imposed by your framework or language, demand better.
- whynotmaybe 3y agoWe use a "service" class with an internal timer that returns ".now()" and that implements an interface. The .ctor initialize the service's date with Datetime.now(). We use dependency injection wherever the service is required. With this, we can manage 2 things 1. The service can go faster or slower than the real time by changing the timer's interval so we can simulate time depend behaviour in the app while testing. 2. We can mock the service when testing with a simple counter that's increased each time “.now()" is called.
- nostrademons 3y agoThe issue doesn't generalize across languages because they often have other mechanisms that are simpler. In dynamic languages with modules like Javascript, Ruby, or Python, you can usually monkey-patch the system time methods to return what you want. In Java and other JVM languages you'd use dependency injection and then inject a mock into the test. Most other languages use some variant of Approach 1 or Approach 5.
- squidsoup 3y agoThis made me realise, that despite the many complaints I have about jest and the node ecosystem generally, we’re actually very lucky that it has so many convenient affordances like modern timers.
- Xophmeister 3y agoTime is a dependency like any other, so use dependency inversion. (Which I think is advocated by TFA’s approaches 4 and 5.) Define an interface for timers, which is injected into the object that depends on time. Use a wrapper implementation over the usual timer for the real code; and a simple implementation, which can be controlled parametrically, in the tests.
- calebio 3y agoI agree. I first started doing this (instead of the often recommended global monkey patch) when I started programming in Go. In our main application we pass in a function that returns `time.Now()`, and in tests we pass in a function that returns whatever time object we want.
- BoiledCabbage 3y agoDependency injection of just the function you need is good, but is still the second best approach. The best approach is separating the calling of the external dependency (getting the time) to it's own function, and pass in the results of that call to your main function. Then test your main function. Ie approach 5. This eliminates the need for mocks and creates better factored code.
- theamk 3y agoThat works until you have a function that implements timeout and so needs to query time multiple times per invocation.
- JimmyAustin 3y agoThe function you pass in can keep track of how many times it’s been called, and provide different values on each call.
- Anon1096 3y agoThis is way way worse than passing in a timer and destroys your ability to write multi threaded code (among other defects).
- ForkMeOnTinder 3y agoThe easiest way to do this (in my experience) is libfaketime. I'm surprised not to see it mentioned. https://github.com/wolfcw/libfaketime https://github.com/wolfcw/libfaketime
- irrational 3y agoWhy the surprise? There is too much to know for anyone to know everything.
- theamk 3y agoI don't think it's that easy for the typical unit test though? Most of the time usage need to advance time in controlled fashion. So you'll need to create a shell wrapper that runs faketime with correct FAKETIME_TIMESTAMP_FILE value, hook it up to your build system, create C++ wrapper to write to this file... If you are controlling source code, adding a "time" argument might be easier to implement and easier to understand.
- deivid 3y agolibfaketime's awkward interface (control via timestamp files, needing LD_PRELOAD) is why I wrote an experimental library[0] which takes over the clock at the process level. My library modifies the code for the time-related vDSO functions, which is incredibly sketchy yet effective. There are also Python bindings at [1]. [0]: https://github.com/DavidVentura/tpom https://github.com/DavidVentura/tpom [1]: https://github.com/DavidVentura/py-tpom https://github.com/DavidVentura/py-tpom
- Brybry 3y agoWouldn't the FAKETIME environment variables be easy for typical unit testing?
- deleted 3y ago[deleted]
- p1necone 3y agoJest has pretty robust time mocking, it's a breeze to do time related testing if you're writing JS/TS: https://jestjs.io/docs/jest-object#fake-timers https://jestjs.io/docs/jest-object#fake-timers
- theamk 3y agothis blogpost is specific for C++ -- in most dynamic languages like Python or JS, the default approach is to redefine time functions from standard libraries. This would correspond to option 3 in the blog post, except instead of clock factory you redefine individual methods.
- s4i 3y agoA big blind spot in Jest (and I assume in Vitest although I have no experience) are the timezones. It’s very tricky to run the tests in other TZ’s than what Node has been started with.
- jackblemming 3y agoDon’t do something silly like take a parameter called current_time. Instead, name it what it’s actually used for or doing. This could be send_emails_on or the like. Parameters should tell the reader what not why.
- Sankozi 3y ago`send_emails_on` suggests strongly some scheduling capability. `current_time` says exactly what it is needed for.
- jackblemming 3y ago>`send_emails_on` suggests strongly some scheduling capability Yes. >`current_time` says exactly what it is needed for. Not always.
- loloquwowndueo 3y ago* in C++. On Python, just use freezegun to inject controllable timestamps in response to calls to time methods. https://github.com/spulec/freezegun https://github.com/spulec/freezegun
- boxed 3y agoFreezegun is pretty much unmaintained. You want time-machine https://github.com/adamchainz/time-machine https://github.com/adamchainz/time-machine
- pmontra 3y agoThe Ruby equivalent is Timecop https://github.com/travisjeffery/timecop https://github.com/travisjeffery/timecop Dynamic languages have the advantage to be able to rewrite the standard library classes at runtime.
- sroussey 3y agoI used to change an objects class itself in runtime in JS by changing its __proto__ property. I don’t think that works anymore, but at the time I could create plain value objects and later make them class instantiations (on a small system way back when, this was a nice speed up due to a now uncommon pattern of having data first and waiting for the classes to load).
- Brybry 3y agoOther languages can use LD_PRELOAD (or DLL injection on Windows) as long as the binary isn't statically linked. See other comments in the thread about libfaketime.
- pmontra 3y agoI read the documentation about libfaketime and I saw that it doesn't work in combination with other libraries that also play with LD_PRELOAD. Timecop could fail if something else also wraps the date and time classes but its block scoped form should keep working. I didn't check. Furthermore I don't know what's out there that does it. I should write my own code.
- dissident_coder 3y agoAs a Ruby/Rails developer, time mocking is awesome. It's built right into rails or you can use Timecop and it's just as simple as adding: around { |example| Timecop.freeze("2023-12-20T12:00:00Z".to_datetime) { ex.run } } or around { |example| travel_to("2023-12-20T12:00:00Z".to_datetime) { ex.run } } as setup hooks to your tests and it stubs all of the relevant datetime helpers for you for each test case and properly restores everything between tests.
- mx_03 3y ago>This is better. Now the only code that differs between a unit test build and a production build is what clock_factory::get_clock() returns. So you are effectively testing code that will never see the light in Production. That's called a Mock and some people hate mocks. I have no problem against it taking for granted the actual lib has its own tests.
- eitland 3y ago> That's called a Mock and some people hate mocks. Don't worry to much about them. Mocks are perfectly OK. It is like putting a mechanical device in a jig for a durability test or the exhaust extractors that mechanics use when they run engines inside the workshop.
- bluGill 3y agoMocks are okay, if they are used correctly. Often I see them over constraining code and then you cannot change it. If the code will not change delete the tests at they will never fail.
- eitland 3y ago> If the code will not change delete the tests at they will never fail. Or, unless they are expensive, just keep them. The code doesn't change so they won't break. But for the next maintainer they will tell them a lot. One particularly nice thing about some code I maintained recently was that it allowed me to run through different scenarios in my debugger even during my first days in that code base. Also I don't think I will ever again assume that some code that is in production will never change.
- bluGill 3y agoI generally assume that production code will change in the future. Thus if the tests you write today do not guide those changes to not break other existing functionality since they so constrain your implementation that those tests will need to be deleted then the tests are not valuable. I've seen code where every dependency was mocked and all tests ended up being the implementation calls some function with some arguments. Most of the functions can be called many different ways to get the same answers, but the tests forced a specific number.
- thrtythreeforty 3y agoI do not think Approach 2 is sound. It proposes specialization of a template without needing to recompile the library source, implying that the specialization will be done only in a separate translation unit. The standard §13.9.4 says: > If a template [...] is explicitly specialized, a declaration of that specialization shall be reachable from every use of that specialization that would cause an implicit instantiation to take place What's going to happen in practice is the compiler will instantiate and inline the unspecialized template in the library, and the program will silently not use the mock time. Or not, it's entirely unconstrained.
- deleted 3y ago[deleted]
- pimlottc 3y ago> If you can live with the loss of precision from Approach 5, pass time stamps, I was confused why a passed-in timestamp would have less granularity than the return value of gettime() (or whatever) directly. The issue isn’t a loss of /precision/, but a loss of /accuracy/ due to added latency between observing the time outside the function and making the decision within the function.
- ikari_pl 3y agoInteresting observation: my first language is Polish, and we'd use the same word for accuracy and precision ("dokładność").
- V-2 3y agoReally? Precision translates to (unsurprisingly) "precyzja", and in the specific context of calculations and measurements, the distinction is well established, see eg.: * https://centrumdruku3d.pl/jak-zrozumiec-roznice-pomiedzy-dokladnoscia-precyzja-a-tolerancja/ https://centrumdruku3d.pl/jak-zrozumiec-roznice-pomiedzy-dok... * https://automatykab2b.pl/technika/54301-dokladnosc-i-precyzja-kluczowe-w-systemach-pomiarowych-i-automatyki https://automatykab2b.pl/technika/54301-dokladnosc-i-precyzj... * https://learn.microsoft.com/pl-pl/office/troubleshoot/access/floating-calculations-info https://learn.microsoft.com/pl-pl/office/troubleshoot/access... * https://pl.wikipedia.org/wiki/Dok%C5%82adno%C5%9B%C4%87_i_precyzja_metody_pomiaru https://pl.wikipedia.org/wiki/Dok%C5%82adno%C5%9B%C4%87_i_pr... etc.
- geraldhh 3y agosame with efficiency and effectiveness, a lot of ppl just don't know the difference
- zubairq 3y agoSome useful tips I can use, thanks
- matsemann 3y agoI recently wrote some logic that's supposed to run every minute, and look at the data from the last minute. Normally I would've ended up doing queries ala event_time BETWEEN now() and now() - 1m. However, one constraint was that I needed to also run this on existing data. Which led to me instead passing in time. A small thing, but suddenly this logic beame so incredibly testable. I can call it 5 times with 5 different times and see that it behaves as expected, or simulate it over a whole day of test data. Even tweak the code a bit, delete the data it generates in prod, and re-run it with the new ruleset. So flexible. Such a small thing, but without the additional constraint I probably wouldn't have arrived at this design.
- ianwong1999 3y agoThe code example in approach 2 looks ill-formed though. If you change line 12 to auto to let compiler deduce the return type, you will see error message "explicit specialization of 'clock_impl' after instantiation". ref: https://stackoverflow.com/questions/36997351/using-a-template-before-its-specialized https://stackoverflow.com/questions/36997351/using-a-templat... So not very sure how to elegantly inject our mock into the main code via the test section.
- abaymado 3y agoUsually, when I come across an article like this, I tend to glance at it and skip it, since I don't know C++. However, kudos to OpenAI; ChatCGPT shines in moments like this. The core knowledge of the article would have been bounded to C++ but, now you can just translate the article code with what you are comfortable with.
- zokier 3y agoWhile it would not be appropriate for unit testing, for more generally time related testing it annoys me greatly that while Linux has time namespaces, they only affect monotonic and boottime clocks, and not realtime clock. I understand that there are good reasons why that is the case, yet it is still something I've considered patching in myself
- cloogshicer 3y agoA bit disappointed by the article. It's more about how to test any external dependency using different injection techniques, not just time. What I was hoping for was something like simulations or UIs/games. Complex user interactions like "Sarah clicked here, waited a few seconds, then clicked this, then waited for a network request to finish..."
- bogota 3y agoThat sounds like integration tests if the network is involved
- cloogshicer 3y agoDoesn't have to be. Could also mock that part.
- theK 3y agoIm a bit surprised that this (very C++ biased) post gets such traction on hacker news. Isn't this a solved problem viz moving side effects like IO (getting the time) out of the logic parts and just unit/property testing those? Am I missing something here? EDIT: added last sentence
- mrkeen 3y ago> Isn't this a solved problem viz moving side effects like IO (getting the time) out of the logic parts and just unit/property testing those? Yep, and it delivers even more than what was asked, because now you can test "code that depends on X", not just time. Why it hasn't gained much traction is the camp of coders who will "want to make it simpler" by removing such indirection and making the code ask for the time directly.
- bsder 3y agoTime is state. State is passed in at the boundaries. End of story. Anything which relies on time coming in as an implicit variable as internal state is doomed to have legions of undebuggable problems.
- heax 3y agoTime in software is fascinating. If you say time - everyone thinks he understands it perfectly as we use it in our everyday. But it has so many pitfalls in the details.
- federiconafria 3y agoAlso applicable to "randomness" https://youtube.com/shorts/G2uaZROAWx0?si=5ecJFGU0BfF5y4u7 https://youtube.com/shorts/G2uaZROAWx0?si=5ecJFGU0BfF5y4u7
- otikik 3y agoOk but what I’m really interested in is integration tests that depend on time. Waiting for a set amount of time (i.e. literally calling to wait(5) in the test code) is slow and, even worse than that, flaky. If the test server is busy it might take it longer than 5 seconds to do what the test needs to happen. The best approach I have found so far is “pulling for the side effects” (i.e. if the test is awaiting the creation of a flobber object, keep pulling the list of flobber objects until a new one pops up). This is not always possible (the interface for pulling the flobber objects is not always available) and it is also inconsistent with multiple threads/processes(one thread might have the new flobber and tell your test that everything is ok, but the next step in the test you happen to be served by a different thread which doesn’t have the new flobber yet).
- MoreQARespect 3y agoMy approaches: 1. libfaketime 2. Make the test resilient to changing time 3. Dependency inject at an even higher level
- groestl 3y agoA multi-tenant timeline (i.e. a tenant local clock) with a side channel for the testing harness to modify the time is what I found most practical when doing integration tests, and the test itself depends on time. (Edit: when testing Java servers)
- ivan_gammel 3y agoIn modern Java it’s pretty common to inject an application-level clock object in all components depending on time. Writing an unit test becomes easy, because you have full control over what time is returned on each call.
- ragebol 3y agoRecently had this problem, went with option 5 indeed, passing timestamps. This also made it trivial to replay logs to the application and get back in the exact same state, which is great for debugging of course.
- angelaguilera 3y agoThe book "Test Driven: Practical TDD and Acceptance TDD for Java Developers" (Lasse Koskela, ed. Manning) includes a chapter titled "Test-driving the unpredictable", with several techniques for testing time-based functionality. Highly recommended.
- hknmtt 3y agoIn Go, I would normally have my own time function which would return the std lib time function, which works flawlessly for testing my own code. The problem is if there is foreign code(ie. 3rd library dependency) that uses time and which makes obviously calls to the std library time, making it untestable. The only solution is to overwrite time on language level by some trickery(not all languages allow this) or literally change the OS's clock and test manually one test at a time. This is one of those unsolved programming issues.
- nine_k 3y ago"What do we say to the God of Unit Testing? — Not today()!" Stub out the calls responsible for reading system clock, then set it deterministically, and advance it programmatically where a test depends on the passage of time. This works really well, reads really well in the source, and allows to test general cases and edge cases equally easily.
- SideburnsOfDoom 3y agoIn the .NET world, the TimeProvider class, is in the .NET framework from version 8 onwards. (1) And a "wall clock" or "test clock" injected using DI as needed. (equivalent to option 1 + 4) Write your own "test clock" or use the supplied fake: (2) In earlier versions, we rolled our own "clock" interface. 1) https://learn.microsoft.com/en-us/dotnet/api/system.timeprovider?view=net-8.0 https://learn.microsoft.com/en-us/dotnet/api/system.timeprov... 2) https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.time.testing.faketimeprovider https://learn.microsoft.com/en-us/dotnet/api/microsoft.exten...
- crabbone 3y agoThis whole "how to" is very simplistic. It shows that it was written by a programmer who never done any serious testing. Testing code that depends on time is very complicated, and solutions aren't about whether you have factory methods or use C++ templates. That's by and large irrelevant. To the best of my knowledge there isn't any strategy that would handle all aspects of testing time-sensitive code. The goals and means are very different between different test cases. I wouldn't even dare to try to catalogue all the possible problems with testing time-sensitive programs... this is just too big of a subject to try to cover in few paragraphs.
- Fire-Dragon-DoL 3y agoI take an optional parameter time that when unset defaults to time.now. Makes testing trivial and operative code trivial