7 ms·
Opnieuw: A simple and intuitive retrying library for Python
- timwaagh 7y agoStill seems a decent company. Nice that you take the time to properly deal with technical problems instead of just doing the whole 'plough forward even though you're just ploughing air' thing that happens just about everywhere.
- sgt 7y agoI like the name. Maybe this should be merged into requests at some point?
- rkrzr 7y agoWe did an internal naming competition for the library and one of the suggestions was "retreitz - Retries for humans".
- sammorrowdrums 7y agoEn jullie hebben een kantoor in Utrecht! Gezellig. (And you have an office in Utrecht! Cozy.) So that's why you guys speak Dutch anyway!
- sammorrowdrums 7y agoDo you speak Dutch? It's like the Python Zeep (SOAP - API protocol) client is Dutch for soap, Opnieuw is Dutch for again. I always wandered if that was a nod to Guido Van Rossum that people like naming python packages in Dutch?
- tijmendj 7y agoChannable is a Dutch company, and a lot of people who work here speak Dutch. I don't remember where we got it from but it is possibly a nod to Zeep. I joked that we should name our library such that you would have to do from driemaal import scheepsrecht
- Kim_Bruning 7y ago"There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch." -- Tim Peters, The Zen of Python O:-)
- lawlorino 7y agoI believe at least one of the contributors to Opnieuw is Dutch.
- sgt 7y agoNo, but I speak Afrikaans, which is closely related to Dutch. So "opnieuw" is easily understood. In Afrikaans it would be "weer" (or "weer oor").
- mixmastamyk 7y agoMost English names are already taken on PyPI. Using another language is one way to get around it.
- tiku 7y agoThe title is a dutch word, in English translated to "again" or "repeat".
- Kim_Bruning 7y agoYes, but it's one of those words, where pitch and intonation (and -in this case- past memories of dread) are as important as the actual syllables.
- jillesvangurp 7y agoThe vowel combinations always trip up foreigners. Nieuw is Dutch for new and it is pronounced essentially the same. The ew sound basically transliterates to ieuw. Op (on) combined with Nieuw, basically means "on new", or 'again' in proper English.
- mercer 7y agoI'll be the pedant and argue that if you want to figure out how to sound proper Dutch, there's a difference. "new" sounds like "neeuuuuu" in US English, and a diphtong "nuuu" in British English. "nieuw" sounds like "niii" followed by a very short "oooo" as in "do" followed by a slightly more pronounced "w" at the end. I'm aware that there are proper ways to codify pronunciation, but it's more fun this way and I know nothing of it.
- deleted 7y ago[deleted]
- karlicoss 7y agoWould be interesting to read more specific comparisons with existing mature libraries, e.g. backoff [0]. Feeling I get from reading the motivation behind a new library is that most of it can be achieved by using backoff and perhaps a custom delegating decorator that remaps parameter names and does extra checks if necessary. [0] https://github.com/litl/backoff https://github.com/litl/backoff
- leetrout 7y agoI don’t have a better solution but I was taken back by the length of the very verbose / descriptive param retry_window_after_first_call_in_seconds Am I the only one that thinks that is a bit excessive?
- m12k 7y agoIf you're using something like this, then the details of when, what and how you retry is probably something you care about, so having the library be explicit about it and making it possible to know what something means without having to look at the docs seems sensible to me.
- anton_gogolev 7y agoThis is some Java-level explicitness. Maybe, drop the `_seconds` suffix and replace it with something like `retry_window_after_first_call = '60s'`?
- tijmendj 7y agoI agree it might look a bit excessive, but it greatly cuts down on complexity and possible unexpected behavior. If you accept '60s' as 60 seconds, then the next questions are: - Will it also accept '60000ms'? - What does it do if I pass an int or float anyway? Will it implicitly take it to be seconds? - Will it throw an exception if I use an unexpected value? If so, which exceptions can I expect? Will it just chug along and use a default?
- VBprogrammer 7y agoSomething I picked up from another developer is to use timedelta for these type of parameters. It avoids hoping that you get the right granularity for everyone. retry_window_after_first_call=timedelta(minutes=3.5) retry_window_after_first_call=timedelta(days=2) retry_window_after_first_call=timedelta(minutes=50)
- 7y ago
- tignaj 7y agoIf you are not familiar with how retrying is used and why exponential back off, jitter, etc are useful here is a shameless plug to a blog post I wrote recently https://blog.najaryan.net/posts/autonomous-robustness/ https://blog.najaryan.net/posts/autonomous-robustness/ (the first half pretty much is about all these things).
- oefrha 7y agoPersonally I would avoid this kind of npm-esque micro-dependency in my Python projects. I’ve written essentially the same decorator for many of my projects; it takes ten minutes and a few dozen lines of code.
- whateveracct 7y agoWould you really want to roll your own exponential backoff w/jitter for every project though? It's not the easiest to get right, and since you're looping in a somewhat complicated way, it's definitely prone to accidental infinite loops in my experience. Retrying seems perfect for a library to me. For instance, Haskell has a popular retry library https://hackage.haskell.org/package/retry-0.8.1.0/docs/Control-Retry.html https://hackage.haskell.org/package/retry-0.8.1.0/docs/Contr... and I wouldn't consider it bad to just bring it in. Is this more symptom of Python deps being "riskier" than Haskell deps due to types or versioning or something?
- cjfd 7y agoAny dependency is also a risk. What if the next version does not just contain a nice feature that you want but also a nasty and subtle bug that you notice only later? Also, maintenance of a small library is by no means guaranteed to remain at the same level of quality. Generally, one is much safer with larger and more serious libraries. I would say one should just have dependencies that provide some rather serious set of features that would cost a lot of time to implement oneself.
- guitarbill 7y agoNot at all, I think they just have some notion that npm = bad and npm = (small) deps, so (small) deps = bad. I agree, it's harder to get right that it seems from first glance. The number of implementations I've seen with broken or not-honoured timeouts... Python also had a retrying library; apparently it's no longer maintained. I've used the tenacity "fork", which has also evolved/introduced some great features like combinable and chainable wait/backoff.
- 7y ago
- miguendes 7y agoWhat's the difference between this one and tenacity? [1] https://github.com/jd/tenacity https://github.com/jd/tenacity
- BiteCode_dev 7y agoAt least a good readme, which is always a good sign.
- mixmastamyk 7y agoFrom the article, they claim that the parameter names are easier to understand and therefore easier to enter correctly. As I've not used either lib directly, I'll have to take their word for it.
- sdan 7y agoI think I’ll try this, but for devs: a simple while True + try/catch + time.sleep(n*2) will do exponential backoff pretty easily.
- BiteCode_dev 7y agoBut no jitter, and no timeout.
- x3ro 7y agoThe post clearly states the problems they were trying to solve, none of which are taken into account here (consistency, ease of parameterization,..) It seems like people are reading too many "X in Y lines of code" posts :D
- bluebasket 7y agohttps://github.com/invl/retry https://github.com/invl/retry works well for me. only problem is that logging issue that's still open i believe. seems to me that the authors wrote their own library instead of trying to make the ones that didn't meet their need better. well, they never mentioned they tried... anyways, some of their choices like that long variable name are odd to me. i can understand the unit problem though. golang solves that so nicely!
- ekimekim 7y agoThe thing that always gets me with retries is error handling. What errors should be considered retryable, vs raise immediately? Do you log a retryable error? I've had systems mysteriously "hung" before which on closer inspection were infinitely retrying with a retry library that didn't log anything... And if you do exceed your max retries, which error do you raise? The first one? The last one? There's no guarentee they'll be the same. A completely different "TooManyRetries" error? These are the concerns I'd like to see addressed in a high level overview of a library like this one.
- inetknght 7y agoAlso: what if you get different retriable errors every time you re-try: should you have different retry count for each distinct retriable error? When you finally give up, should you report _all_ of the distinct errors or just the earliest or latest retriable error, or ...