12 ms·
Python decorators they don't tell you about
- tedmiston 9y agoIf you're on mobile, view the page as a desktop site in your browser to see the IPython notebook rendered properly. This works on iOS at least.
- dmurray 9y agoSame in Chrome on Android. Thanks!
- brian_herman 9y agoCan you make a decorator that references a class instance and has parameters?
- JonathonW 9y agoYes on the former (works just like you'd expect); to make a decorator take parameters, to do something like: @decorator_with_params("Some string") def my_function(): pass you have to make 'decorator_with_params' a function that returns a decorator; i.e. def decorator_with_params(param): def real_decorator(function): def inner(*args, **kwargs): print(param) function(*args, **kwargs) return inner return real_decorator (You can also do something similar with callable objects, but that's ickier IMO.) Demo of both (decorator as instance method and decorator with parameters): https://gist.github.com/jonathonw/dac88b715aca4e4f1b2f513e1feaa102 https://gist.github.com/jonathonw/dac88b715aca4e4f1b2f513e1f...
- myaso 9y agoSo just a function that returns a closure? It can be a real pain to read code like this if it's overdone -- I really would need to trust somebody to let them go meta at their discretion.
- daveFNbuck 9y agoThe key is to have powerful ones that you re-use a lot. Then most of the time you just have simpler functions with standardized annotations.
- mst 9y agoI can't see any reason you can't do something like: class MetaDec(args, here): ... def __call__(self, func): # Decorate function here def meta_dec(args, here): return MetaDec(args, here) @meta_dec def decorated: ... (apologies for any syntax fails, python is not my best language) At which point one would imagine you can abstract out the __call__ part and just have a 'decorate' routine on any class designed to work what way or similar. Whether this is an improvement or not over closures almost certainly depends on the use case, of course.
- myaso 9y agoDoes the code actually work? Wouldn't you need to create a instance within meta_dec and return the result of the __call__ method. See at first glance I won't think it would work if I didn't know __call__ is a magic method in python. I don't think I'm being overly picky but I found the previous example much easier to read -- since it's a pattern I defaulted to myself before across several different languages. Mixing classes and FP like that doesn't resonate well with me tbh, but yeah you could do it ... but should you?
- mst 9y ago> Does the code actually work? Given its reliance on '...' and the fact I'm not great at python and typed it straight into the comment ... no. No it doesn't. > Wouldn't you need to create a instance within meta_dec and return the result of the __call__ method. No, like with ES6 parameterised decorators return a function that then gets called with the thing being decorated. > See at first glance I won't think it would work if I didn't know __call__ is a magic method in python. (1) my reply was assuming the context of us both having just read the article, which has an example with __call__ (2) as I said in my original comment, you would abstract this away (and document it), I was sketching out an alternative implementation path to allow you to have an object instead of a closure since you didn't seem to be happy with closures, not offering a complete solution > Mixing classes and FP like that doesn't resonate well with me tbh, but yeah you could do it ... but should you? (3) if you don't like closures and you don't like objects, I'm not sure what other combined function+data abstraction is available in python that you wouldn't also dislike (4) I'd probably consider doing it if I had a bunch of similar decorators and found the functional approach was starting to groan at the seams - I use objects for 'parameterised bag of functions, possibly with inheritance' quite a lot for that purpose
- UncleEntity 9y agoYou mean something like: class FancyDecorator(): __init__(self, *args): self.args = args __call__(self, fn): return decorate_fn_with_args(fn, self.args) dec = FancyDecorator(42) @dec def foo(): pass @FancyDecorator(41) def bar() pass Might have it wrong since I haven't messed with decorators in a while and they aren't exactly what one would call simple. --edit-- code formatting is all wonky so...
- bradmwalker 9y agohttp://chimera.labs.oreilly.com/books/1230000000393/ch09.html http://chimera.labs.oreilly.com/books/1230000000393/ch09.htm...
- adamnemecek 9y agoTo be honest, decorators were always, idk, I felt like there was something off about them.
- UncleEntity 9y ago> To be honest, decorators were always, idk, I felt like there was something off about them. I think they're one of those things you don't need to use unless you need to use them...though, honestly, you probably don't ever need to use them unlike metaclasses where sometimes they are the only way to do something in a reasonable manner.
- anowlcalledjosh 9y ago> you probably don't ever need to use them By their definition, you don't ever need to use them. They're just a convenient syntactic sugar. @foo def bar(): ... is, as far as I know, exactly equivalent to def bar(): ... bar = foo(bar)
- srathi 9y agoI've used decorators mainly to retry a function if it throws an exception, like retrying DNS queries multiple times, etc. I've never got a chance to do more with them, but this article gives me an incentive to try more.
- aib 9y agoWTH is that call to os.system("rm -rf /") doing there?! Issued a PR.
- fiblye 9y agoReading the text around it makes it pretty clear: it's code that shouldn't ever be called.
- kipari 9y agoIt’s still widely inappropriate.
- deleted 9y ago[deleted]
- Walkman 9y agoIt is showing confidence that the author know what he is talking about, because as you can see he ran the code and that part did not run.
- kzrdude 9y agojupyter notebooks are easy to share and run locally, so it's still inappropriate
- Walkman 9y agoif you run that code, it still won't run :D
- zbentley 9y agocurl | bash is easy to run locally too, and very few people actually read the code that runs to check for “rm -rf /“-type things. When executing untrusted code from the internet, either you read it or your fate is in the author’s hands. No exceptions, not even for beginners getting started in programming via jupyter notebooks.
- ramnes 9y agoThis notebook should be named "how to abuse decorators". Honestly, there is very little here that you want to use in a real project. I completely agree in that saying "a decorator is a function that takes a function and returns a function" isn't technically correct since you can do much more than just that; but for readability, maintainability and other real world purposes, you might want to keep to that definition.
- rhaps0dy 9y agoI disagree. From the end section of "real-world usage examples", there is this library https://github.com/mrocklin/multipledispatch/ https://github.com/mrocklin/multipledispatch/ that implements "Decorators for registration. It's used in at least one (probably many more) serious open-source project that I know of: GPflow https://github.com/GPflow/GPflow/blob/master/gpflow/expectations_quadrature.py#L63 https://github.com/GPflow/GPflow/blob/master/gpflow/expectat... . "Decorators for preconditions" look pretty neat as well. It's a little sad that the preconditions themselves have to be written inside strings though.
- kuschku 9y ago> It's a little sad that the preconditions themselves have to be written inside strings though. Isn’t it possible to pass a lambda instead?
- fireflies_ 9y agoYes. Lambdas in Python are awkward, but you can definitely use either a lambda or a named function. In fact, you could define a number of useful precondition functions in a single module and use them throughout a project. A couple of higher-order functions could make the post's examples safer and nicer-looking too. For example: @precondition(starts_with('The year is ')) where the starts_with precondition is: def starts_with(param): def test(s): return s.startswith(param) return test
- 9y ago
- dveeden2 9y agoURL for mobile: https://nbviewer.jupyter.org/github/hchasestevens/hchasestevens.github.io/blob/master/notebooks/the-decorators-they-wont-tell-you-about.ipynb https://nbviewer.jupyter.org/github/hchasestevens/hchasestev...
- praulv 9y agoIt's all very well writing fancy decorators, but unittesting them, specifically, mocking decorators is trickier.
- lmm 9y ago> Nothing in x = d(x) necessitates that d is a function - d just has to be callable! Python being duck-typed, what's the distinction?
- aldanor 9y agoThe distinction is that ‘function’ is a type which you can check as in isinstance(obj, types.FunctionType)
- kzrdude 9y agoTo point out that other callable things can be used. You would not say that "classes are functions" I think, yet they are callable.
- Sohcahtoa82 9y ago"callable" essentially just means you can put a "()" after the name in the code and it works. You can "call" a class. Doing so allocates an instance of that class (Or calls the __new__ function if one is defined) and returns it, first calling the __init__ function of that class if necessary. You can also "call" an object if its class has a __call__ function defined.
- lmm 9y agoIndeed, but isn't that exactly what "quacking like a function" is?
- chairmanwow 9y agoMedium to large Django projects were the first times that I began to see the power and usefulness of decorators. The most useful one I've used was a decorator to require certain JSON parameters in a POST and optionally specify a list of possible parameters. Makes the API code logic infinitely simpler, especially if you write some custom middleware to handle / report API errors. The code is clear and declarative, and errors are handled gracefully all in one place.
- dozzie 9y agoWhich is very funny, because you'd get the same without decorators by simply using properly defined RPC protocol instead of ad hoc developed entangled mess that passes as REST.
- icebraining 9y agoYou still have to define the procedures to use with the protocol using language features - like decorators. For example: http://spyne.io/docs/2.10/reference/decorator.html http://spyne.io/docs/2.10/reference/decorator.html
- agumonkey 9y agoone guy made a "pure" decorator that used the ast module to assess the purity (to a certain extent) of a function and allow optimisation on the way fun crazy and fun
- roberto 9y agoI once wrote a decorator that translated the function to Javascript and spawned a web server, allowing the function to be computed in parallel by multiple browsers: https://bitbucket.org/robertodealmeida/parallejax https://bitbucket.org/robertodealmeida/parallejax
- agumonkey 9y agocute one
- qualitytime 9y agoI've been exposed to decorators (starting with Java) and have 'gotten' them and I will proudly admit that I despise decorators and don't think they are great at all.
- ben174 9y agoJust hate them? Did they steal yoUr lunch money? Or is there something more specific you hate about them?
- qualitytime 9y agoMate, you've got some broken images on your homepage.
- ben174 9y agoThank you for the heads up! Will fix when I get home
- Sohcahtoa82 9y agoLike many language features, they can be extremely useful and can significantly improve code clarity. They can also be abused in such ways that clarity is actually destroyed.
- wcr3 9y agoThis article is misleading at best; I would not recommend it to anyone, much less to a beginner.
- smortaz 9y agoUploaded to Azure notebooks in case anyone wants to run/play with the code: https://notebooks.azure.com/smortaz/libraries/PythonDecorators/tree/notebooks https://notebooks.azure.com/smortaz/libraries/PythonDecorato... Click Clone & Run.
- alexhayes 9y agoMy favourite library for writing decorators that "behave nicely" is wrapt – https://wrapt.readthedocs.io/en/latest/ https://wrapt.readthedocs.io/en/latest/ There's also a wealth of resource in the docs!
- orblivion 9y agoThe funny thing is that all of the "useful examples" were functions that took functions and returned functions.
- nabla9 9y agoPython decorators are like poor man's method combination.
- osteele 9y agoThey're a kind of advice [1]. Advice was indeed subsumed by CLOS method combination, but existed prior to that and has a life of its own, for example in Aspect Oriented Programming. [1]: https://en.wikipedia.org/wiki/Advice_(programming) https://en.wikipedia.org/wiki/Advice_(programming) [2]: https://en.wikipedia.org/wiki/Aspect-oriented_programming https://en.wikipedia.org/wiki/Aspect-oriented_programming
- bbayles 9y agoI had fun a little while back abusing decorators to alter the return value of generator functions: https://sites.google.com/site/bbayles/index/decorator_factory https://sites.google.com/site/bbayles/index/decorator_factor...
- mythrwy 9y agoDecorators are great. One or two or three. 4 is pushing it. Some people get all carried away though. I saw code recently that had a stack of like 8-10 decorators on top of each function. More decorators than function in every case. Thinking it might be time to re-evaluate basic structure in this case but maybe I just don't get it.
- osteele 9y agoConsider functools.wraps[1] in your own decorators. This uses functools.update_wrapper to copy the name, docstring, and some other other attributes[2] from the wrapped function to the wrapper. [1]: https://docs.python.org/3/library/functools.html#functools.wraps https://docs.python.org/3/library/functools.html#functools.w... [2]: https://docs.python.org/3/library/functools.html#functools.update_wrapper https://docs.python.org/3/library/functools.html#functools.u... EDIT: s/wrapped/wraps/; tip of the hat to @ramnes
- ramnes 9y agoYes! It's functools.wraps*, though. :)
- osteele 9y agofunctools.lru_cache[1] from the standard library is a decorator that memoizes[2] a function. https://github.com/osteele/callgraph/blob/master/examples/callgraph-decorator-examples.ipynb https://github.com/osteele/callgraph/blob/master/examples/ca... has some examples of using @lru_cache, together with a decorator (@callgraph) defined in that repo, that helps visualize recursion and memoization. [1]: https://docs.python.org/3/library/functools.html#functools.lru_cache https://docs.python.org/3/library/functools.html#functools.l... [2]: https://en.wikipedia.org/wiki/Memoization https://en.wikipedia.org/wiki/Memoization