13 ms·
Functools – The Power of Higher-Order Functions in Python
- kumarvvr 5y agoI am delighted every time the Python Standard Library is mentioned. Always has a nice surprise. The last time I was surprised was the itertools library.
- lnyan 5y agoI feel the same way. Last week, I was surprised that `pow` in Python is actually a modpow: `pow(base, exp[, mod])`. Also check https://pymotw.com/3/ https://pymotw.com/3/. It's a great tour of the Python standard library
- hashmush 5y agoAnd since 3.8 the exponent can be negative, which lets you compute modular inverses without having to reimplement the extended Euclidean algorithm. >>> pow(5, -1, 13) 8 >>> 5 * 8 % 13 1
- jjice 5y agoHuh, it's the little things that are the neatest surpises. Having to write the EEA multiple times during my crypto class to do the homework was a real pain in the ass.
- magnio 5y agoIn case you haven't known this, check out data classes. It is so obvious yet so useful that I don't want to write Python lower than 3.7 anymore.
- nwomack 5y agoYes I agree, I also use typing.NamedTuple a lot
- nerdponx 5y agoWorthy of note: https://attrs.org https://attrs.org, which inspired dataclasses and has more features.
- leadingthenet 5y agoI’ll also plug one of my favourite python libraries: https://pydantic-docs.helpmanual.io/ https://pydantic-docs.helpmanual.io/
- gorgoiler 5y agoHeh, a younger me would definitely be walking around with a black T-shirt with: from functions import * …on the front and @cached_property def troll_face(self): return ‘:)’ …on the back. functools and itertools are amazing and I love them both. They are especially useful for teaching high school CS without having to stray from Python, which the kids at all levels know well and are comfortable with. However. Using @cached_property feels like a bad code smell and it’s a controversial design decision. The example given is more like an instance of a factory? Perhaps if the result of the render function was an Important Object™ (not just a mere string) then the function’s callers might not be so cavalier about discarding the generated instance. I would like to see the calling code that calls `render` more than once in two different places (hence the need for caching with no option for cache invalidation?). When every stack frame is a descendant of main() then there’s no such thing as “two different places”! def main(): page = HN().render() while ‘too true’: do_work(page) tea_break(page) Ironically it doesn’t feel very functional.
- wbradley 5y agoPlease do not promote the use of wildcard imports.
- matsemann 5y agoBecause of the GIL the cache is often pretty useless on a webserver anyway, as each and every request gets a different process not sharing that cache. So then one has to deploy Redis in addition and complicate the stack. Makes sense with a shared cache when there's multiple instances, but often it's overkill. Having to go "external" to fetch the data I might as well not cache anything.
- zmmmmm 5y agoThe worst part of python is the lack of utility functions for munging collections. But it sits at a slightly higher level than this - things that are idiomatic in other scripting languages like groupBy(fn), sortBy(fn), countBy(fn), collate, are all inexplicably like IKEA self assembled furniture in Python instead of first class entities. It makes lots of routine data munging and algorithmic work much more annoying than it needs to be (comparing to Groovy, Ruby, etc).
- BurningFrog 5y agoIt will take Python 10 more years to allow a flatten()...
- staticautomatic 5y agomore_itertools ftw
- masklinn 5y ago10 years is very optimistic: flatten has little use without a functional pipeline (something like clojure’s threading macro) as in other contexts it’s a trivial comprehension.
- joshuamorton 5y agoDo you need the recursive form? Because a single flatten is itertools.chain.from_iterable
- nerdponx 5y agoI have written this in serious code before, and I don't regret it: from itertools import chain flatten = chain.from_iterable def lflatten(x): return list(flatten(x)) def sflatten(x): return set(flatten(x))
- BurningFrog 5y agoI know it exists, but the universally known name is 80% of the utility of having a standard flatten() function. I could tolerate a slightly different name, but `itertools.chain.from_iterable` is absurd to me.
- pyuser583 5y agoDon’t use lru_cache. It stores pointers to the values, so if a mutable value is changed downstream it gets changed in the cache. That’s not proper caching.
- SV_BubbleTime 5y agoThis seems like it would have been an easy fix on their end. Was the issue memory and storage?
- jakear 5y agoIt’d be annoying for every type used in a cache to need to properly implement __deepcopy__(), and it’d pose significant performance impact, especially if the cached objects are large (which there’s a good chance they are, given you’ve felt the need to cache them rather than build them from scratch). Much better off using the same assignment semantics used throughout Python and let people choose to deepcopy() all the objects they read from the cache if they really really want to modify them later; it would even work as a simple decorator to stack onto the existing one for such cases. Aside: If we’re missing “deep coping” anything by default in Python, it’s definitely default parameters :) deep copying caches makes much less sense than that.
- nerdponx 5y agoFor anyone wondering what exactly the assignment semantics are, see: https://nedbatchelder.com/text/names1.html https://nedbatchelder.com/text/names1.html and https://www.youtube.com/watch?v=_AEJHKGk9ns https://www.youtube.com/watch?v=_AEJHKGk9ns
- tialaramex 5y ago> An important fact about assignment: assignment never copies data. Is that really what's going on here? I'm in way too deep to be sure what's best for beginner programmers, but I feel like Python must surely optimise... sheep = 1 goats = sheep sheep = sheep + 10 ... by simply copying that 1 into goats, rather than tracking that goats is for now an alias to the same value as sheep and then updating that information when sheep changes on the next line. Now, if we imagine those numbers are way bigger (say 200 digits), Python still just works, whereas the low level languages I spend most time with will object because 200 digit integers don't fit in a machine word. You could imagine that copying isn't cheaper in that case, but I don't think I buy it. The 200 digit integer is a relatively small data structure, still probably cheaper to copy it than mess about with small objects which need garbage collecting.
- frogcoder 5y agoSite is down. I believe this is the same article by the author. https://towardsdatascience.com/functools-the-power-of-higher-order-functions-in-python-8e6e61c6e4e4 https://towardsdatascience.com/functools-the-power-of-higher...
- frogcoder 5y agoI strongly think Python should have more functional programming support. Functional Programming Languages are usually scary to look at for many programmers. Python would not only be a good FP introduction to them, it would also benefit greatly. Years ago I found out the Guido wouldn't let tail recursion included, and even tried to remove map function from built in functions. Therefore I got the impression that python wouldn't have further support in FP. I really wish that is not the case. With the coming pattern matching in 3.10, My hope is high again. I have very high respect for Guido van Rossum, I'm not here to discredit him. He's one of the authors in PEP 634. I wish python would have simpler syntax for lambda which is currently using the lambda expression, even JS is doing better on this. A built-in syntax for partial would also be great. It could be even better if we can have function composition. Some problems are better solved the FP way. It could even make the program more readable which is one of the strengths of Python.
- mjburgess 5y agoThey added "pattern matching" as a statement. The irrational hatred of FP is still there, to the point of the absurdity of implementing a hobbled procedural version of pattern matching! It is bordering on insane.
- toxik 5y agoPlease suggest a pattern matching expression syntax that doesn’t require completely changing how the language works. Pattern matching took so long because it was extremely hard to find that compromise, and I actually think they did an okay job. No, it won’t cover every case, but it will also cover the important ones.
- uryga 5y agox = match y: case 0: "zero" case 1: "one" case x: y = f(x) y+y print(x) it can require semicolons or parentheses or even braces somewhere if that's necessary to make the grammar work, idc. just don't make me type out `x = ...` a hundred times... (yes, i know you could use the walrus operator in the third case) (also, if anyone wants to reply with "use a named function because READABILITY", please save it)
- sysctl21 5y agoLink doesn't have the content. #PlsFix
- andreareina 5y agoCross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://martinheinz.dev:8080/api/v1/posts/52 https://martinheinz.dev:8080/api/v1/posts/52. (Reason: CORS request did not succeed).
- accidental_spec 5y agosomething wrong with the page
- pytlicek 5y agoSite is up again :)
- accidental_spec 5y agopage is broken
- pytlicek 5y agoSite is up again :)
- pytlicek 5y agoNicely written article. I didn't even know that these things could be done as described in the article. Well-crafted topic. Worth reading :)
- de_keyboard 5y agoFunctional programming in Python sucks for a few reasons: - Lack of pipe operator - Multiple arguments everywhere instead of currying - No do-notation or equivalent - Reference equality rather than structural equality for objects etc. If you want to program in this style, consider using F#, R or even JavaScript with a few Babel plugins.
- henearkr 5y agoWhich Babel plugins you would recommend?
- de_keyboard 5y agoBabel pipeline is a good one: https://babeljs.io/docs/en/babel-plugin-proposal-pipeline-operator https://babeljs.io/docs/en/babel-plugin-proposal-pipeline-op... Combine it with Lodash FP: https://github.com/lodash/lodash/wiki/FP-Guide https://github.com/lodash/lodash/wiki/FP-Guide
- henearkr 5y agoThanks a lot!!
- eddyg 5y agoFigured it’s worth mentioning “funcy”: https://funcy.readthedocs.io/en/stable/overview.html https://funcy.readthedocs.io/en/stable/overview.html
- ionforce 5y agoAfter having done Scala for a few years, it's finally a pleasure to be able to read and understand a thread like this. FP gains are real. Anyone who tells you otherwise doesn't know what FP is. To anyone out there who isn't on the FP train yet, get on. You'll become a better programmer for it.
- deleted 5y ago[deleted]
- Zababa 5y agoThat's one of the thing I love about JS. Sure, you don't have all the good static typing that comes with Scala or OCaml, however map, filter, reduce and lambdas are easily accessible. We may get record, tuples, pattern matching and a pipeline operator at some point!
- rthz 5y agoFor people wanting to do more functional programming in Python, have a look at toolz (https://toolz.readthedocs.io/en/latest/ https://toolz.readthedocs.io/en/latest/). It goes a bit further in that direction.
- loa_in_ 5y agoNote to author: I am unable to parse this sentence: "It's a simple wrapper on top of the lru_cache which omits the max_size argument making it smaller and after as it doesn't need to evict old values."
- ehsankia 5y agos/after/faster
- pantsforbirds 5y agoThe only thing I really think is missing from python is good pattern matching (curious to try out the new implementation), and a pipe operator. Chaining functions is somewhat annoying in python if you want to do any sort of cross library work (which is super common in the data side of the language). Pandas has nice function chaining, but if you want chain data through cleaning functions and maybe do some nlp work you don't end up with a very readable or nice syntax IMO.
- ncc-erik 5y agoBig shoutout to lru_cache. I tossed in two lines of code and was able to get a 60x speedup in my code by reducing the amount of regular expression compilations I had to do.
- jkbbwr 5y agoPython sucks for functional programming. Mutability, references and hidden objects everywhere. If you want a real dynamic language functional programming experience, you would be far better with Elixir.
- sidmitra 5y agoNot really for use with python, but still worth a look since it brings some interesting syntax along. http://coconut-lang.org/ http://coconut-lang.org/ > Coconut is a functional programming language that compiles to Python. Since all valid Python is valid Coconut, using Coconut will only extend and enhance what you're already capable of in Python. > Why use Coconut? Coconut is built to be useful. Coconut enhances the repertoire of Python programmers to include the tools of modern functional programming, in such a way that those tools are easy to use and immensely powerful; that is, Coconut does to functional programming what Python did to imperative programming. And Coconut code runs the same on any Python version, making the Python 2/3 split a thing of the past.
- Semiapies 5y agoThe sum reduce example really should use 0 instead of 1 as the initializer. 2, 6, 8, and 3 don't actually add up to 20.