10 ms·
Functional Python Programming
- FrustratedMonky 3y agoHow new is this? Seems like there is more functional tools in Python than I remember. Didn't see the new 'match' statement in this doc.
- rirze 3y agoIs the match-case statement a functional concept? I see it in functional languages but I'm not sure it is inherently functional...
- agumonkey 3y agofunctional I can't say, but match was highly related to structural typing which was very much central in FP/LP culture
- FrustratedMonky 3y agoThink this is answer. It isn't technically needed to be called 'functional', but it is so integral to type systems that it seems to be used in all functional languages. Maybe we can call 'match' statements an emergent phenomena of functional languages.
- agumonkey 3y agoand i'm sure the idea floated around in non FP languages too, but 1) mutable programming shifts the idioms very very far away from passive destructuring, they like to deref pointers and reuse memory cells 2) it's also linked to parametric type systems I believe, which was missing until cpp/java5 in the mainstream (while FP had this since milner which was in the 70s)
- vintagedave 3y agoThanks for posting this. Many years ago (2006?) I was a just-out-of-university programmer and a friend of a friend who wrote Python for Canonical sat me down and tried to teach me functional programming using Python. I was a poor learner: I didn't 'get it' (functional programming in general) and, to my regret, learned nothing. Much time has passed and I hope I'm a more aware and open-to-new-things person today :) I use Python these days though in a traditional procedural / OO way. This post reminded me that maybe I should look into writing it functional-style. Thankyou!
- FrustratedMonky 3y agoI find that learning functional style in a language that does not enforce it, is extra difficult. If the language does not provide guard rails, then new learners will fall back on non-functional methods, and not even realize it.
- OkayPhysicist 3y agoI very much support this view. If you want to teach someone functional programming, you go straight to Haskell or a Lisp. They don't need to master the language or anything, but the cold shower of "There are fundamentally different paradigms in programming than those I am used to" combined with a language that will allow nothing but functional code tends to work a lot better than "Hey guys! Let's torture Java to support map/filter/reduce!"
- qbasic_forever 3y agoIMHO broaden your boundaries a bit and learn a lisp or clojure-based functional style, like grab a getting started in clojure book and work through it. You will learn a lot more than this guide goes into, it's really just looking at functions in a functional style but doesn't spend nearly enough on data structures and state which are really the core of functional programming (and what python doesn't do well out of the box for functional programming). Learning a bit of clojure will really open your eyes to what functional programming means, and you can take some of those learnings back to python.
- amne 3y agocan you make anonymous functions with more than 1 expression now?
- PurpleRamen 3y agoJust curious, but from a pragmatic view, what is the advantage of using anonymous functions instead of named functions for any mildly complex code?
- raphaelj 3y agoIt's pretty common to define "higher-order" functions in functional programming. These will sometimes be applied to code blocks. For example, Python's with statement could have been entirely programmed as a library function if lambas supported code blocks: with(open("file.txt"), lambda f: content = f.read() return sum(int(c) for c in content.split(" ")) ) Functional languages usually have a pretty limited "core" and implement most of their features through their standard library, usually using higher-order functions, overloaded operators ...
- PurpleRamen 3y agoBut I can do all that with python already? Your example would work this way too: def _(f): content = f.read() return sum(int(c) for c in content.split(" ")) with(open("file.txt"), _)
- sodapopcan 3y agoI don't think they were saying you can't do it in Python, just offering what is arguably a nicer syntax, but of course that is subjective. The anonymous function version lets you write things in the order they happen and also "binds" the operations together, ie, someone can't accidentally add a completely unrelated line between the `def` and the `with`. It ultimately doesn't matter, people just get used to the conventions of their preferred languages and we all trudge on.
- 3y ago
- raphaelj 3y agoSadly, Python is a pretty poor functional language. The core of functional programming is about avoiding mutable states, not much about anonymous functions or passing functions as data. To do proper functional programming in Python, there should be IMO: - a way to enforce non-mutable variables/objects; - non-mutable collections; - proper support for recursion and tail-recursion optimization; - a better syntax for anonymous functions than single statement lambdas. Generators are nice, but do not have much to do with functional programming.
- qbasic_forever 3y agoI agree but it's not so dire, the tuple type (and variants like namedtuple) is pretty powerful, non-mutable and often my first choice for data structure in python. Everything else can be built on top of it if you're really motivated.
- amanj41 3y agoI would add frozenset as the other immutable collection. The other points are well taken though
- linkdd 3y agoFor tail recursion, you can use this snippet of code: class Recurse(Exception): def __init__(self, *args, **kwargs): self.args = args self.kwargs = kwargs class Terminate(Exception): def __init__(self, retval): self.retval = retval def tailrec(func): def wrapper(*args, **kwargs): while True: try: func(*args, **kwargs) except Recurse as r: args = r.args kwargs = r.kwargs except Terminate as t: return t.retval return wrapper @tailrec def fact(n, acc=1): if n == 0: raise Terminate(acc) else: raise Recurse(n - 1, acc * n) Of course, it will be slow because it relies on exceptions :P
- _aavaa_ 3y agoYou can do a similar thing (that's around 2-3x faster) with a trampoline: def factorial(n): res = fac(n) while callable(res): res = res() return res def fac(n, acc=1): if n == 1: return acc else: return lambda: fac(n-1, n*acc)
- stayfrosty420 3y ago@dataclass is the new final
- _dain_ 3y agoDoes the type system let you express that a class shouldn't be subclassed? I remember this possibility was mentioned in a PEP and got deferred. It would be really useful with the new match/case pattern matching feature, because then you could have proper sum types, and mypy could enforce exhaustiveness. AFAIK you have to do a workaround with a "assert False" or similar at the end.
- nerdponx 3y agoYes, that's what @typing.final is: https://docs.python.org/3/library/typing.html#typing.final https://docs.python.org/3/library/typing.html#typing.final But that's only checked by the type checker. At runtime you still need to do something like this to prevent subclassing: class DontSubclassMe: def __init_subclass__(self): raise TypeError("Don't subclass me!")
- hfhdjdks 3y agoIt's not what you were asking for (class that can't be subclassed), but `typing` has an `assert_never` to check exhaustiveness: https://typing.readthedocs.io/en/latest/source/unreachable.html https://typing.readthedocs.io/en/latest/source/unreachable.h...
- nerdponx 3y agoThanks for this trick for exhaustiveness checking, I've just been putting assert False, "This is unreachable." in my code until now.
- Mizza 3y agoI thought I knew functional programming from knowing Python, but looking back I didn't really "grok" it until I moved to Elixir. Now I like it and prefer it. I don't run into any of the types of bugs I used to create in Python, mostly from being lazy and fiddling with data inside of a loop. I miss do miss early returns though.
- macintux 3y agoI moved from Erlang to Python and was miserable. I still write most of my code in a functional style, but I dearly miss using a language that enforced those constraints.
- sammy_rulez 3y agoFor those who are interest I maintain a small open source library to use type safe monads in python https://github.com/sammyrulez/typed-monads https://github.com/sammyrulez/typed-monads
- tandav 3y agoshameless plug: I maintain a small library to do functional pipes. You can write: ( range(10) | Map(lambda x: x * 10) | Filter(lambda x: x % 2 == 0) | Reduce(lambda a, b: a + b) ) instead of: x = range(10) x = map(lambda x: x * 10, x) x = filter(lambda x: x % 2 == 0, x) x = reduce(lambda a, b: a + b, x) and more. https://tandav.github.io/pipe21/ https://tandav.github.io/pipe21/
- sampo 3y agoimport pandas as pd import functools ( pd.Series(range(10)) .apply(lambda x: x * 10) .where(lambda x: x % 2 == 0) .pipe(lambda s: functools.reduce(lambda x, y : x + y, s)) )
- melx 3y agoReadability isn't the best. Also what you present here is method chaining and not functional pipes.
- paolosimone 3y agoI tried to preach functional programming style to my data analysts colleagues once... it ended poorly, but it was fun! If you want to smile, here is the presentation I made (warn: old memes ahead) https://paolosimone.github.io/python-is-functional https://paolosimone.github.io/python-is-functional
- haspok 3y agoWhat blows my mind is that Python was designed for maths and data science by a maths and data science guy. It should be natural to work with immutable values and pipelines in Python, which are natural functional constructs. Yet everything in Python is mutable and pipeline-style programming is not really supported. ??? (A long time ago I tried to teach programming fundamentals to a maths person. She was completely puzzled by x=x+1. X equals to X+1? That was nonsense in their eyes.)
- tgv 3y ago> Python was designed for maths and data science I don't think so. > During Van Rossum's stay at CNRI, he launched the Computer Programming for Everybody (CP4E) initiative, intending to make programming more accessible to more people, with a basic "literacy" in programming languages, similar to the basic English literacy and mathematics skills required by most employers. Python served a central role in this
- constantcrying 3y agoMathematics is an extremely broad field, the mathematics of a numerical analyst and someone doing algebraic geometry are very different and a programming language for each would look quite different. In python the C and even C++ heritage is quite clear, but it is in no way designed like Miranda and Haskell, which actually target to some extent expressing particular mathematical constructs in code, something which neither C, C++ not python ever aspired to.
- mikebenfield 3y ago> What blows my mind is that Python was designed for maths and data science by a maths and data science guy. Neither of these is true. I actually still find it a little weird that Python became so widely adopted by data science people, and that definitely doesn’t seem to be the main user GvR originally had in mind when creating the language.
- tgv 3y agoThat isn't weird. It's what they've been taught. CS departments started teaching Python to other departments because of its ease, and it stuck.
- lindbergh 3y agoIn their definition of programming languages types, I'm not sure I see how SQL which they cateogrize as declarative really differs from FP, especially with lazy evaluation. Both are a succession of functions applied onto a previously declared variable : just think of a long SQL query with many ctes modifying the previous ones; each of them can be thought of as a variable which takes its value from the application of a function on other variable (a bit like the let function in ocaml). I always thought of LaTeX as being the prime example of declarative programming.
- peterkelly 3y agoSQL does not support higher-order functions or lambdas (except possibly in some special dialect I'm not aware of)
- lindbergh 3y agoRight, good point. I just find the structure and the mindset to be similar.
- gnufx 3y agoIterators are a fundamentally non-functional construct. Long ago Henry Baker wrote "Signs of Weakness in Object-Oriented Languages": https://plover.com/~mjd/misc/hbaker-archive/Iterator.html https://plover.com/~mjd/misc/hbaker-archive/Iterator.html
- heyoni 3y agoYou just broke my brain. I will read it though, just give me 6 months!
- Jeff_Brown 3y agoWhat's so useful about iterators and generators. The article says how to use them but not why. If you already know how to make list comprehensions and use "for elt in coll", do they let you do anything new?
- xapata 3y agoFundamentally? No, it's all just machine language in the end. Practically? Yes, generators are often more memory-efficient, and thereby often more compute-efficient.
- Jeff_Brown 3y agoIs it only about efficiency, and not expressivity? If so, can the compiler figure out where to transform my equivalent structures into iterators and generators?
- xapata 3y agoThe expressivity gain depends on what you're left with without the feature. It was hard for me to imagine where the previous comment draws the line around iterators and generators. An optimizing compiler would need to know what actions are pure in order to rewrite code to be lazy. Python allows so much dynamism, I can only see that happening with a very sophisticated tracing JIT.
- crabbone 3y agoYou could probably find more uses, but here are some: Generators allow one to easily transform code from non-buffered to buffered. Consider this example: for y in x: do(z) Now, x may be a collection that was eagerly evaluated in previous steps, let's say a list, but then you've discovered that this list is too big to fit in memory, and you want to generate it in manageable fixed-size chunks, so you replace the code that created x to make x a generator. You don't have to touch the code above -- it will work the same way because generators have the same interface as collections. Another use: generators are used to implement async / await. I personally find this idea ridiculously stupid, but a lot of people (and especially those who don't understand what it does) like it a lot. Yielding mechanism, which is a feature of generators, is the one that's used to communicate / switch between co-routines (tasks) of asyncio. Another aspect, besides bufferization is that you might want to delegate control over how much looping you want to do to a separate chunk of code. I.e. you may want to separate the generation of elements (hence, generator) from eg. filtering them, or transforming them in some way, or reducing them etc. If you didn't have this ability, you'd have to generate the entire collection upfront, and if your computation takes multiple steps that you'd like to separate into different code chunks, you'd have to also generate intermediate collections, even though, potentially, you don't need some of the elements in those collections. Consider, for example: def powers(start, end, power): return (x ** power for x in range(start, end)) def flt(a, b, c): return a + b == c def disprove_flt(upto, upto_power): for power in range(upto_power): for a, b, c in zip(powers(1, upto - 2, power), powers(2, upto - 1, power), powers(3, upto, power)): if flt(a, b, c): return a, b, c return None, None, None Which is, of course not a correct way to search for the counterexample to Fermat's last theorem, but I tried to find a popular enough subject so that the example was easier to follow. An exercise to the reader: rewrite the code above in such a way as to eliminate "upto" and "upto_power", i.e. to search until a counterexample is found (or indefinitely).
- xwowsersx 3y agoFun to work on doing FP in languages that don't really support it, but in my view a language has to be built for FP for it to be a practical option in any real applications. Several obvious reasons for Python being a poor lang in which to do FP: - mutable data structures - no built-in function composition - limited support for HOF - no tail call optimization (AFAIK) - performance in general isn't great and I imagine it's even worse when relying heavily on recursion, etc Still, a fun project!
- jgilias 3y agoIt doesn’t have to be all or nothing though. I’ve definitely seen (and written) functional Python in ‘real applications’ sprinkled here and there in an otherwise normal codebase. In fact, I’d wager that any Python programmer worth their salt would suggest using a list comprehension and a lambda over a loop in most data transformation situations. Also currying comes in handy often once you have it in your toolkit. It’s a bit silly to call two very mature libraries that are part of the Python standard library a ‘fun project’ just because the language itself supports multiple programming paradigms.
- xwowsersx 3y agoNo, it doesn't need to be all or nothing, but the scenario I was directing my point at was where people try to "do FP in lang X", not just sprinkling in functional concepts here are there. At this point, almost every language has some functional programming concepts in it, but I wasn't addressing that. I could've been clearer, but I've seen many cases where people would rather be programming in Haskell or some other functional language and then when "forced" to use some other language, like Python, they try to essentially recreate Haskell in that language and my point is that this really doesn't work or, at the very least, it's more trouble than it's worth and the resulting codebase is a complete mess because you've tried your hardest to do FP in a language that ultimately isn't suited for it. You are absolutely right though that it needs not be all or nothing. To the extent you can rely on some of these constructs and enable equational reasoning by avoiding mutation, etc. you'll be better off, no doubt (I do this too in Python and other languages because wherever I can make it easier to reason about my code, I will do so). I'm just cautioning against trying to do something akin to pure FP in Python.
- harveywi 3y agoShameless plug for Scala programmers who are marooned in Python land and grasping for something to help dig them out: https://github.com/miiohio/ziopy https://github.com/miiohio/ziopy
- crabbone 3y agoWorthless functionality in a worthless language. Only topped by the bizarre and unverified claims in its documentation: Most programming languages are procedural: Did the author count? -- The answer is a clear and resounding "No". The author pulled this factoid out if his rear end. Maybe. Maybe not. It's not even clear by what's meant by "all languages" -- all possible languages? all languages known to author? all languages used in Github? And why should anyone care about this kind of multitude? Lisp Again, people who've never seen any Lisp, or vaguely remember their college days when some course requested from them to write a function to figure out if a string is a palindrome in Scheme think that Lisp is a single language. The author just decided to demonstrate his blistering ignorance by including something that he thought would render him as more experienced than their readers. Later the author perpetuates all sorts of absurd myths about "functional programming" s.a. increased modularity or ease of debugging. Apparently, author had never used step-debugger nor had he wrote anything in any popular programming language that advertises itself as functional to experience first-hand this "ease" he's talking about. Needless to say that nothing in functional programming prevents programmers from writing long functions... In practice, however, some languages which advertise themselves as "functional" have pathologically bad / hard to read syntax (eg. Haskell), and functions longer than some 10 lines or so become too difficult to understand even to people who believe themselves to be proficient in those languages. Author is simply lying when he claims that generators are a kind of function, which can be simply verified: >>> def generate_ints(n): ... for i in range(n): ... yield i ... >>> type(generate_ints(1)) <class 'generator'> >>> type(generate_ints(1)).mro() [<class 'generator'>, <class 'object'>] >>> isinstance(generate_ints(1), type(generate_ints)) False Generators and functions are unrelated. Neither is a kind of other. On top of that, author frequently violates Python's coding conventions (eg. capital letters in variable names, assigning lambdas to variables). --- But, bottom line: crappy language deserves no better documentation than this.
- eska 3y agoHave a beer, bro