7 ms·
Functional programming with Python
- nubela 14y agobeautiful html slides, anyone knows what framework this is?
- oliveoil 14y agohttps://github.com/hakimel/reveal.js/ https://github.com/hakimel/reveal.js/
- beambot 14y agoForgot one big con: Python lambda functions aren't "proper" functions (ie. arbitrary multi-line code blocks). Guido van Rossum has addressed this many times, saying (in effect) "it adds undue complexity and would be un-Pythonic", eg: http://www.artima.com/weblogs/viewpost.jsp?thread=147358 http://www.artima.com/weblogs/viewpost.jsp?thread=147358 This is one of my (very few!) gripes with FP in Python.
- jewbacca 14y agoI don't know if this is a problem if you're going hardcore with your FP -- one line is one expression, and multiple expressions are probably going to be stateful, which should probably be factored out into something with a name. In Haskell, the only places (that occur to me right now) where you'll have multiple lines are: * Pattern matching function arguments * Giving things names with `let` or `where` * In a monadic `do` block, which is just sugar that reduces to single-expression chain of lambdas And all of these are language design decisions that require more than just multi-statement lambdas.
- samdk 14y ago...or anywhere you're using a 'let' statement. I write a lot of OCaml, and I write a lot of multi-line anonymous functions that aren't stateful. I think it's less common to do this in Haskell, but that's more because it's Haskell than because there aren't any side effects. Random other tidbit that doesn't deserve its own comment: In Python, instance.method is the same as doing ClassName.method(instance). This can be helpful when using map/filter, because you get to do map(ClassName.method,lst) instead of map(lambda x: x.method,lst).
- koenigdavidmj 14y agoOr you can do operator.methodgetter("method") and not need to know the class name.
- ville 14y agooperator.methodcaller('method') #ftfy
- jewbacca 14y agoNOTE: Ooops, it seems the 'edit' window expired on my comment. Expanding, unfortunately, in a separate comment: ---- I don't know if this is a problem if you're going hardcore with your FP. When we're talking about "lines" in Python, one line is exactly one statement, and if you're doing FP, that statement is an expression. Multiple statements are going to be either stateful, or declaring a temporary name for an intermediate value. I am probably not being sufficiently imaginative right now, but I can't think of any fully-fledged functional pattern which would require multiple statements in a lambda, which would not be better abstracted out and given a name, and I would genuinely like to hear some examples of cases where a multi-statement lambda make for "better" code (or to hear how my presentation of this problem might be flawed, because I haven't done any serious Python programming recently enough to be very close to the issue). ---- In Haskell, a language where the syntax has been exhaustively designed to facilitate FP, the only places where you'll see multiple lines (I'm pretty sure this is exhaustive, without using GHC extensions, eg. Arrow `proc` blocks) are: * Pattern matching function arguments * Giving things names with `let` or `where` * In a monadic `do` block -- which is just sugar that reduces to single-expression chain of lambdas, and are not equivalent to multi-statement lambdas
- vsbuffalo 14y agoAnd you can't pass them as simple functions to do stuff via the multiprocessing module because they can't be pickled. I don't understand this infatuation with FP in Python. To me the essence of functional programming is immutable state. Python has mutable state to the core. List comprehensions are novel syntax for many FP-esque operations (so much so that map, reduce, & filter were almost removed from Python at some point) yet, to their core they're full of mutable-state disgustingness. For example: >>> a = range(12) >>> [a.pop(i) for i in range(3)] [0, 2, 4] >>> a [1, 3, 5, 6, 7, 8, 9, 10, 11] As long as list methods have methods that change state of the object, even when processing said object, Python handles FP at a very superficial syntactic level.
- icebraining 14y agoYou could always use tuples; they're essentially immutable lists. >>> a = tuple(3*x for x in range(5)) >>> a[2] 6 >>> map(lambda x: str(x), a) #can be iterated ['0', '3', '6', '9', '12'] >>> a[2] = 9 TypeError: 'tuple' object does not support item assignment >>> a.pop() AttributeError: 'tuple' object has no attribute 'pop'
- vsbuffalo 14y agoYea, but still gross. map() isn't even an endomorphism.
- omaranto 14y agoI'm not sure what you mean. map is not supposed to be an endomorphism, rather it takes a function f and returns a list homomorphism.
- haldean 14y agoI think it'd be nice if it were an endomorphism, though; Making map(f, tuple) -> tuple, map(f, list) -> list, etc. would save users from having to write tuple(map(f, tuple)) (which is something I've done a bunch).
- rat87 14y agoI fail to see how this makes them non-proper functions, a better way to put it is they are a subset of possible functions.
- beambot 14y agoYes, single-expression functions are a subset of all possible functions... but that's not exactly what I was getting at. I put "proper" in quotes just to signify that they have subtle differences that, from a purely type-based abstraction, are different from named functions. For example, you can't serialize (pickle) a lambda function.
- aschwo 14y agoI'm curious, on the Currying (Standard Library) slide, the author uses `attrgetter` from the `operator` module -- I've always used `getattr` to achieve the same thing. Am I missing something by not using `attrgetter`?
- walrus 14y agoShort answer: not really. Long answer: In [1]: from operator import attrgetter In [2]: class Person: ...: def __init__(self, name): ...: self.name = name ...: In [3]: me = Person('walrus') In [4]: getattr(me, 'name') Out[4]: 'walrus' In [5]: attrgetter('name')(me) Out[5]: 'walrus' This is potentially useful if you're using `map`: In [6]: people = [me, Person('aschwo')] In [7]: list(map(lambda obj: getattr(obj, 'name'), people)) Out[7]: ['walrus', 'aschwo'] In [8]: list(map(attrgetter('name'), people)) Out[8]: ['walrus', 'aschwo'] Really, though, the functional programming idioms don't meld too well with Python. I think this is a lot nicer: In [9]: [person.name for person in people] Out[9]: ['walrus', 'aschwo']
- andreasvc 14y agoOn the last point, a list comprehension is just a much a functional programming idiom--Python got them from Haskell! I use both in my code, whichever is shortest. PS: map returns a list, no need to call list() on it.
- walrus 14y agoIn Python 3, map returns a iterator, not a list: In [1]: sq = lambda x: x * x In [2]: map(sq, [1,2,3,4]) Out[2]: <builtins.map at 0x2135050> In [3]: list(_) Out[3]: [1, 4, 9, 16]
- apendleton 14y agoattrgetter takes an attribute name and returns a function that, when passed an object, returns that attribute from that object. So it's a curried getattr. These are equivalent: f = attrgetter("some_property") f = lambda obj: getattr(obj, "some_property")
- dustingetz 14y agoi started learning functional programming with python, it falls apart at ~300 loc which is absurdly low, which makes python basically useless for production FP. python's data structures are mutable, and mutable data structures necessarily expose a different interface than immutable data structures. using mutable datastructures in immutable style necessarily has severe performance penalties; and to do meaningful stuff you need to use a statement, and you can't put a statement in a python lambda. And Guido thinks lambda's are un-pythonic and a misfeature which he seems to regret [1]. python is a great sandbox for imperative programmers interested in FP, and these are great slides. it's awesome to play with these concepts in a familiar language - thats how i started - but its important to note that FP is just not pythonic, and thus it's probably a mistake to put functional-style python into production. [1] http://news.ycombinator.com/item?id=3962316 http://news.ycombinator.com/item?id=3962316 if you are interested in functional python, i wrote about it a bit this year in my blog, because despite all this, python is a nice language for evangelism aimed at imperative developers. [2] implement generators without yield http://www.dustingetz.com/2012/10/03/implement-python-generators-without-yield.html http://www.dustingetz.com/2012/10/03/implement-python-genera... [3] easy monads http://www.dustingetz.com/2012/10/07/monads-in-python-identity-maybe-error.html http://www.dustingetz.com/2012/10/07/monads-in-python-identi... [4] hard monads http://www.dustingetz.com/2012/10/02/reader-writer-state-monad-in-python.html http://www.dustingetz.com/2012/10/02/reader-writer-state-mon... [5] peter norvig's Lispy refactored using monads https://github.com/dustingetz/monadic-interpreter https://github.com/dustingetz/monadic-interpreter
- Gakusei 14y agoGreat job (some ruby code comparisons would have been welcomed, though). A little mistake on page 48: the Clojure code will fail. Use (map #(* % 2) '(1 2 3)) or (map (partial * 2) '(1 2 3)) instead of: (map #(%*2) '(1 2 3))
- ericmoritz 14y agoGreat slides. I've been toying casually with a FP project for Python. It's an attempt to bring common FP abstractions and lazy evaluation of sequences using generators. It's a work in progress, and I often think to myself, "Is this a good idea". Here's the latest iteration: I'm in the middle of documenting things for an alpha release: https://github.com/ericmoritz/fp/blob/feature/documentation/fp/__init__.py https://github.com/ericmoritz/fp/blob/feature/documentation/...
- haldean 14y agoNote that `reduce` has been, if not deprecated, then at least discouraged (not that I agree with this, but reduce gets taken away in Python 3, so it's probably good form to stop using it). The author does `reduce(operators.add, lst)` a lot; you can do that using `sum(lst)` instead.
- jewbacca 14y agoI was writing a really long response to your post, explaining how reduce/inject/fold is abstracting out an absurdly common iteration pattern, and using it can expose structural similarities between many superficially different computations, and how (unlike map and filter) it can't be expressed as a list comprehension, so removing it was super lame. But in the process of translating a comment I wrote in another forum from Ruby to Python, it became apparent that, as long as it's pythonic to implement `__add__` for any monoidal type, the resulting polymorphic `sum`, together with list comprehensions, will cover pretty much everything[0]. So I'm actually coming around to Guido's side here. ---- [0] Assuming there's a consistent __mul__/product for alternative monoid instances, and with `all` and `any` to handle numerically-cast booleans.
- tmhedberg 14y agoIf __add__ is taken to be any associative operation of a monoid, then Python's sum is merely equivalent to Haskell's mconcat, which is not nearly as general as reduce/foldl. Plus, unless you implement something akin to newtype wrappers for alternative monoid instances (hardly Pythonic), you're going to be restricted to a single foldable function for any given type. I think you were right the first time; removing reduce from Python is very lame.
- benhoyt 14y agoIt seems to me if you want to write code like this, you shouldn't write it in Python. Python has much simpler, built-in ways to do most of the things he's suggesting. For instance, he says "good" for this code: reduce(operator.add, map(len, ss)) No! The simpler (and more efficient, as it doesn't have to build a list of results!) way to do that is using the built-in sum() with a generator expression: sum(len(s) for s in ss) The rest of the slides are full of similarly-complex idioms. Another one: def square_sum(a, b): return sum(map(lambda x: x**2, range(a, b+1))) Please, no! Generator expressions are there for a reason. Much simpler and much more efficient would be: def square_sum(a, b): return sum(x**2 for x in range(a, b+1)) In fact, most places you see map() in Python can be rewritten as generator expressions. For stuff like functools.partial, in simple cases, isn't it clearer to just write this? def debug(message): log('debug', message) I like his use of operator.itemgetter/operator.methodcaller, though. They're useful for sort keys and stuff, and faster and often clearer than lambdas. Also: namedtuple is great instead of "using classes as attribute containers".
- babarock 14y agoI may be going out on a limb here, but I don't think he's actually suggesting using `reduce(operator.add, ...)` instead of `sum()`. He argues himself a few slides later that short functions are always better than long ones. If you look at the title of the slides, I think he's simply presenting what FP is: a chain of maps, filters and folds. It's true that Python will use a list comprehension or a generator expression where you would normally do this, but Python 3 still includes map() and filter(). (Despite Guido not being a big fan of these functions - http://www.artima.com/weblogs/viewpost.jsp?thread=98196 http://www.artima.com/weblogs/viewpost.jsp?thread=98196). My point is simply that the presentation does a good job at exposing what FP is and how to implement it in Python. Moving from maps to comprehensions comes naturally, once you get the hang of the idea. PS: I have the same question about functools.partial. Can somebody answer this?
- atdt 14y ago> ...faster and often clearer than lambdas. $ python -m timeit 'import string;from operator import itemgetter;[itemgetter('slice') for x in string.letters]' 100000 loops, best of 3: 14.2 usec per loop $ python -m timeit 'import string;from operator import itemgetter;[lambda x: x.slice for x in string.letters]' 100000 loops, best of 3: 10.4 usec per loop > Also: namedtuple is great instead of "using classes as attribute containers". Er, OK. Just don't look at the implementation[1]. Cannot be unseen. [1]: http://hg.python.org/cpython/file/2.7/Lib/collections.py#l233 http://hg.python.org/cpython/file/2.7/Lib/collections.py#l23...
- jrockway 14y agoI'm not sure I agree with "don't write classes". This is Python: classes are the state container. That's the structure Python provides for grouping related pieces of information together, and that's the structure all other Python libraries and programmers expect you to use. Now if you don't want to type the code every time, that's fine: use collections.namedtuple.
- andreasvc 14y agoA class without methods is just syntactic sugar wrapping around a dictionary of attributes, so in that sense it could be simpler to use the dictionary directly.
- jrockway 14y agoThe example from the slides was a __str__ method for a data structure, which makes a class the perfect fit in Python. (They did not call it __str__, of course.)
- nextstep 14y agoWow, this presentation is very easy to step through in mobile Safari! I like the style a lot. I wish I had the speaker to give a little more with each slide though.