7 ms·
Python has closures
- crizCraig 16y agoThe crappy thing about Python closures is that you have to define the function above where you call it. I think this kind of puts the cart before the horse and messes of readability. JavaScript, on the other hand, allows you to define the inner function anywhere in the local scope and makes closures much more attractive, IMO. Changing this may mean big changes to the interpreter and so may not be worth it. I wonder if lambda created closures are any different...probably not.
- Jach 16y agoYou can of course define single line lambdas inline, and I think those are quite attractive, but when you want to get multiline lambdas you get in quite a sticky situation with Python. For a simple example, see the top answer here: http://stackoverflow.com/questions/1233448/no-multiline-lambda-in-python-why-not http://stackoverflow.com/questions/1233448/no-multiline-lamb... In order to make it work Python would have to introduce a scope-blocking structure like braces, parens, or an 'endlambda', all of which I think are steps in the wrong direction, especially given Python's great feature of semantic whitespace. I'll take a clean function defined above over braces. I do know of a way to hack a multi-line Lambda with existing syntax, though, but I have no guarantees about it. I don't think I've used it in serious code, since it's just real ugly. k = lambda x: eval(compile(""" print x y = 4 print x + y x = 8 print x """, '', 'exec')) x = 5 k(2) # 2 # 6 # 8 print x # 5
- jimbokun 16y ago"I don't know whether Guido will ever add it for, er, closure, but I do know that I have _never_ _once_ needed this keyword in hundreds of thousands of lines of real Python code I have written." This is the blubbiest non-ironic statement I can remember reading.
- cdavid 16y agoCould you give examples of useful closures (actually multi-line lambda) ? I am relatively proeficient in python, and am learning javascript, and I really don't like how non trivial anonymous functions are pervasive in the later. I understand it may just be habit, though.... OTOH, one nice aspect in say ruby is the combo "real" anonymous functions + the syntax f a equivalent of f(a), which is what makes DSL possible in ruby (and impractical in python in general). I think it comes at a price in readability, though.
- jpr 16y agoBy the way, what is the idiomatic way to do this in Python? (let ((x 1)) (defun less () (incf x)) (defun more () (decf x)))
- m0th87 16y agoInstead of adding an "intermediate" keyword, I think JavaScript's approach is preferable. That is, > var x = 0 Creates a new variable. > x = 0 Assigns a variable if it has already been declared in a parent scope, or creates a new global variable otherwise. Of course, this work break all existing Python code, so it would never happen.
- tptacek 16y agoThis semantic is pretty much anathema to Python, which tries to be explicit and to default conservatively. I'm pretty experienced with each of these languages and I routinely forget to qualify variables with "var" and end up stomping on globals.
- AndyKelley 16y agoI think you're throwing the baby out with the bathwater. Using a keyword like "var" is probably the cleanest way to define where the variable is defined. And we don't need to have the default case be implied global. If you reference a variable which has not been declared with "var", have it crash and die. I think this is the best of both worlds.
- tptacek 16y agoThe only way your argument makes sense is if variables are default-global and tightly scoped only when the keyword is applied. If you adopt that scheme, Python doesn't suffer from the syntactical inconvenience this article talks about. Otherwise, you're making the same point that the article is making, which is that you could introduce a scoping keyword to make it easier to deal with closed-over variables. Either way, I don't care; I'm a Ruby developer now.
- m0th87 16y agoIt's not the same point: you would never have to explicitly tell the interpreter where the variable is (global, local or nonlocal). If you required all variable declarations to use var, the interpreter could automatically determine the scope of the variable when it is referenced or updated. This is cleaner IMO.
- supersillyus 16y agoI like the part where he notes that this isn't a real problem except that it creates centithreads of argument from people who believe it to be a real problem. As I grow older and less idealistic, I'm more sympathetic to the notion that problems in theory but not in practice aren't really problems.
- alec 16y agoI write "real code" in Common Lisp, and I used something like what he dismissed as "useless in practice" twice today to make my code more readable and maintainable. I do not consider myself a Lisp advocate when I bemoan the lack of sane scoping semantics most similar languages just get right; I just want to write better code.
- jerf 16y agoYes, but Lisp code and Python code aren't the same when program in the native idioms. In Python if your function references are buried within each other deeply enough that this is even an issue you're already doing it wrong. This is not because Python is lacking power, but because you're not using the power Python actually has. With the generators, comprehensions, objects that implement __call_, bound method references, and all of the other tools available to you, you just shouldn't be defining functions in functions in functions. In Haskell where it's all functions everywhere, a function that is actually ten layer nested functions calling each other makes sense. (When using the "do" syntax sugar, every line that has <- is another nested function, so if you see ten <-s, you've got a ten-layer deep nested function. (Or eleven, if I've committed a fencepost error.)) If you're still fighting with Python by trying to jam functions in three layers deep, take a moment and try to learn to be more idiomatic. If you're in Haskell or Lisp and you never have functions three layers deep, take a moment and try to be more idiomatic. In practice, you do not need this in Python.
- deleted 16y ago[deleted]
- andolanra 16y agoPython 3 already has the nonlocal keyword which does exactly what it sounds like. It's explained in mind-numbing detail by PEP 3104 at http://www.python.org/dev/peps/pep-3104/ http://www.python.org/dev/peps/pep-3104/ but it's really what you'd expect. def make_counter(): n = 0 def incr(): nonlocal n temp, n = n, n+1 return temp return incr
- tkahn6 16y agoI think `nonlocal` is a prime example of a leaky abstraction. Can anyone explain the advantages of using references for normally primitive variables?
- artsrc 16y agononlocal 'leaks' that assigning an new identifier declares a new local variable. This is part of the abstraction not a leak. I don't think I understand your other question.
- parfe 16y agoWhat is your definition of primitive?
- kgo 16y agoThere are no primitive variables in python. Even ints, strings, floats, etc point to a reference of an object. All nonlocal does is point the variables reference to a different object. As to the advantages... Well for one you can use the variables in closures...
- andolanra 16y agoI'm not entirely sure what you mean by "using references for normally primitive variables." Nonlocal doesn't change whether variables are references or not, only the semantics of assignment. By default, all assignments take place in the local scope; if you've indicated that a variable is nonlocal, then instead of creating a new variable in that scope you assign instead to the variable of that name in the nearest enclosing scope. All variables in Python are references to objects, so there is no such thing as a "primitive" type, but that was true even before the nonlocal keyword was introduced, so it's not relevant here. Lisp has had proper support for lexical scoping and closures for ages, and nobody has ever accused it of being a leaky abstraction there, either. The Scheme example of the same program would be (define (make-counter) (let ((n 0)) (lambda () (let ((temp n)) (set! n (+ n 1)) temp)))) All that's happening here is that let is creating a new variable, and set! is looking for the nearest enclosing scope--which turns out to be the let--and assigning to that one. (Okay, there are minor semantic differences, because let creates a new scope instead of creating a new variable in the local scope, but that's not really relevant to the discussion at hand.)
- dfox 16y agoAs I understand CPython, problem with this, is that python's "closure" does not contain reference to parent environment frame, but it's copy, so this is not only syntactic issue. Also putting reference to parent environment would create reference cycle, which was clearly undesirable in times when python did only reference counting.
- nas 16y agoNo, I don't think so. Closures in Python are very similar to Scheme (or modern Lisps). Python doesn't have an equivalent to set!. Instead assignment both defines a local variable (if it doesn't already exist) and sets it. You can use "global" or the new "nonlocal" keywords to prevent the definition. Personally I don't mind the Python approach. It's more convenient than let/set! and I very rarely miss being able to set a variable in an outer scope.
- stcredzero 16y agoOne of the most powerful features of Smalltalk Blocks is the ability to reference everything in the parent context. This gives one the ability to Don't Repeat Yourself in even the stickiest of situations. A lot of tricks for not repeating yourself involve passing a Block from the current context into a function, where it is executed, possibly inside an exception handling block. 99% of everything you'd be hard pressed to avoid repeating yourself with can be taken care of using tricks based on this idea. EDIT: Come to think of it, this is just a good-enough substitute for a fraction of the stuff you can do in Lisp with Macros.
- rlpb 16y agoI always wondered what closures actually were. I'd read up on them, but apparently Python didn't have them so I always felt that I was missing something. Thanks for clearing this up.
- Hemospectrum 16y agoA lot of people don't seem to understand the difference between closures and lambdas, and a lot of people get it backwards as soon as someone points it out to them. Good to know there's one clear demonstration out there. Actually, if you could pass this on to people when you hear them refer to lambdas as closures, that would be pretty awesome.
- KirinDave 16y agoDoes it actually reference the parent, or does it just make a copy? One is a closure, the other is an implementation artifact.
- rlpb 16y agoFrom a quick test, it appears that it can reference everything the parent can reference and does not make a copy.
- gxti 16y agoIt doesn't make a copy. It creates a special object (which you don't see unless you're digging in the internals) called a "cell" which holds a reference to the object for as long as the function exists. To see for yourself, look at `somefunc.func_closure` -- it'll be `None` on regular functions, and a tuple of cells on a closure function.
- abstractbill 16y ago_But_ the addition would be useless in practice because this problem almost never arises in real code. Its only purpose would be to answer Lisp and Ruby advocates who want to say that Python doesn't really have closures. A great example of the Blub paradox in action ;)
- btilly 16y agoA great example of the Blub paradox in action ;) Actually not. There is no code you can write using closures modifying external state that cannot be trivially rewritten in Python using a one-element array in the outer scope which you modify in the inner (without assigning). Therefore Python has that feature in effect. It just doesn't have a piece of dedicated syntax for it.
- ubernostrum 16y agoYeah, and Haskell programmers love having it pointed out that their language makes them stupid by not allowing them to do this ;) Oh wait. Nobody says that about Haskell programmers? They only say it about Python? Huh. When they both have this same shortcoming with their so-called "closures"? Wonder why that is?
- jpr 16y agoNot allowing what? Rebinding of variables? That's what IORefs, MVars etc. are for.
- chromatic 16y agoAnyone who's reached that point with Haskell is comfortable with the idea of immutable state. Anyone who's reached that point with Python is very comfortable with the idea of mutable state... everywhere else.
- ubernostrum 16y agoWhich is funny, because in both cases they're symptoms of trade-offs made in the language design. But a Haskell programmer who understands this and decides to accept it is just considered normal. A Python programmer who understands this and decides to accept it (as in the linked post) is declared to be a Blub user, which is essentially a polite-enough-for-HN way to say "LOL STOOPID NOOOB".
- lispm 16y agoUnfortunately in the example there is not a single closure shown. A closure is a function and its bindings. But there are no function objects created. What he shows is some attempt to describe some kind of lexical binding. But even that looks very suspicious. def foo(): global y x = 0 def bar(): print x, y bar() x = y = 1 bar() x = y = 2 bar() x = y = 3 bar() x = y = 4 bar() x = y = 5 foo() 0 0 1 1 2 2 3 3 4 4 Above is really trivial. The problem is that the assignments change a variable and the function BAR sees the changes. So there is only one variable binding for X. In a variant where each assignments were really bindings, BAR would only see the binding in its scope and the number would not change. Above would be in Lisp: (defvar *y* 0) (defun foo () (let ((x 0)) (flet ((bar () (print (list x *y*)))) (bar) (setf *y* 1 x *y*) (bar) (setf *y* 2 x *y*) (bar) (setf *y* 3 x *y*) (bar) (setf *y* 4 x *y*) (bar) (setf *y* 5 x *y*)))) Again, no closure. Just a function with a local function that gets called several times. He then describes that in a 'subfunction' a variable from an outside scope can't be set, but that is almost a side issue. It also may be that this is not often found in his code, but it is found in a lot of Lisp code that actually uses closures. The most trivial example is a counter closure: (let ((fn (let ((x 0)) (lambda () (setf x (+ x 1)))))) (print (funcall fn)) (print (funcall fn)) (print (funcall fn))) 1 2 3 In above code FN gets bound to a closure. This closure is called three times. Each time it updates the variable X in its lexical scope.
- btilly 16y agoNo, they absolutely are full closures. He just didn't demonstrate that fact to your satisfaction. Here is a code example that may be more convincing to you. def make_counter(name): position = [0] def counter(): position[0] += 1 return (name, "incr", position[0]) def show_counter(): return (name, "show", position[0]) return counter, show_counter foo, see_foo = make_counter("foo") bar, see_bar = make_counter("bar") for i in range(5): print foo() print see_foo() for i in range(5): print bar() print see_bar() for i in range(5): print foo() print foo() print see_foo() print bar() print see_bar() If you run that you'll see that the returned closures retain lexical bindings to the scope they were defined in. Closures generated at different times do not interfere with each other. And closures generated in the same scope are pointing at the same variables. What more do you want?