17 ms·
Python's Mutable Default Problem
- neutronicus 16y ago"stuff = stuff or []" This idiom is baked into the perl community. It's kind of funny to see a Pythonista deciding it's a good idea. (You can tell it hurts him too).
- jacobolus 16y agoIt’s not a good idea in Python. There are legitimate reasons to pass falsy objects as function arguments.
- robinhouston 16y agoFunnily enough, Perl (since version 5.10) actually has a solution to this. The expression $foo // "default" evaluates to "default" if $foo is undefined, and to $foo otherwise (even if it is defined but false).
- dagw 16y agoI've been programming python for on and off 12 years, and full time for the past 3, and this is the first time I've seen anyone recommending that idiom. So on the whole the python community does not think it's a good idea.
- deleted 16y ago[deleted]
- code_duck 16y agoJavaScript, too: stuff = stuff||[] since JavaScript doesn't even support default argument values.
- spenrose 16y agoYes, it's a wart. You learn the patterns he mentions pretty quickly.
- code_duck 16y agoI knew the pattern well from other languages, but would have assumed that it wasn't necessary in Python. So, this is good to note.
- sophacles 16y agoI'm not sure this constitutes any problem other than a lack of understanding of the python runtime. What the author describes as: "the mutable default parameter quirk is an ugly corner worth avoiding" could also be described as: "a natural outcropping of python's late binding, "names are references" variable model, and closure mechanisms, which provide a consistency to the language that is often crufted up in others" I do somewhat agree with the author that this particular functionality should be a "use only when needed" feature. I don't think it should be avoided at all costs tho, because there are times where the mutable default allows for a lot of saved code. In fact in a few cases the code to work around using mutable defaults can get into some serious voodoo because frequently the writer is actually trying to work around the bigger mutable/immutable objects and names are references "issues" in python. This also reminds me of something I was reading on the front page today about the old 'use the whole language' vs 'simplicity is king' holy war.
- neutronicus 16y ago"a natural outcropping of python's late binding, "names are references" variable model, and closure mechanisms, which provide a consistency to the language that is often crufted up in others" My mileage varies. I'd prefer default parameters to honor referential transparency, whatever hoops the runtime has to jump through to make this happen.
- emehrkay 16y agoPylint (or maybe it was pep8) has told me not to make dicts default arguments when running it against my code, but didn't explain why. Thanks for the post.
- baq 16y agoit's a gotcha but it makes perfect sense once you understand why it works that way - i.e. the difference between evaluating a function definition and calling it.
- aston 16y agoAs noted in the comments, DON'T use stuff = stuff or [] because if you pass an empty list, you'll get a new one rather than mutating the one you passed. stuff = stuff if stuff is not None else [] is wordy, but at least it's correct.
- btilly 16y agoI prefer the usual alternative if stuff is None: stuff = []
- itsnotvalid 16y agoSo indeed the auth of the article is really too clever on this.
- xiongchiamiov 16y agoAnd as usual, being clever in Python equates to getting yourself in trouble. Dammit, people - in Python, "That's clever" is an insult. Learn the philosophy of a language and you'll be much happier using it.
- code_duck 16y agoThat's exactly why I prefer Python over Ruby: not the language, but the philosophy of the community. The Ruby community seems to adore 'clever', while the Python community explicitly shuns it. I view the latter as being more mature and borne of experience.
- chc 16y agoPython itself is pretty "clever" compared to many other languages (GC, lack of obvious 1:1 correspondance between code written and code executed, dynamic typing, significant indentation, decorators, generators, etc.). If Python programmers were really opposed to cleverness, they'd be writing in straightforward assembly or a very thin veneer over it.
- perlgeek 16y agoFWIW, Perl 6 implicitly treats default values as closures, and calls them when no argument is passed that could bind to the optional argument. That way you get a fresh array each time, and you can even use defaults that depend on previous arguments: sub integrate(&integrand, $from, $to, $step = ($to - $from) / 100) { ... }
- clay 16y ago"stuff = stuff or []" This would fail to have expected behaviour here: fill_list = [] stuff = function(info, full_list) print fill_list use the if list is None: paradigm
- wyuenho 16y agoIt's on SO's Python FAQ too. This is the only thing in Python that's really bitten me. I remember it took me like a week to figure this out when I was tearing down my algorithm bit by bit to find out whether my prove was wrong or the code. http://stackoverflow.com/questions/1132941/least-astonishment-in-python-the-mutable-default-argument http://stackoverflow.com/questions/1132941/least-astonishmen...
- viraptor 16y agoNot sure if that's a "problem" if the only other way to implement function-static variables would be to add a variable visible for the whole module, or tricks with decorators... @statics(blah=[]) def foo(normal_args, **kwargs): # or def foo(normal_args, blah): It messes up the idea of looking at the definition to find the function signature.
- riobard 16y agoYou just need to fully understand when Python does evaluation. import types def function(item, stuff = lambda: []): if type(stuff) == types.FunctionType: stuff = stuff() stuff.append(item) print stuff function(1) # prints '[1]' function(2) # prints '[2]' In Scala it has a better syntax because of typed function object: trait Map[A, B] { … def getOrElse (key: A, default: ⇒ B): B … } the `default` parameter is a function, so when you do getOrElse(someKey, defaultValue) The `defaultValue` becomes a function that generates the value you put there when called.
- pedro3005 16y agoYou can use this trick for memoization. Example: http://paste.pocoo.org/show/341849/ http://paste.pocoo.org/show/341849/
- underwater 16y agoMemorization is an implementation detail and shouldn't be part of the arguments list.
- code_duck 16y agoThe article is right, though, in stating that later viewers of your code may not be expecting this behavior, and it culd lead to problems.
- mark-r 16y agoI keep seeing this "problem" come up, but I don't understand how it's realistic. If you have a function that modifies a parameter as a side effect, why would you have a default value for the parameter? And since the site's comments seem to be taken over by link spam, is this mention on Hacker News just a clever way to juice the Google rank of said spam?
- ylem 16y agoI saw it bite some people where I work. It's now used for discussion with potential hires.
- jfm3 16y agoThe Pythonist doth protest too much, methinks.
- drstrangevibes 16y agoi think its clearer and more pythonic in this case to do def function(item, stuff): .... blah blah def function(item): function(item, [])
- dustingram 16y agoThis 'problem' can actually come in handy when used with a regex callback function. See if you can determine what this does: def cbk(match, nb = [0] ): if len(match.group())==len(nb): nb[-1] += 1 elif len(match.group())>len(nb): nb.append(1) else: nb[:] = nb[0:len(match.group())] nb[-1] += 1 return match.group()+' '+('.'.join(map(str,nb))) str = re.compile('^(#+)',re.MULTILINE).sub(cbk,str)
- d0mine 16y agoThe code converts: ## # To: ## 0.1 # 1 str is builtin, don't use it as a variable name especially if you use it in its original role.
- dustingram 16y agoCorrect, & thanks for the tip! But don't worry, I changed the variable name from my copy & paste and just didn't give it a second thought.
- billmcneale 16y agoThe problem is not so much about mutable, it's that default parameters escape the scope of their method. Which is very, very messed up (but not the first thing Python messed up).