6 ms·

Python and the Principle of Least Astonishment

207 points by mattyb 15y ago

Kilimanjaro 15y ago
Differentiating properties and methods from internal methods is as simple as using another char, like dots and colons. So box.length would not be the same as box:length, that way you could assign all names you want without fear of collision. Poor choices in language design will always haunt you til the end of times. That being said, python is my favorite language right now and the only complain I have is about underscores which I try to avoid. Now, GO being a new language, I can't really understand why it does not implement methods for primitive types like "hello".ToUpper(), instead we have to import "strings" and call strings.ToUpper("hello"). Easier for the compiler but harder for the programmer has never been my mantra.
- lucian1900 15y ago
  I'm not sure about different namespaces for special methods. Python's object system is so self-referential and so transparent that it'd be surprising. Go devs are of the opinion that what behaves like a function should be a function, with no exception. It's also the reason they don't do operator overloading, which I think is just dumb.
- dalke 15y ago
  Would you mean this to be a namespace which only the Python implementation could access? For example, if the protocol "it:next" is added (which is equivalent to next(it)) then is there any way to support that syntax in an older version? With next() as a builtin it's easy; try: next except NameError, and in case of exception, implement the function yourself. But if I can't implement it myself then there's no good migration path. With your particular choice of ":", then what would "d={a:length()}" be? Currently it's a 1-element set. What would "words[middle:length()]" do?
  - Kilimanjaro 15y ago
    "is there any way to support that syntax in an older version" Nop, that is exactly what I meant with decisions haunting you forever. You better start a new language than radically change the current implementation. With regard to my particular choice of ':' there are three solutions: 1. pick any of the 100k unicode chars for internal methods but imperative is to differentiate them to freely use attributes and methods as pleased. 2. force the compiler and the coder to disambiguate the possible same use as internal method: first assign then use n=a:length(), d={n} 3. design the language to use .. as slicer and {a=b} as dictionary assignment. Again, we are talking about design decisions for a new language, not to change the core of an existing language which is not an easy task. That is exactly why language design is not an exact science an very few make it to stardom.
    - dalke 15y ago
      My point is that it isn't "as simple as using another char." #1 doesn't work if you want things visible on most keyboards in the world. #2 is counter to a design principle (which Python uses) that humans are less good at resolving ambiguity than people (see "template <template <int>>" in C++ pre 2011), and #3 as a specific case means problems getting methods of floats, as in "1..hex()". Of course Python has that already with "1 .bit_length()" being different than "1.bit_length()". I agree that seemingly minor decisions can have a big impact in the future of a language. What I disagree with is that this choice of a special syntax for built-in methods is "simple", and I believe that doing it yields a language not meant for stardom.
- njharman 15y ago
  how is ':' less onerous than underscores i.e. '.__'? Sounds like you just have irrational hatred of underscore (or possibly love of colon?) btw having two or more access operators '.', ':' and distinguishing between "internal/language" and "user" methods is a poor language design choice over having a standard but unenforced __ means "special" and "internal/language" and "user" all being coequal in semantics, syntax, rights and responsibilities.
  - Kilimanjaro 15y ago
    "how is ':' less onerous than underscores i.e. '.__'?" There is no difference at all, just one char vs five if we count ending double-underscores too, and more visual noise. They both accomplish the same. Being both the same, what difference do you think there is between, in one hand .init() and .__init__(), and on the other hand .init() and :init() Just syntactic sugar, I rather pick my colon ;-)
anonymoushn 15y ago
For extra fun.... a = 5 def print_a(): print a # Prints 5 print_a() def print_and_assign_a(): print a a = 2 # raises UnboundLocalError print_and_assign_a() class PrintOnInitSet(set): def __init__(self, *args, **kwargs): print "init!" set.__init__(self, *args, **kwargs) # Creates a PrintOnInitSet and prints "init!" a = PrintOnInitSet([1,2]) # Creates a PrintOnInitSet and prints "init!" b = PrintOnInitSet([3,4]) # Creates a PrintOnInitSet and prints nothing c = a | b
- kbd 15y ago
  Well, your first example makes sense. It ensures you're always referring to the same-scoped 'a' throughout your function. FWIW if you said 'global a' at the start of 'print_and_assign_a', Python wouldn't have a problem: >>> a = 5 >>> def print_and_assign_a(): ... global a ... print(a) ... a = 2 ... >>> print_and_assign_a() 5 >>> print_and_assign_a() 2 Your second example, however, seems to show an implementation detail of Python leaking out, and is probably a bug. Union starts off by making a copy of 'a' and adds the elements of 'b' to it, rather than creating a new object and adding the elements of both. On a hunch that PyPy's implementation would be less special-cased, I installed PyPy and and it actually does print "Init!" on "c = a | b". So, I vote bug in CPython rather than an inconsistency in the language. One more question arises, however. Should "a = set([1,2])" be equivalent to "a = set(); a.add(1); a.add(2)"? I overloaded 'add' on your PrintOnInitSet and it's not: >>>> a = PrintOnInitSet(); a.add(1); a.add(2) init! Called add with item: 1 Called add with item: 2 >>>> a = PrintOnInitSet([1,2]) init!
  - anonymoushn 15y ago
    The 'print_and_assign_a' case comes up more often with a in an enclosing local scope, so you couldn't really get it to work prior to the introduction of 'nonlocal' in Python 3. The best you could do was something silly like a = [5] def print_and_assign_a(): print(a[0]) a[0]=2 It's cool that the second case works in other implementations. I hadn't thought to test that.
gasull 15y ago
I miss something like the Principle of Least Astonishment for design. Why every time there's a revamp of a website the new design is more astonishing and less clear? Why the search button in Google Calendar is now blue? Why new design is usually a tease on our muscle memory?
spacemanaki 15y ago
This is a nice article, as a non-Python programmer who's dabbled a bit in order to read some random Python code, it gave me some appreciation for the "Pythonic"-way. I have to say though, that the thing that astonished me the most about Python is the Java-like "closures". I sort of thought it would be like Ruby, which I thought was closer to Scheme and JavaScript, and then I realized that Ruby isn't quite like them either. (I know Ruby is close with its lambdas and blocks, but it's not intuitive to me.) This isn't necessarily to the detriment of Python (or Ruby) but just something I observed when trying to explain Java's lack of closures to someone who only knows Python.
- JeremyBanks 15y ago
  I have only a trivial amount of experience experience with Scheme, could you explain how its closures are different from Python's? I haven't used Java a lot either, but from what I have done Python's function seems more similar to JavaScript than to Java.
  - irahul 15y ago
    > I have only a trivial amount of experience experience with Scheme, could you explain how its closures are different from Python's? In pre Python 3, the closed over value isn't mutable unless it's a reference to a mutable object. def counter(num): def foo(): num += 1 return num return foo c = counter(5) c() This won't work because you can't mutate the closed variable `start`. This would work in languages with proper closures(Ruby, Perl, Scheme...). Here is how you do it in Ruby: def counter(n) lambda { n += 1 } end c = counter(5) c[] # returns 6 c.call() # Alternate syntax. returns 7 c[] # returns 8 The above python will work in Python 3 if the closed variable is declared `nonlocal`. def counter(num): def foo(): nonlocal num num += 1 return num return foo Or you can have workarounds in pre Python3. def counter(num): def foo(): foo.num += 1 return foo.num foo.num = num return foo
    - kbd 15y ago
rmccue 15y ago
Personally, I love that joining is ' '.join(list), since PHP (e.g.) works the same way, in that the first argument to implode() is the string.
- xtal 15y ago
  It being consistent with PHP is pretty much the same thing as being inconsistent.
andrewflnr 15y ago
Overall it's a good article, but I don't fully understand his complaint about decorators, and I think he may not understand them. @foo and @foo() intentionally mean very different things. I don't see how you could "add a parameter to a previously parameter-less decorator" without drastically changing the meaning of the whole thing. They may be tricky to wrap your head around, but there's nothing too surprising about decorators.
- irahul 15y ago
  > I think he may not understand them. I won't think so. He codes a lot, in general, and in Python. I have read some of his posts and code, and he has a deep understanding of Python and programming in general. His complain was about @foo and @foo() requires different decorator implementations. If @foo by default meant @foo(), that makes introducing parameters at a later time a bit more straightforward. I am not arguing about it being a valid expectation. I am just explaining what I think he meant.
- MostAwesomeDude 15y ago
  If I have the following code: @auth_required def some_view(): pass And later I decide to change that decorator to be parameterized: @auth_required("basic") def some_view(): pass I have now created a massive amount of pain for myself if I decide to define auth_required() such that it defaults its argument to "basic", unless I force people to call it as @auth_required(), because of the differing callable signatures. Some people feel @auth_required() is ugly. That's all.
  - the_mitsuhiko 15y ago
    > Some people feel @auth_required() is ugly. That's all. The biggest issue here is backwards compatibility. Nowadays I just make a separate decorator that accepts arguments and keep the old one around unchanged. I learned my lesson :)
    - d0m 15y ago
      You may also use a simple wrapper to create decorators which would automatically define @decorator and @decorator().
pak 15y ago
It's a bit dated, but the Python Cookbook does a great job of teaching you practical examples of Python if you already know other languages and want to quickly grok what is "pythonic." It starts with string crunching and hits just about every other general sysadmin use case, covering most of the standard library along the way.
- hello_moto 15y ago
  May I know why it is a bit dated? because I'm thinking to buy the book and the year is 2005, which is good enough for me since 2005 means Python 2.5 at least (or even more).
  - sateesh 15y ago
    The 2nd edition of Python Cookbook doesn't cover Python 2.5. It covers language features that were there till Python 2.4. (http://oreilly.com/catalog/9780596007973 http://oreilly.com/catalog/9780596007973) The 3rd edition of Python Cookbook is expected to appear by end of this year. http://dabeaz.blogspot.com/2010/12/oreilly-python-cookbook-python-3-all.html http://dabeaz.blogspot.com/2010/12/oreilly-python-cookbook-p...
astrofinch 15y ago
"I was quite fond of Raymond Hettinger's talk about what makes Python unique because he showed a bunch of small examples that almost exclusively used the good parts of the standard library and hardly any user written logic." Is this talk available anywhere?
- room606 15y ago
  I'm not certain but I think it was one of these http://ep2011.europython.eu/conference/speakers/raymond-hettinger http://ep2011.europython.eu/conference/speakers/raymond-hett...
  - mfieldhouse 15y ago
    Maybe this http://ep2011.europython.eu/conference/talks/what-makes-python-so-awesome http://ep2011.europython.eu/conference/talks/what-makes-pyth...
bluesnowmonkey 15y ago
So... Python has len() because it would be too hard to standardize on .len(). Any by the way, len() calls .__len__(), on which we have standardized. Got it.
- aston 15y ago
  The better explanation is that there are a series of fallback actions for all of the top level functions that delegate to the .__whatever__ methods. For example, if you write a class that quacks like a collection that has a length (it implements .__len__) Python can use that to get a value for bool() even if you didn't implement .__nonzero__.
  - julian37 15y ago
    When people inquire about the len() function it is mainly because it is (or at least looks like) a global function rather than an object method. In short, people wonder why you have to call len(x) instead of x.len(). This is a valid concern seeing that Python's standard library is, by and large, object oriented (c.f. string methods like "upper" or list methods like "append" or file methods like "read"). Neither the article nor your comment has a good explanation of why that is so. The article only harps on about why it's good that it's not called length() or getLength() or size() or getSize(), implying that len() is somehow the god-given name for this feature. And the fallback mechanism you refer to could just as well be implemented if len() was a method of the root class (object) rather than a global method. I like Python a lot but this is one that I never quite understood and I believe there isn't really a good reason for it, I think it was just a historical accident. Might as well admit it rather than trying to come up with half baked justifications after the fact.
    - mixmastamyk 15y ago
      Guido on the subject: http://mail.python.org/pipermail/python-3000/2006-November/004643.html http://mail.python.org/pipermail/python-3000/2006-November/0... tldr - He thought it more intuitive.
drallison 15y ago
The goal in language design is not "least astonishment" but inferability. And, while Python may not meet the least astonishment goal for new users coming from another language, Python is remarkably inferable and internally consistent.
njharman 15y ago
Written by someone who actually knows Python (which is rare for articles like this). The actual surprises section is great.

207 points by mattyb 15y ago

Differentiating properties and methods from internal methods is as simple as using another char, like dots and colons. So box.length would not be the same as box:length, that way you could assign all names you want without fear of collision. Poor choices in language design will always haunt you til the end of times. That being said, python is my favorite language right now and the only complain I have is about underscores which I try to avoid. Now, GO being a new language, I can't really understand why it does not implement methods for primitive types like "hello".ToUpper(), instead we have to import "strings" and call strings.ToUpper("hello"). Easier for the compiler but harder for the programmer has never been my mantra.

lucian1900 15y ago
I'm not sure about different namespaces for special methods. Python's object system is so self-referential and so transparent that it'd be surprising. Go devs are of the opinion that what behaves like a function should be a function, with no exception. It's also the reason they don't do operator overloading, which I think is just dumb.
dalke 15y ago
Would you mean this to be a namespace which only the Python implementation could access? For example, if the protocol "it:next" is added (which is equivalent to next(it)) then is there any way to support that syntax in an older version? With next() as a builtin it's easy; try: next except NameError, and in case of exception, implement the function yourself. But if I can't implement it myself then there's no good migration path. With your particular choice of ":", then what would "d={a:length()}" be? Currently it's a 1-element set. What would "words[middle:length()]" do?
- Kilimanjaro 15y ago
  "is there any way to support that syntax in an older version" Nop, that is exactly what I meant with decisions haunting you forever. You better start a new language than radically change the current implementation. With regard to my particular choice of ':' there are three solutions: 1. pick any of the 100k unicode chars for internal methods but imperative is to differentiate them to freely use attributes and methods as pleased. 2. force the compiler and the coder to disambiguate the possible same use as internal method: first assign then use n=a:length(), d={n} 3. design the language to use .. as slicer and {a=b} as dictionary assignment. Again, we are talking about design decisions for a new language, not to change the core of an existing language which is not an easy task. That is exactly why language design is not an exact science an very few make it to stardom.
  - dalke 15y ago
    My point is that it isn't "as simple as using another char." #1 doesn't work if you want things visible on most keyboards in the world. #2 is counter to a design principle (which Python uses) that humans are less good at resolving ambiguity than people (see "template <template <int>>" in C++ pre 2011), and #3 as a specific case means problems getting methods of floats, as in "1..hex()". Of course Python has that already with "1 .bit_length()" being different than "1.bit_length()". I agree that seemingly minor decisions can have a big impact in the future of a language. What I disagree with is that this choice of a special syntax for built-in methods is "simple", and I believe that doing it yields a language not meant for stardom.
njharman 15y ago
how is ':' less onerous than underscores i.e. '.__'? Sounds like you just have irrational hatred of underscore (or possibly love of colon?) btw having two or more access operators '.', ':' and distinguishing between "internal/language" and "user" methods is a poor language design choice over having a standard but unenforced __ means "special" and "internal/language" and "user" all being coequal in semantics, syntax, rights and responsibilities.
- Kilimanjaro 15y ago
  "how is ':' less onerous than underscores i.e. '.__'?" There is no difference at all, just one char vs five if we count ending double-underscores too, and more visual noise. They both accomplish the same. Being both the same, what difference do you think there is between, in one hand .init() and .__init__(), and on the other hand .init() and :init() Just syntactic sugar, I rather pick my colon ;-)

anonymoushn 15y ago

For extra fun.... a = 5 def print_a(): print a # Prints 5 print_a() def print_and_assign_a(): print a a = 2 # raises UnboundLocalError print_and_assign_a() class PrintOnInitSet(set): def __init__(self, *args, **kwargs): print "init!" set.__init__(self, *args, **kwargs) # Creates a PrintOnInitSet and prints "init!" a = PrintOnInitSet([1,2]) # Creates a PrintOnInitSet and prints "init!" b = PrintOnInitSet([3,4]) # Creates a PrintOnInitSet and prints nothing c = a | b

kbd 15y ago
Well, your first example makes sense. It ensures you're always referring to the same-scoped 'a' throughout your function. FWIW if you said 'global a' at the start of 'print_and_assign_a', Python wouldn't have a problem: >>> a = 5 >>> def print_and_assign_a(): ... global a ... print(a) ... a = 2 ... >>> print_and_assign_a() 5 >>> print_and_assign_a() 2 Your second example, however, seems to show an implementation detail of Python leaking out, and is probably a bug. Union starts off by making a copy of 'a' and adds the elements of 'b' to it, rather than creating a new object and adding the elements of both. On a hunch that PyPy's implementation would be less special-cased, I installed PyPy and and it actually does print "Init!" on "c = a | b". So, I vote bug in CPython rather than an inconsistency in the language. One more question arises, however. Should "a = set([1,2])" be equivalent to "a = set(); a.add(1); a.add(2)"? I overloaded 'add' on your PrintOnInitSet and it's not: >>>> a = PrintOnInitSet(); a.add(1); a.add(2) init! Called add with item: 1 Called add with item: 2 >>>> a = PrintOnInitSet([1,2]) init!
- anonymoushn 15y ago
  The 'print_and_assign_a' case comes up more often with a in an enclosing local scope, so you couldn't really get it to work prior to the introduction of 'nonlocal' in Python 3. The best you could do was something silly like a = [5] def print_and_assign_a(): print(a[0]) a[0]=2 It's cool that the second case works in other implementations. I hadn't thought to test that.

gasull 15y ago

I miss something like the Principle of Least Astonishment for design. Why every time there's a revamp of a website the new design is more astonishing and less clear? Why the search button in Google Calendar is now blue? Why new design is usually a tease on our muscle memory?

spacemanaki 15y ago

This is a nice article, as a non-Python programmer who's dabbled a bit in order to read some random Python code, it gave me some appreciation for the "Pythonic"-way. I have to say though, that the thing that astonished me the most about Python is the Java-like "closures". I sort of thought it would be like Ruby, which I thought was closer to Scheme and JavaScript, and then I realized that Ruby isn't quite like them either. (I know Ruby is close with its lambdas and blocks, but it's not intuitive to me.) This isn't necessarily to the detriment of Python (or Ruby) but just something I observed when trying to explain Java's lack of closures to someone who only knows Python.

JeremyBanks 15y ago
I have only a trivial amount of experience experience with Scheme, could you explain how its closures are different from Python's? I haven't used Java a lot either, but from what I have done Python's function seems more similar to JavaScript than to Java.
- irahul 15y ago
  > I have only a trivial amount of experience experience with Scheme, could you explain how its closures are different from Python's? In pre Python 3, the closed over value isn't mutable unless it's a reference to a mutable object. def counter(num): def foo(): num += 1 return num return foo c = counter(5) c() This won't work because you can't mutate the closed variable `start`. This would work in languages with proper closures(Ruby, Perl, Scheme...). Here is how you do it in Ruby: def counter(n) lambda { n += 1 } end c = counter(5) c[] # returns 6 c.call() # Alternate syntax. returns 7 c[] # returns 8 The above python will work in Python 3 if the closed variable is declared `nonlocal`. def counter(num): def foo(): nonlocal num num += 1 return num return foo Or you can have workarounds in pre Python3. def counter(num): def foo(): foo.num += 1 return foo.num foo.num = num return foo
  - kbd 15y ago

rmccue 15y ago

Personally, I love that joining is ' '.join(list), since PHP (e.g.) works the same way, in that the first argument to implode() is the string.

xtal 15y ago
It being consistent with PHP is pretty much the same thing as being inconsistent.

andrewflnr 15y ago

Overall it's a good article, but I don't fully understand his complaint about decorators, and I think he may not understand them. @foo and @foo() intentionally mean very different things. I don't see how you could "add a parameter to a previously parameter-less decorator" without drastically changing the meaning of the whole thing. They may be tricky to wrap your head around, but there's nothing too surprising about decorators.

irahul 15y ago
> I think he may not understand them. I won't think so. He codes a lot, in general, and in Python. I have read some of his posts and code, and he has a deep understanding of Python and programming in general. His complain was about @foo and @foo() requires different decorator implementations. If @foo by default meant @foo(), that makes introducing parameters at a later time a bit more straightforward. I am not arguing about it being a valid expectation. I am just explaining what I think he meant.
MostAwesomeDude 15y ago
If I have the following code: @auth_required def some_view(): pass And later I decide to change that decorator to be parameterized: @auth_required("basic") def some_view(): pass I have now created a massive amount of pain for myself if I decide to define auth_required() such that it defaults its argument to "basic", unless I force people to call it as @auth_required(), because of the differing callable signatures. Some people feel @auth_required() is ugly. That's all.
- the_mitsuhiko 15y ago
  > Some people feel @auth_required() is ugly. That's all. The biggest issue here is backwards compatibility. Nowadays I just make a separate decorator that accepts arguments and keep the old one around unchanged. I learned my lesson :)
  - d0m 15y ago
    You may also use a simple wrapper to create decorators which would automatically define @decorator and @decorator().

pak 15y ago

It's a bit dated, but the Python Cookbook does a great job of teaching you practical examples of Python if you already know other languages and want to quickly grok what is "pythonic." It starts with string crunching and hits just about every other general sysadmin use case, covering most of the standard library along the way.

hello_moto 15y ago
May I know why it is a bit dated? because I'm thinking to buy the book and the year is 2005, which is good enough for me since 2005 means Python 2.5 at least (or even more).
- sateesh 15y ago
  The 2nd edition of Python Cookbook doesn't cover Python 2.5. It covers language features that were there till Python 2.4. (http://oreilly.com/catalog/9780596007973 http://oreilly.com/catalog/9780596007973) The 3rd edition of Python Cookbook is expected to appear by end of this year. http://dabeaz.blogspot.com/2010/12/oreilly-python-cookbook-python-3-all.html http://dabeaz.blogspot.com/2010/12/oreilly-python-cookbook-p...

astrofinch 15y ago

"I was quite fond of Raymond Hettinger's talk about what makes Python unique because he showed a bunch of small examples that almost exclusively used the good parts of the standard library and hardly any user written logic." Is this talk available anywhere?

room606 15y ago
I'm not certain but I think it was one of these http://ep2011.europython.eu/conference/speakers/raymond-hettinger http://ep2011.europython.eu/conference/speakers/raymond-hett...
- mfieldhouse 15y ago
  Maybe this http://ep2011.europython.eu/conference/talks/what-makes-python-so-awesome http://ep2011.europython.eu/conference/talks/what-makes-pyth...

bluesnowmonkey 15y ago

So... Python has len() because it would be too hard to standardize on .len(). Any by the way, len() calls .__len__(), on which we have standardized. Got it.

aston 15y ago
The better explanation is that there are a series of fallback actions for all of the top level functions that delegate to the .__whatever__ methods. For example, if you write a class that quacks like a collection that has a length (it implements .__len__) Python can use that to get a value for bool() even if you didn't implement .__nonzero__.
- julian37 15y ago
  When people inquire about the len() function it is mainly because it is (or at least looks like) a global function rather than an object method. In short, people wonder why you have to call len(x) instead of x.len(). This is a valid concern seeing that Python's standard library is, by and large, object oriented (c.f. string methods like "upper" or list methods like "append" or file methods like "read"). Neither the article nor your comment has a good explanation of why that is so. The article only harps on about why it's good that it's not called length() or getLength() or size() or getSize(), implying that len() is somehow the god-given name for this feature. And the fallback mechanism you refer to could just as well be implemented if len() was a method of the root class (object) rather than a global method. I like Python a lot but this is one that I never quite understood and I believe there isn't really a good reason for it, I think it was just a historical accident. Might as well admit it rather than trying to come up with half baked justifications after the fact.
  - mixmastamyk 15y ago
    Guido on the subject: http://mail.python.org/pipermail/python-3000/2006-November/004643.html http://mail.python.org/pipermail/python-3000/2006-November/0... tldr - He thought it more intuitive.

drallison 15y ago

The goal in language design is not "least astonishment" but inferability. And, while Python may not meet the least astonishment goal for new users coming from another language, Python is remarkably inferable and internally consistent.

njharman 15y ago

Written by someone who actually knows Python (which is rare for articles like this). The actual surprises section is great.