7 ms·

Python Oddities

teddyh 9y ago
I’ve got one! First, some context: This is normal assignment unpacking: >>> a, b = 1, 2 >>> [a, b] [1, 2] (Mismatch gives errors) >>> a, b = [1] ValueError >>> a, b = [1, 2, 3] ValueError Extra parentheses works too: >>> (a, b) = [2, 3] >>> [a, b] [2, 3] Unpacking a single value: >>> (a,) = [4] >>> a 4 Works without parentheses too: >>> a, = [5] >>> a 5 No values does not work: >>> , = [] SyntaxError Neither does empty parentheses: >>> () = [] SyntaxError Now, the oddity: >>> [] = [] >>> This raises no error! Neither does this: >>> [] = () >>>
- makecheck 9y ago
  This isn't doing what you think though. There are container objects underneath. For instance "(a, b)" is "a tuple object containing two things, the values named "a" and "b" so you can still refer to them separately. "[a, b]" is a new list with those same elements, not an alias for "(a, b)".
  - masklinn 9y ago
    The oddity is that unpacking an empty iterable works when the target is a "list", but not when it's a "tuple". In fact it so doesn't work it actually fails during parsing. And there are no collections underneath the LHS, it compiles to an UNPACK_SEQUENCE opcode in both cases: >>> @dis.dis ... def foo(): ... (a, b) = bar() ... 3 0 LOAD_GLOBAL 0 (bar) 3 CALL_FUNCTION 0 (0 positional, 0 keyword pair) 6 UNPACK_SEQUENCE 2 9 STORE_FAST 0 (a) 12 STORE_FAST 1 (b) 15 LOAD_CONST 0 (None) 18 RETURN_VALUE >>> @dis.dis ... def foo(): ... [a, b] = bar() ... 3 0 LOAD_GLOBAL 0 (bar) 3 CALL_FUNCTION 0 (0 positional, 0 keyword pair) 6 UNPACK_SEQUENCE 2 9 STORE_FAST 0 (a) 12 STORE_FAST 1 (b) 15 LOAD_CONST 0 (None) 18 RETURN_VALUE >>> @dis.dis ... def foo(): ... [] = bar() ... 3 0 LOAD_GLOBAL 0 (bar) 3 CALL_FUNCTION 0 (0 positional, 0 keyword pair) 6 UNPACK_SEQUENCE 0 9 LOAD_CONST 0 (None) 12 RETURN_VALUE
    - makecheck 9y ago
      It’s a good point that Python is smart enough not to build a mutable list on the right-hand side when not needed (e.g. for "x = [1, 2]" it will and for "y, z = [1, 2]" it will not). A right-hand tuple however is still a separate type, and if you disassemble the original cases you can see that "[] = []" is more wasteful than "[] = ()". >>> import dis >>> >>> @dis.dis ... def f(): ... a, b = 1, 2 ... (a, b) = [2, 3] ... (a,) = [4] ... a, = [5] ... [] = [] ... [] = () ... 3 0 LOAD_CONST 6 ((1, 2)) 3 UNPACK_SEQUENCE 2 6 STORE_FAST 0 (a) 9 STORE_FAST 1 (b) 4 12 LOAD_CONST 2 (2) 15 LOAD_CONST 3 (3) 18 ROT_TWO 19 STORE_FAST 0 (a) 22 STORE_FAST 1 (b) 5 25 LOAD_CONST 4 (4) 28 STORE_FAST 0 (a) 6 31 LOAD_CONST 5 (5) 34 STORE_FAST 0 (a) 7 37 BUILD_LIST 0 40 UNPACK_SEQUENCE 0 8 43 LOAD_CONST 7 (()) 46 UNPACK_SEQUENCE 0 49 LOAD_CONST 0 (None) 52 RETURN_VALUE
- StavrosK 9y ago
  Hmm, why is that an oddity? Unpacking a list of N items into N variables works fine, where N >= 0.
  - masklinn 9y ago
    It works when you use a "list literal" as target, but with a "tuple literal" it only works for N > 0.
- jbg_ 9y ago
  This appears to no longer be the case in Python 3: $ python2 Python 2.7.13 (default, Jul 21 2017, 03:24:34) >>> () = [] File "<stdin>", line 1 SyntaxError: can't assign to () >>> $ python3 Python 3.6.2 (default, Jul 20 2017, 03:52:27) >>> () = [] >>>
  - masklinn 9y ago
    > This appears to no longer be the case in Python 3: It was fixed in Python 3.6 specifically, the error still occurs under 3.5.
dom0 9y ago
Some of these are pretty simple and not odd, e.g. immutability of tuples, iterator behaviour or modifying collections during iteration. ————————————————————————————————————— Somewhat related video: https://www.youtube.com/watch?v=qCGofLIzX6g https://www.youtube.com/watch?v=qCGofLIzX6g
wyclif 9y ago
Best use of Twitter Moments I've ever seen.
- StavrosK 9y ago
  First use of Twitter Moments I've ever seen. What is this? A collection of tweets? I'm confused because they all seem to be by the author, did he tweet them himself and collected them? The page's UI looks weird, I'm not sure if Twitter had a redesign or if I'm blocking some of their CSS.
  - th 9y ago
    I think they're pretty much a collection of tweets. This was the first time I used moments and I'm not sure where I saw them used before I made this. They're mostly by me because I made a goal for myself to tweet out one of these every week for a year. Others have shared their own and I've tweeted my own since then, but Twitter's hashtag search is pretty bad so I made a moment.
    - StavrosK 9y ago
      I think it works quite well, I just hadn't seen the feature before. I disagree with you a bit, though, in that many seemed pretty regular, and weren't really oddities, in my opinion.
sitkack 9y ago
Good thing CPython is the spec for the language. That said, many of these "oddities" are disingenuous to the point of lying, many of the error messages are omitted. Or the functionality converts to list before doing an operation. Not that it is good library or language design, but this is a great way to inflate bug counts.
- th 9y ago
  I think you may be taking the word oddity different than I meant it. These are things that trip up my students when they're learning Python. I left out error messages (and occasionally other things) because Twitter. I don't think any of the ones I put in this moment should be considered bugs. Many of these are core language features.
  - sitkack 9y ago
    Some of them seem like litmus test for a Python programmer interview, others feel like WTFs Python LoL! It is hard to disambiguate them. Standing on their own in that twitter feed, with little context, there is next to no enlightenment. Lots of the oddities could be explained by In [1]: list({'a':1, 'b':2, 'c':3}) Out[1]: ['a', 'b', 'c'] When people would expect In [2]: list({'a':1, 'b':2, 'c':3}.items()) Out[2]: [('a', 1), ('b', 2), ('c', 3)] Lots of Python makes sense. About 3% does not. It is good to know the 3% so it doesn't bite one in the ass. Maybe switch to image posts? 1 megapixel limit.
gpvos 9y ago
Aha, three ways to concatenate arrays! I see Python conforms to the TIMTOWTDI principle. :)
- sametmax 9y ago
  There should be one _obvious_ way to do it. The obvious way is using +, and is sufficient for 90% of programs.
Pxtl 9y ago
Am I the only one who hated the mobile pop-out slideshow Twitter uses for this? Also, the one about appending to arrays didn't seem too crazy.
coldtea 9y ago
There's no oddity in being able to extend an array in several ways (.extend, += other array, etc)
brilee 9y ago
Vast majority of these are not actually oddities. A few stuck out at me, though: - since True/False are aliased to 1/0, you cannot have both 1 and True as dictionary keys. This could plausibly happen in real code. - there's no distinction between args/kwargs, at least when it comes to a function's __defaults__ attr. An arg can be turned into a kwarg by messing with the __defaults__ attr. - you cannot have a nested tuple/dict/list with more than sys.MAX_RECURSION levels, because each list access apparently counts as a new call frame? (This could plausibly happen when generating a dict from json or html or xml)
- deleted 9y ago
  [deleted]
- masklinn 9y ago
  > there's no distinction between args/kwargs At the python level, all named args[0] are kwargs. The reverse is not true in Python 3, although it is in Python 2. At the C level, arguments can be exclusively one or the other even in Python 2. > at least when it comes to a function's __defaults__ attr. An arg can be turned into a kwarg by messing with the __defaults__ attr. No, an arg can be turned optional by manipulating __defaults__, that is a different (though not quite orthogonal) axis, especially in Python 3 (which supports python-level required keyword arguments). > - since True/False are aliased to 1/0, you cannot have both 1 and True as dictionary keys. They are not aliased (`True is 1` will return False) but they are equal and hash identically. [0] *args notwithstanding
rajathagasthya 9y ago
Here's one from https://stackoverflow.com/questions/15171695/whats-with-the-integer-cache-inside-python https://stackoverflow.com/questions/15171695/whats-with-the-...: In [1]: a = 1 In [2]: b = 1 In [3]: a is b Out[3]: True In [4]: a = 257 In [5]: b = 257 In [6]: a is b Out[6]: False
- raverbashing 9y ago
  Yes, because the smaller ints are 'pre-constructed' in Python (hence all 1's are the same "int(1)" object)
eesmith 9y ago
cpython oddity: The following program (call it "20.py") takes a minute to run, but the reported time for the "x=..." line is a few microseconds. import time if 1: t1 = time.time() x = ((((((((0,)*20,)*20,)*20,)*20,)*20,)*20,)*20,)*20 t2 = time.time() print(t2-t1) % /usr/bin/time python 20.py 3.09944152832e-06 70.91 real 70.91 user 0.24 sys
- dom0 9y ago
  That's probably ARC.
  - eesmith 9y ago
    ARC? The GIS language? Age Restricted Content? Advance Reading Copy? Activity-Regulated Cytoskeleton-associated protein?
    - dom0 9y ago
      Automatic Reference Counting, i.e. garbage collection. That's why the timer wouldn't attribute the CPU time to the line you mention; the GC cleans the structure up when the "x" goes out of scope, so after the timing has ended.
      - eesmith 9y ago
        Thanks. No, it's not due to reference counting, at least, not according to the tracker issue which reports this oddity. Add: import os os._exit(0) to the bottom of the code and it still takes a minute to run, even though there's no final garbage collection. Also, that code creates only a few hundred objects, and the 'x' data structure is acyclic.
marksomnian 9y ago
Reminds me of WTFJS: https://github.com/denysdovhan/wtfjs/blob/master/README.md https://github.com/denysdovhan/wtfjs/blob/master/README.md
seasonalgrit 9y ago
Why does the _MyClass__number example work the way it does?
- hroman 9y ago
  https://docs.python.org/3/tutorial/classes.html#private-variables https://docs.python.org/3/tutorial/classes.html#private-vari...
- masklinn 9y ago
  Within class bodies, symbols prefixed with two underscores get "mangled" (prefixed with the class name). This is intended to avoid risks of collisions for classes built specifically for inheritance so you'd usually use it with attributes e.g. `self.__name = name` (that way if a subclass defines its own "name" things will work as expected for both). I was unaware this also worked for straight local variables. I was unaware it happened for all symbols, but I guess it makes sense as the Python compiler does very little analysis.
sli 9y ago
>>> a = [1] >>> a += (2,) >>> a [1, 2] >>> b = (1,) >>> b += [2] TypeError It's an oddity that an immutable type cannot be mutated in place?
- th 9y ago
  This one is an odd one because initially most people I show this one to assume the oddity is that immutable types cannot be mutated in-place. But tuples can be mutated in-place: >>> b = (1,) >>> b += (2,) >>> b (1, 2) The odd thing is that lists can be in-place added (+='d) to any iterable, while tuples can only be in-place added to tuples. I think it works this way so that += on lists works consistently with the extend method on lists, which also accepts any iterable.
  - masklinn 9y ago
    > But tuples can be mutated in-place: >>> b = (1,) >>> b += (2,) >>> b (1, 2) That's not an in-place mutation any more than >>> a = 1 >>> a += 2 is. The oddity is that most people assume `a += b` is the same as `a = a + b`, but Python allows overloading both separately and list.__iadd__ is an alias for list.extend, hence being able to += any iterable onto a list whereas `list + non_list` generates a type error. The original intent is an optimisation (to avoid copying the subject list) but it's a broken one as it behaves very inconsistently with regular concatenation/addition: it mutates the subject in-place and allows a different (wider) set of parameters.

I’ve got one! First, some context: This is normal assignment unpacking: >>> a, b = 1, 2 >>> [a, b] [1, 2] (Mismatch gives errors) >>> a, b = [1] ValueError >>> a, b = [1, 2, 3] ValueError Extra parentheses works too: >>> (a, b) = [2, 3] >>> [a, b] [2, 3] Unpacking a single value: >>> (a,) = [4] >>> a 4 Works without parentheses too: >>> a, = [5] >>> a 5 No values does not work: >>> , = [] SyntaxError Neither does empty parentheses: >>> () = [] SyntaxError Now, the oddity: >>> [] = [] >>> This raises no error! Neither does this: >>> [] = () >>>

makecheck 9y ago
This isn't doing what you think though. There are container objects underneath. For instance "(a, b)" is "a tuple object containing two things, the values named "a" and "b" so you can still refer to them separately. "[a, b]" is a new list with those same elements, not an alias for "(a, b)".
- masklinn 9y ago
  The oddity is that unpacking an empty iterable works when the target is a "list", but not when it's a "tuple". In fact it so doesn't work it actually fails during parsing. And there are no collections underneath the LHS, it compiles to an UNPACK_SEQUENCE opcode in both cases: >>> @dis.dis ... def foo(): ... (a, b) = bar() ... 3 0 LOAD_GLOBAL 0 (bar) 3 CALL_FUNCTION 0 (0 positional, 0 keyword pair) 6 UNPACK_SEQUENCE 2 9 STORE_FAST 0 (a) 12 STORE_FAST 1 (b) 15 LOAD_CONST 0 (None) 18 RETURN_VALUE >>> @dis.dis ... def foo(): ... [a, b] = bar() ... 3 0 LOAD_GLOBAL 0 (bar) 3 CALL_FUNCTION 0 (0 positional, 0 keyword pair) 6 UNPACK_SEQUENCE 2 9 STORE_FAST 0 (a) 12 STORE_FAST 1 (b) 15 LOAD_CONST 0 (None) 18 RETURN_VALUE >>> @dis.dis ... def foo(): ... [] = bar() ... 3 0 LOAD_GLOBAL 0 (bar) 3 CALL_FUNCTION 0 (0 positional, 0 keyword pair) 6 UNPACK_SEQUENCE 0 9 LOAD_CONST 0 (None) 12 RETURN_VALUE
  - makecheck 9y ago
    It’s a good point that Python is smart enough not to build a mutable list on the right-hand side when not needed (e.g. for "x = [1, 2]" it will and for "y, z = [1, 2]" it will not). A right-hand tuple however is still a separate type, and if you disassemble the original cases you can see that "[] = []" is more wasteful than "[] = ()". >>> import dis >>> >>> @dis.dis ... def f(): ... a, b = 1, 2 ... (a, b) = [2, 3] ... (a,) = [4] ... a, = [5] ... [] = [] ... [] = () ... 3 0 LOAD_CONST 6 ((1, 2)) 3 UNPACK_SEQUENCE 2 6 STORE_FAST 0 (a) 9 STORE_FAST 1 (b) 4 12 LOAD_CONST 2 (2) 15 LOAD_CONST 3 (3) 18 ROT_TWO 19 STORE_FAST 0 (a) 22 STORE_FAST 1 (b) 5 25 LOAD_CONST 4 (4) 28 STORE_FAST 0 (a) 6 31 LOAD_CONST 5 (5) 34 STORE_FAST 0 (a) 7 37 BUILD_LIST 0 40 UNPACK_SEQUENCE 0 8 43 LOAD_CONST 7 (()) 46 UNPACK_SEQUENCE 0 49 LOAD_CONST 0 (None) 52 RETURN_VALUE
StavrosK 9y ago
Hmm, why is that an oddity? Unpacking a list of N items into N variables works fine, where N >= 0.
- masklinn 9y ago
  It works when you use a "list literal" as target, but with a "tuple literal" it only works for N > 0.
jbg_ 9y ago
This appears to no longer be the case in Python 3: $ python2 Python 2.7.13 (default, Jul 21 2017, 03:24:34) >>> () = [] File "<stdin>", line 1 SyntaxError: can't assign to () >>> $ python3 Python 3.6.2 (default, Jul 20 2017, 03:52:27) >>> () = [] >>>
- masklinn 9y ago
  > This appears to no longer be the case in Python 3: It was fixed in Python 3.6 specifically, the error still occurs under 3.5.

dom0 9y ago

Some of these are pretty simple and not odd, e.g. immutability of tuples, iterator behaviour or modifying collections during iteration. ————————————————————————————————————— Somewhat related video: https://www.youtube.com/watch?v=qCGofLIzX6g https://www.youtube.com/watch?v=qCGofLIzX6g

wyclif 9y ago

Best use of Twitter Moments I've ever seen.

StavrosK 9y ago
First use of Twitter Moments I've ever seen. What is this? A collection of tweets? I'm confused because they all seem to be by the author, did he tweet them himself and collected them? The page's UI looks weird, I'm not sure if Twitter had a redesign or if I'm blocking some of their CSS.
- th 9y ago
  I think they're pretty much a collection of tweets. This was the first time I used moments and I'm not sure where I saw them used before I made this. They're mostly by me because I made a goal for myself to tweet out one of these every week for a year. Others have shared their own and I've tweeted my own since then, but Twitter's hashtag search is pretty bad so I made a moment.
  - StavrosK 9y ago
    I think it works quite well, I just hadn't seen the feature before. I disagree with you a bit, though, in that many seemed pretty regular, and weren't really oddities, in my opinion.

sitkack 9y ago

Good thing CPython is the spec for the language. That said, many of these "oddities" are disingenuous to the point of lying, many of the error messages are omitted. Or the functionality converts to list before doing an operation. Not that it is good library or language design, but this is a great way to inflate bug counts.

th 9y ago
I think you may be taking the word oddity different than I meant it. These are things that trip up my students when they're learning Python. I left out error messages (and occasionally other things) because Twitter. I don't think any of the ones I put in this moment should be considered bugs. Many of these are core language features.
- sitkack 9y ago
  Some of them seem like litmus test for a Python programmer interview, others feel like WTFs Python LoL! It is hard to disambiguate them. Standing on their own in that twitter feed, with little context, there is next to no enlightenment. Lots of the oddities could be explained by In [1]: list({'a':1, 'b':2, 'c':3}) Out[1]: ['a', 'b', 'c'] When people would expect In [2]: list({'a':1, 'b':2, 'c':3}.items()) Out[2]: [('a', 1), ('b', 2), ('c', 3)] Lots of Python makes sense. About 3% does not. It is good to know the 3% so it doesn't bite one in the ass. Maybe switch to image posts? 1 megapixel limit.

gpvos 9y ago

Aha, three ways to concatenate arrays! I see Python conforms to the TIMTOWTDI principle. :)

sametmax 9y ago
There should be one _obvious_ way to do it. The obvious way is using +, and is sufficient for 90% of programs.

Pxtl 9y ago

Am I the only one who hated the mobile pop-out slideshow Twitter uses for this? Also, the one about appending to arrays didn't seem too crazy.

coldtea 9y ago

There's no oddity in being able to extend an array in several ways (.extend, += other array, etc)

brilee 9y ago

Vast majority of these are not actually oddities. A few stuck out at me, though: - since True/False are aliased to 1/0, you cannot have both 1 and True as dictionary keys. This could plausibly happen in real code. - there's no distinction between args/kwargs, at least when it comes to a function's __defaults__ attr. An arg can be turned into a kwarg by messing with the __defaults__ attr. - you cannot have a nested tuple/dict/list with more than sys.MAX_RECURSION levels, because each list access apparently counts as a new call frame? (This could plausibly happen when generating a dict from json or html or xml)

deleted 9y ago
[deleted]
masklinn 9y ago
> there's no distinction between args/kwargs At the python level, all named args[0] are kwargs. The reverse is not true in Python 3, although it is in Python 2. At the C level, arguments can be exclusively one or the other even in Python 2. > at least when it comes to a function's __defaults__ attr. An arg can be turned into a kwarg by messing with the __defaults__ attr. No, an arg can be turned optional by manipulating __defaults__, that is a different (though not quite orthogonal) axis, especially in Python 3 (which supports python-level required keyword arguments). > - since True/False are aliased to 1/0, you cannot have both 1 and True as dictionary keys. They are not aliased (`True is 1` will return False) but they are equal and hash identically. [0] *args notwithstanding

rajathagasthya 9y ago

Here's one from https://stackoverflow.com/questions/15171695/whats-with-the-integer-cache-inside-python https://stackoverflow.com/questions/15171695/whats-with-the-...: In [1]: a = 1 In [2]: b = 1 In [3]: a is b Out[3]: True In [4]: a = 257 In [5]: b = 257 In [6]: a is b Out[6]: False

raverbashing 9y ago
Yes, because the smaller ints are 'pre-constructed' in Python (hence all 1's are the same "int(1)" object)

eesmith 9y ago

cpython oddity: The following program (call it "20.py") takes a minute to run, but the reported time for the "x=..." line is a few microseconds. import time if 1: t1 = time.time() x = ((((((((0,)*20,)*20,)*20,)*20,)*20,)*20,)*20,)*20 t2 = time.time() print(t2-t1) % /usr/bin/time python 20.py 3.09944152832e-06 70.91 real 70.91 user 0.24 sys

dom0 9y ago
That's probably ARC.
- eesmith 9y ago
  ARC? The GIS language? Age Restricted Content? Advance Reading Copy? Activity-Regulated Cytoskeleton-associated protein?
  - dom0 9y ago
    Automatic Reference Counting, i.e. garbage collection. That's why the timer wouldn't attribute the CPU time to the line you mention; the GC cleans the structure up when the "x" goes out of scope, so after the timing has ended.
    - eesmith 9y ago
      Thanks. No, it's not due to reference counting, at least, not according to the tracker issue which reports this oddity. Add: import os os._exit(0) to the bottom of the code and it still takes a minute to run, even though there's no final garbage collection. Also, that code creates only a few hundred objects, and the 'x' data structure is acyclic.

marksomnian 9y ago

Reminds me of WTFJS: https://github.com/denysdovhan/wtfjs/blob/master/README.md https://github.com/denysdovhan/wtfjs/blob/master/README.md

seasonalgrit 9y ago

Why does the _MyClass__number example work the way it does?

hroman 9y ago
https://docs.python.org/3/tutorial/classes.html#private-variables https://docs.python.org/3/tutorial/classes.html#private-vari...
masklinn 9y ago
Within class bodies, symbols prefixed with two underscores get "mangled" (prefixed with the class name). This is intended to avoid risks of collisions for classes built specifically for inheritance so you'd usually use it with attributes e.g. `self.__name = name` (that way if a subclass defines its own "name" things will work as expected for both). I was unaware this also worked for straight local variables. I was unaware it happened for all symbols, but I guess it makes sense as the Python compiler does very little analysis.

sli 9y ago

>>> a = [1] >>> a += (2,) >>> a [1, 2] >>> b = (1,) >>> b += [2] TypeError It's an oddity that an immutable type cannot be mutated in place?

th 9y ago
This one is an odd one because initially most people I show this one to assume the oddity is that immutable types cannot be mutated in-place. But tuples can be mutated in-place: >>> b = (1,) >>> b += (2,) >>> b (1, 2) The odd thing is that lists can be in-place added (+='d) to any iterable, while tuples can only be in-place added to tuples. I think it works this way so that += on lists works consistently with the extend method on lists, which also accepts any iterable.
- masklinn 9y ago
  > But tuples can be mutated in-place: >>> b = (1,) >>> b += (2,) >>> b (1, 2) That's not an in-place mutation any more than >>> a = 1 >>> a += 2 is. The oddity is that most people assume `a += b` is the same as `a = a + b`, but Python allows overloading both separately and list.__iadd__ is an alias for list.extend, hence being able to += any iterable onto a list whereas `list + non_list` generates a type error. The original intent is an optimisation (to avoid copying the subject list) but it's a broken one as it behaves very inconsistently with regular concatenation/addition: it mutates the subject in-place and allows a different (wider) set of parameters.