9 ms·

Python Lists vs. Tuples

79 points by joshbaptiste 10y ago

dozzie 10y ago
> The Cultural Difference is about how lists and tuples are actually used: lists are used where you have a homogenous sequence of unknown length; tuples are used where you know the number of elements in advance because the position of the element is semantically significant. Oh, if only it were so simple! Python itself doesn't understand what the heck tuples are, in that the built-in functions often treat tuples as simply "immutable lists". See isinstance() and issubclass() functions, which expect what's meant to be a list of classes, but you need to pass a tuple. *args is another such example, as the article mentions. If what Python calls tuples were the thing with structure instead of order, they wouldn't be iterable.
ferrari8608 10y ago
It's much simpler to me. Tuples are used when you need a sequence but don't need to modify the contents, for instance returning multiple objects from a function call. Lists are used when you want to add things to it. Namedtuples are wonderful! It's like a class with only attributes, except you don't have to define a new class and get slapped across the back of your head by more experienced programmers for creating a useless class. Access objects by index, by name, or by iteration. It's very versatile.
- pablobaz 10y ago
  If you like namedtuples, you'll love attrs. https://pypi.python.org/pypi/attrs/16.0.0 https://pypi.python.org/pypi/attrs/16.0.0
  - rch 10y ago
    I like namedtuples, but attrs looks so wrong to me.
- shoo 10y ago
  namedtuple is a reasonable way to define a new value type in python. unfortunately the very easy way to define a new type in python is to use `class`, which brings with it a bunch of behaviour that you often don't want or need: comparison by identity rather than comparison by equality, no default support for repr, no default support for ordering, mutability, support for inheritance, etc that said, writing generic code that works transparently with both namedtuple values and objects can be a bit irritating: e.g. you might want to derive a new value/object from an existing value/object that updates one of the attributes of the value/object.
  - sago 10y ago
    First para: I think maybe you're confusing 'class' with inheritance from 'object'. A good tip is to do: class Foo(namedtuple('Foo', (...)): def __repr__(self): ... Your 'Foo' gets ordering, hashing, comparison, because it derives from 'namedtuple'. It gets '__repr__' too, but in the example I'm overriding that.
    - throwaway274739 10y ago
      I would also add __slots__ = () to your class definition so that instances of Foo keep the small memory footprint benefits do namedtuple.
pmiller2 10y ago
I was asked this in an interview. This is essentially the answer I gave. :)
justanotherbody 10y ago
The explanation is alright, but would have really benefited from a brief overview of tuple use in the C API.
- nedbat 10y ago
  The C API seems like a completely separate topic.
  - justanotherbody 10y ago
    C and Python APIs of CPython are really just difference faces of the same thing. Due to the internal use of tuples some of the Python behavior is governed by related C behavior. For example: > Another conflict between the Technical and the Cultural: there are places in Python itself where a tuple is used when a list makes more sense. When you define a function with args, args is passed to you as a tuple, even though the position of the values isn't significant, at least as far as Python knows This is the C behavior of tuple usage being exposed to the Python side. If one explores the C API it becomes clear that an implementation detail of C is being exposed to Python. > The Cultural Difference is about how lists and tuples are actually used: lists are used where you have a homogenous sequence of unknown length; tuples are used where you know the number of elements in advance because the position of the element is semantically significant. Here the Culture has moved away from the Technical roots of Python and now they're at odds. On the C side tuples are not* just used when the length is known ahead of time - tuples are not always immutable on the C side. The context is important if you really want to understand.
pixelmonkey 10y ago
Great article! One more important difference between tuple and list that's worth knowing about: syntax. Ned makes it seem like tuples have the syntax "(x, y)", but they actually have the syntax "x, y". I find that a lot of Python programmers, even long-time ones, don't realize this important difference. It's what allows one to write code constructions like "x, y = 1, 2" or "return 1, 2". One needs the parentheses only to "clarify" this syntax in certain cases; for example, as dict keys, the parens are necessary -- "x = {(1, 2): 2}". Although interestingly, for dict lookup, they are not -- "x[1, 2] == 2". One way I remind myself of this rule is to realize that "x = 1," is the same as "x = (1,)", but "x = (1)" is actually the same as "x = 1". This is quite different from lists, who are syntactically defined by the "[]" brackets, and thus you require them, or a "list()" call, whenever you make lists, making it just that much more heavyweight as a data structure.
- thatswrong0 10y ago
  When I used to write python "x = 1," bit me way more than it should have. And I don't see why you wouldn't want to be explicit about your tuple construction every time..
  - thansharp 10y ago
    I usually use x = tuple((1,) ) so that non-Python programmers on the team know what they're dealing with.
jackweirdy 10y ago
There is an additional technical difference: Tuples are hashable and can be used as keys in a dictionary. grid[(1,1)] = True
- vbit 10y ago
  The article mentions this. Also you can write this as `grid[1, 1] = True`
euske 10y ago
When I taught Python, the way I explained was that a tuple is basically one value, shrinkwrapped. It's one value so that you can put anywhere that expects a value (such as a return value or a key of a dictionary). Also it's one value so you cannot change it. The way you look at a statement like (a,b) = (2,3) is that this is a normal assignment, where you imagine there are one value flowing in the equal sign, from right to left. But it happens to contain two values within its wrap.
colanderman 10y ago
So, I've long since argued the importance of what Ned terms the "cultural difference", or the difference between dynamic homogeneous structures (lists/relational columns), and static heterogeneous structures (tuples/relational rows). There are important performance ramifications regarding the use of one or the other, and conflating the two (i.e. dynamic heterogeneous) is a strong design smell. Some languages "get it", others don't. Python is not a language that "gets it", despite the article's examples. Python chooses the much-less-useful distinction between "immutable sequence" and "mutable sequence"; the lack of any other distinction is borne out by the standard library (as dozzie points out here [1]). Furthermore, the issue of immutable vs. mutable is totally orthogonal to that of dynamic/static and homogeneous/heterogeneous. There's no reason that "cultural" lists need be mutable, or "cultural" tuples need be immutable. So Python's choice doesn't make sense from this angle either. For example, OCaml has both immutable lists and immutable tuples; its type system enforces exactly the "cultural difference". Similarly with C, which has both mutable lists (arrays) and mutable tuples (structs), both of which can be rendered immutable (const). Why Python chose lists (and only lists!) to have both immutable and mutable flavors, with otherwise identical semantics, is beyond me. Erlang used to "get it", but they lost the thread when they came out with heterogeneous syntax support for a homogeneous structure (map). TLA+ totally conflates tuples with lists, but its type system is rich enough to enforce the difference (and as a bonus, allows for the elusive static homogeneous structure). [1] https://news.ycombinator.com/item?id=12323849 https://news.ycombinator.com/item?id=12323849
- nedbat 10y ago
  Can you say more about the important performance ramifications?
  - colanderman 10y ago
    Dynamic homogeneous structures (lists) have uniform structure, but must be able to resize dynamically. The former helps performance (run-time type info can be hoisted out of the structure), the latter harms it (extra runtime machinery needed). Static heterogeneous structures (tuples) have a non-uniform structure, but need not resize. The former harms performance (run-time type info must be kept for each element, though usually few); the latter helps it (most accesses can be compiled to simple pointer arithmetic). On the other hand, dynamic heterogeneous structures (Python's lists) do not have uniform structure, and must be able to resize. So run-time type info must be preserved (adding memory/access overhead), and accesses can't be optimized at compile time (more runtime machinery). It is to avoid these issues that things like numpy and JavaScript's typed arrays exist. There is almost never a reason to use a dynamic heterogeneous structure, so they are best avoided. A dynamic heterogeneous collection can neither support indexing a priori known components (since it is dynamic), nor support iteration (since it is heterogeneous). (Note that a collection of objects of different types but with a shared interface is in fact a homogeneous collection of objects with that interface.) (The final combination is static homogeneous, but these are a subset of both the first two cases, and do not occur often. The only example I can think of are sets of statistics: all statistics have the same type, but there are a fixed number of them which are known at compile time. They can be optimized very well.)
ratsbane 10y ago
Interesting article. I've always heard that about lists and tuples, but this article made me wonder how much less efficient with storage is a list. The following trivial example shows that, at least for very small lists and tuples, except for the fixed overhead of 16 bytes for a tuple vs. 72 bytes for a list they're the same [edit: no it doesn't. I wasn't looking at the diff between a 1-element and 2-element tuple]: Perhaps there's more to it? >>> from sys import getsizeof >>> list = [94] >>> getsizeof(list) 80 >>> list = [94, 37] >>> getsizeof(list) 88 >>> list = [94, 37, 21] >>> getsizeof(list) 96 >>> list = [94, 37, 21, 19] >>> getsizeof(list) 104 >>> tuple = (94) >>> getsizeof(tuple) 24 >>> tuple = (94, 37) >>> getsizeof(tuple) 72 >>> tuple = (94, 37, 21) >>> getsizeof(tuple) 80 >>> tuple = (94, 37, 21, 19) >>> getsizeof(tuple) 88
- colanderman 10y ago
  (94) is not a tuple. (94,) is.
  - ratsbane 10y ago
    D'oh! Thank you, colanderman. >>> tuple = (94,) >>> getsizeof(tuple) 64
- ratsbane 10y ago
  Another (admittedly trivial) experiment with speed and storage: Iterating through a 10,000 element list was about 2% slower than iterating through a 10,000 element tuple, and the list was 16 bytes bigger than the tuple: #!/usr/bin/python from sys import getsizeof from random import sample from time import time l = sample(xrange(10000), 10000) t = tuple(l) print "The tuple's size is " + str(getsizeof(t)) + " and the list's size is " + str(getsizeof(l)) s = time() z = 0 for x in l: z += x print "Iterating through a 10,000 element list took " + str(time() - s) + " seconds." s = time() z = 0 for x in t: z += x print "Iterating through a 10,000 element tuple took " + str(time() - s) + " seconds." doug@supermicro:~$ ./x The tuple's size is 80056 and the list's size is 80072 Iterating through a 10,000 element list took 0.00589299201965 seconds. Iterating through a 10,000 element tuple took 0.00538778305054 seconds. doug@supermicro:~$ ./x The tuple's size is 80056 and the list's size is 80072 Iterating through a 10,000 element list took 0.00553607940674 seconds. Iterating through a 10,000 element tuple took 0.00536203384399 seconds.
ScottBurson 10y ago
It's really too bad that functional data structures weren't much of a thing yet when Guido started developing Python. The language really cries out for them -- particularly with the "mutable initial value" problem and the 'set'/'frozenset' distinction.
- jholman 10y ago
  Can you elaborate on how either of those, uh, "features" would be helped by functional "persistent" data structures? In the case of the mutable initial value problem, that's just a choice of semantics (in my opinion, a very bad choice). Without having looked at the implementation, I would assume that if you have no backwards-compatibility concerns, a fix would take a few minutes. In the case of sets/frozensets, most Python programmers want their sets to be mutable, in a syntax-sugar-y way that's incompatible with using functional persistent data structures.
  - ScottBurson 10y ago
    Isn't it obvious? If the initial value is an empty functional list, set, or map (dict), there's no possibility of it being modified. The problem disappears. > most Python programmers want their sets to be mutable They're accustomed to mutable sets because those are all they've known. But ask a Clojure programmer how they feel about functional collections, and you'll be told they're fine. Functional collections are actually easier to use because you don't have to worry about unintended aliasing. And functional behavior is not strange at all. No programming language I have ever used (and it's a long list) has used shared, mutable semantics for numbers. Consider, in C: int a = 3; int b = a; b++; After this code sequence, 'a' is still 3, of course! Nobody with programming experience expects 'a' to be incremented at the same time 'b' is; it's not even something you stop to think about -- it's completely automatic. This is true even though I wrote 'b++' rather than 'b = b + 1' -- 'b++' could more easily be misread as a mutating operation on a shared integer object, like 's.add(x)'. However, there's no language I'm aware of in which that's what it means. (You can introduce aliasing intentionally in C++ with a reference, but this requires an explicit '&' in the code.)

79 points by joshbaptiste 10y ago

> The Cultural Difference is about how lists and tuples are actually used: lists are used where you have a homogenous sequence of unknown length; tuples are used where you know the number of elements in advance because the position of the element is semantically significant. Oh, if only it were so simple! Python itself doesn't understand what the heck tuples are, in that the built-in functions often treat tuples as simply "immutable lists". See isinstance() and issubclass() functions, which expect what's meant to be a list of classes, but you need to pass a tuple. *args is another such example, as the article mentions. If what Python calls tuples were the thing with structure instead of order, they wouldn't be iterable.

ferrari8608 10y ago

It's much simpler to me. Tuples are used when you need a sequence but don't need to modify the contents, for instance returning multiple objects from a function call. Lists are used when you want to add things to it. Namedtuples are wonderful! It's like a class with only attributes, except you don't have to define a new class and get slapped across the back of your head by more experienced programmers for creating a useless class. Access objects by index, by name, or by iteration. It's very versatile.

pablobaz 10y ago
If you like namedtuples, you'll love attrs. https://pypi.python.org/pypi/attrs/16.0.0 https://pypi.python.org/pypi/attrs/16.0.0
- rch 10y ago
  I like namedtuples, but attrs looks so wrong to me.
shoo 10y ago
namedtuple is a reasonable way to define a new value type in python. unfortunately the very easy way to define a new type in python is to use `class`, which brings with it a bunch of behaviour that you often don't want or need: comparison by identity rather than comparison by equality, no default support for repr, no default support for ordering, mutability, support for inheritance, etc that said, writing generic code that works transparently with both namedtuple values and objects can be a bit irritating: e.g. you might want to derive a new value/object from an existing value/object that updates one of the attributes of the value/object.
- sago 10y ago
  First para: I think maybe you're confusing 'class' with inheritance from 'object'. A good tip is to do: class Foo(namedtuple('Foo', (...)): def __repr__(self): ... Your 'Foo' gets ordering, hashing, comparison, because it derives from 'namedtuple'. It gets '__repr__' too, but in the example I'm overriding that.
  - throwaway274739 10y ago
    I would also add __slots__ = () to your class definition so that instances of Foo keep the small memory footprint benefits do namedtuple.

pmiller2 10y ago

I was asked this in an interview. This is essentially the answer I gave. :)

justanotherbody 10y ago

The explanation is alright, but would have really benefited from a brief overview of tuple use in the C API.

nedbat 10y ago
The C API seems like a completely separate topic.
- justanotherbody 10y ago
  C and Python APIs of CPython are really just difference faces of the same thing. Due to the internal use of tuples some of the Python behavior is governed by related C behavior. For example: > Another conflict between the Technical and the Cultural: there are places in Python itself where a tuple is used when a list makes more sense. When you define a function with args, args is passed to you as a tuple, even though the position of the values isn't significant, at least as far as Python knows This is the C behavior of tuple usage being exposed to the Python side. If one explores the C API it becomes clear that an implementation detail of C is being exposed to Python. > The Cultural Difference is about how lists and tuples are actually used: lists are used where you have a homogenous sequence of unknown length; tuples are used where you know the number of elements in advance because the position of the element is semantically significant. Here the Culture has moved away from the Technical roots of Python and now they're at odds. On the C side tuples are not* just used when the length is known ahead of time - tuples are not always immutable on the C side. The context is important if you really want to understand.

pixelmonkey 10y ago

Great article! One more important difference between tuple and list that's worth knowing about: syntax. Ned makes it seem like tuples have the syntax "(x, y)", but they actually have the syntax "x, y". I find that a lot of Python programmers, even long-time ones, don't realize this important difference. It's what allows one to write code constructions like "x, y = 1, 2" or "return 1, 2". One needs the parentheses only to "clarify" this syntax in certain cases; for example, as dict keys, the parens are necessary -- "x = {(1, 2): 2}". Although interestingly, for dict lookup, they are not -- "x[1, 2] == 2". One way I remind myself of this rule is to realize that "x = 1," is the same as "x = (1,)", but "x = (1)" is actually the same as "x = 1". This is quite different from lists, who are syntactically defined by the "[]" brackets, and thus you require them, or a "list()" call, whenever you make lists, making it just that much more heavyweight as a data structure.

thatswrong0 10y ago
When I used to write python "x = 1," bit me way more than it should have. And I don't see why you wouldn't want to be explicit about your tuple construction every time..
- thansharp 10y ago
  I usually use x = tuple((1,) ) so that non-Python programmers on the team know what they're dealing with.

jackweirdy 10y ago

There is an additional technical difference: Tuples are hashable and can be used as keys in a dictionary. grid[(1,1)] = True

vbit 10y ago
The article mentions this. Also you can write this as `grid[1, 1] = True`

euske 10y ago

When I taught Python, the way I explained was that a tuple is basically one value, shrinkwrapped. It's one value so that you can put anywhere that expects a value (such as a return value or a key of a dictionary). Also it's one value so you cannot change it. The way you look at a statement like (a,b) = (2,3) is that this is a normal assignment, where you imagine there are one value flowing in the equal sign, from right to left. But it happens to contain two values within its wrap.

colanderman 10y ago

So, I've long since argued the importance of what Ned terms the "cultural difference", or the difference between dynamic homogeneous structures (lists/relational columns), and static heterogeneous structures (tuples/relational rows). There are important performance ramifications regarding the use of one or the other, and conflating the two (i.e. dynamic heterogeneous) is a strong design smell. Some languages "get it", others don't. Python is not a language that "gets it", despite the article's examples. Python chooses the much-less-useful distinction between "immutable sequence" and "mutable sequence"; the lack of any other distinction is borne out by the standard library (as dozzie points out here [1]). Furthermore, the issue of immutable vs. mutable is totally orthogonal to that of dynamic/static and homogeneous/heterogeneous. There's no reason that "cultural" lists need be mutable, or "cultural" tuples need be immutable. So Python's choice doesn't make sense from this angle either. For example, OCaml has both immutable lists and immutable tuples; its type system enforces exactly the "cultural difference". Similarly with C, which has both mutable lists (arrays) and mutable tuples (structs), both of which can be rendered immutable (const). Why Python chose lists (and only lists!) to have both immutable and mutable flavors, with otherwise identical semantics, is beyond me. Erlang used to "get it", but they lost the thread when they came out with heterogeneous syntax support for a homogeneous structure (map). TLA+ totally conflates tuples with lists, but its type system is rich enough to enforce the difference (and as a bonus, allows for the elusive static homogeneous structure). [1] https://news.ycombinator.com/item?id=12323849 https://news.ycombinator.com/item?id=12323849

nedbat 10y ago
Can you say more about the important performance ramifications?
- colanderman 10y ago
  Dynamic homogeneous structures (lists) have uniform structure, but must be able to resize dynamically. The former helps performance (run-time type info can be hoisted out of the structure), the latter harms it (extra runtime machinery needed). Static heterogeneous structures (tuples) have a non-uniform structure, but need not resize. The former harms performance (run-time type info must be kept for each element, though usually few); the latter helps it (most accesses can be compiled to simple pointer arithmetic). On the other hand, dynamic heterogeneous structures (Python's lists) do not have uniform structure, and must be able to resize. So run-time type info must be preserved (adding memory/access overhead), and accesses can't be optimized at compile time (more runtime machinery). It is to avoid these issues that things like numpy and JavaScript's typed arrays exist. There is almost never a reason to use a dynamic heterogeneous structure, so they are best avoided. A dynamic heterogeneous collection can neither support indexing a priori known components (since it is dynamic), nor support iteration (since it is heterogeneous). (Note that a collection of objects of different types but with a shared interface is in fact a homogeneous collection of objects with that interface.) (The final combination is static homogeneous, but these are a subset of both the first two cases, and do not occur often. The only example I can think of are sets of statistics: all statistics have the same type, but there are a fixed number of them which are known at compile time. They can be optimized very well.)

ratsbane 10y ago

Interesting article. I've always heard that about lists and tuples, but this article made me wonder how much less efficient with storage is a list. The following trivial example shows that, at least for very small lists and tuples, except for the fixed overhead of 16 bytes for a tuple vs. 72 bytes for a list they're the same [edit: no it doesn't. I wasn't looking at the diff between a 1-element and 2-element tuple]: Perhaps there's more to it? >>> from sys import getsizeof >>> list = [94] >>> getsizeof(list) 80 >>> list = [94, 37] >>> getsizeof(list) 88 >>> list = [94, 37, 21] >>> getsizeof(list) 96 >>> list = [94, 37, 21, 19] >>> getsizeof(list) 104 >>> tuple = (94) >>> getsizeof(tuple) 24 >>> tuple = (94, 37) >>> getsizeof(tuple) 72 >>> tuple = (94, 37, 21) >>> getsizeof(tuple) 80 >>> tuple = (94, 37, 21, 19) >>> getsizeof(tuple) 88

colanderman 10y ago
(94) is not a tuple. (94,) is.
- ratsbane 10y ago
  D'oh! Thank you, colanderman. >>> tuple = (94,) >>> getsizeof(tuple) 64
ratsbane 10y ago
Another (admittedly trivial) experiment with speed and storage: Iterating through a 10,000 element list was about 2% slower than iterating through a 10,000 element tuple, and the list was 16 bytes bigger than the tuple: #!/usr/bin/python from sys import getsizeof from random import sample from time import time l = sample(xrange(10000), 10000) t = tuple(l) print "The tuple's size is " + str(getsizeof(t)) + " and the list's size is " + str(getsizeof(l)) s = time() z = 0 for x in l: z += x print "Iterating through a 10,000 element list took " + str(time() - s) + " seconds." s = time() z = 0 for x in t: z += x print "Iterating through a 10,000 element tuple took " + str(time() - s) + " seconds." doug@supermicro:~$ ./x The tuple's size is 80056 and the list's size is 80072 Iterating through a 10,000 element list took 0.00589299201965 seconds. Iterating through a 10,000 element tuple took 0.00538778305054 seconds. doug@supermicro:~$ ./x The tuple's size is 80056 and the list's size is 80072 Iterating through a 10,000 element list took 0.00553607940674 seconds. Iterating through a 10,000 element tuple took 0.00536203384399 seconds.

ScottBurson 10y ago

It's really too bad that functional data structures weren't much of a thing yet when Guido started developing Python. The language really cries out for them -- particularly with the "mutable initial value" problem and the 'set'/'frozenset' distinction.

jholman 10y ago
Can you elaborate on how either of those, uh, "features" would be helped by functional "persistent" data structures? In the case of the mutable initial value problem, that's just a choice of semantics (in my opinion, a very bad choice). Without having looked at the implementation, I would assume that if you have no backwards-compatibility concerns, a fix would take a few minutes. In the case of sets/frozensets, most Python programmers want their sets to be mutable, in a syntax-sugar-y way that's incompatible with using functional persistent data structures.
- ScottBurson 10y ago
  Isn't it obvious? If the initial value is an empty functional list, set, or map (dict), there's no possibility of it being modified. The problem disappears. > most Python programmers want their sets to be mutable They're accustomed to mutable sets because those are all they've known. But ask a Clojure programmer how they feel about functional collections, and you'll be told they're fine. Functional collections are actually easier to use because you don't have to worry about unintended aliasing. And functional behavior is not strange at all. No programming language I have ever used (and it's a long list) has used shared, mutable semantics for numbers. Consider, in C: int a = 3; int b = a; b++; After this code sequence, 'a' is still 3, of course! Nobody with programming experience expects 'a' to be incremented at the same time 'b' is; it's not even something you stop to think about -- it's completely automatic. This is true even though I wrote 'b++' rather than 'b = b + 1' -- 'b++' could more easily be misread as a mutating operation on a shared integer object, like 's.add(x)'. However, there's no language I'm aware of in which that's what it means. (You can introduce aliasing intentionally in C++ with a reference, but this requires an explicit '&' in the code.)