10 ms·

A Guide to Python's Magic Methods

152 points by yingxu 13y ago

CraigJPerry 13y ago
I'm bookmarking this, this is an awesome reference. Since this article was written, __cmp__ is has now been deprecated. The article already mentions total_ordering and you can use that instead.
hugorodgerbrown 13y ago
> The use case for these magic methods might seem small, and that may very well be true. I won't spend too much more time on reflection magic methods because they aren't very important If you want a reason to use the instancecheck magic method, the mocking of test objects is a good one. I recently came unstuck with testing Django models and mocking out DateFields, and the instancecheck method was the solution - http://tech.yunojuno.com/mocking-dates-with-django http://tech.yunojuno.com/mocking-dates-with-django
JelteF 13y ago
Why is this not just in the regular python docs.
- aet 13y ago
  You may find some stuff here: http://docs.python.org/3.3/reference/datamodel.html http://docs.python.org/3.3/reference/datamodel.html
- vanni 13y ago
  It is: http://stackoverflow.com/a/18392248/366332 http://stackoverflow.com/a/18392248/366332
- ketralnis 13y ago
  From the very first section of the article ("Introduction"): > They're also not as well documented as they need to be. All of the magic methods for Python appear in the same section in the Python docs, but they're scattered about and only loosely organized. There's hardly an example to be found in that section (and that may very well be by design, since they're all detailed in the language reference, along with boring syntax descriptions, etc.). > So, to fix what I perceived as a flaw in Python's documentation, I set out to provide some more plain-English, example-driven documentation for Python's magic methods
nichochar 13y ago
Great article, definitely bookmarking this. I think it's important to give a reminder somewhere that modifying the magic methods really should be done with care. People new to Python could use this in a brute force way, when logic could do the trick. Working for example with a framework like web2py, that re-implements a bunch of these, makes development slow and laborious sometimes.
jessaustin 13y ago
This would be better if abstract base classes were demonstrated. It's better to build a sequence class with collections.abc.Sequence, for instance, to avoid all that boilerplate.
agf 13y ago
This is a great, thorough overview. To me, the main omission is next / __next__. It was misnamed in Python 2 (corrected to be a "magic method" in Python 3) but belongs on the list nonetheless. It is the method of an iterator that returns the next item, and is core to (almost) any iterable object in Python (except sequences that can be sequentially indexed from zero).
- dirkk0 13y ago
  The author states that this article is on github and can be improved there.
Too 13y ago
Some comments on the first example. 1. Closing a file by relying on the, nondeterministic, GC seems like a cover-up solution for sloppy programming. I've seen quite a few C# code bases where people even, instead of closing, put the equivalent of assert(self.file.closed) inside the destructor to get an early warning when someone forgets to close an external handle. 2. Why call del self.file inside __del__? If the object itself is being destroyed that means anything self refers to will be useless already and the refcount should already be decremented.
- recursive 13y ago
  I thought the GC in most python implementations was deterministic. According to http://www.digi.com/wiki/developer/index.php/Python_Garbage_Collection http://www.digi.com/wiki/developer/index.php/Python_Garbage_... it does use a reference counting system, which is some inconclusive evidence. Does anyone know for sure?
  - ggchappell 13y ago
    From the documentation of the Python 3.3 data model[1]: > CPython currently uses a reference-counting scheme with (optional) delayed detection of cyclically linked garbage, which collects most objects as soon as they become unreachable, .... Other implementations act differently and CPython may change. Do not depend on immediate finalization of objects when they become unreachable (ex: always close files). > object.__del__(self) Called when the instance is about to be destroyed. This is also called a destructor. ... It is not guaranteed that __del__() methods are called for objects that still exist when the interpreter exits. Summary: Don't count on it. Use "with". [1] http://docs.python.org/3.3/reference/datamodel.html http://docs.python.org/3.3/reference/datamodel.html
  - ketralnis 13y ago
    Sort of. CPython's main GC strategy is is refcounting, which is deterministic. It also does have a cycle-detecting GC that's not deterministic But this is an implementation detail, so it could change in the future. And CPython isn't the only implementation, either. IronPython, pypy, and Jython all have different GC characteristics.
batbomb 13y ago
I love the way SQLAlchemy uses the magic methods SQL expressions.
- goostavos 13y ago
  I've never used SQLAlchemy. Do you have an example of how they use them?

152 points by yingxu 13y ago

I'm bookmarking this, this is an awesome reference. Since this article was written, __cmp__ is has now been deprecated. The article already mentions total_ordering and you can use that instead.

hugorodgerbrown 13y ago

> The use case for these magic methods might seem small, and that may very well be true. I won't spend too much more time on reflection magic methods because they aren't very important If you want a reason to use the instancecheck magic method, the mocking of test objects is a good one. I recently came unstuck with testing Django models and mocking out DateFields, and the instancecheck method was the solution - http://tech.yunojuno.com/mocking-dates-with-django http://tech.yunojuno.com/mocking-dates-with-django

JelteF 13y ago

Why is this not just in the regular python docs.

aet 13y ago
You may find some stuff here: http://docs.python.org/3.3/reference/datamodel.html http://docs.python.org/3.3/reference/datamodel.html
vanni 13y ago
It is: http://stackoverflow.com/a/18392248/366332 http://stackoverflow.com/a/18392248/366332
ketralnis 13y ago
From the very first section of the article ("Introduction"): > They're also not as well documented as they need to be. All of the magic methods for Python appear in the same section in the Python docs, but they're scattered about and only loosely organized. There's hardly an example to be found in that section (and that may very well be by design, since they're all detailed in the language reference, along with boring syntax descriptions, etc.). > So, to fix what I perceived as a flaw in Python's documentation, I set out to provide some more plain-English, example-driven documentation for Python's magic methods

nichochar 13y ago

Great article, definitely bookmarking this. I think it's important to give a reminder somewhere that modifying the magic methods really should be done with care. People new to Python could use this in a brute force way, when logic could do the trick. Working for example with a framework like web2py, that re-implements a bunch of these, makes development slow and laborious sometimes.

jessaustin 13y ago

This would be better if abstract base classes were demonstrated. It's better to build a sequence class with collections.abc.Sequence, for instance, to avoid all that boilerplate.

agf 13y ago

This is a great, thorough overview. To me, the main omission is next / __next__. It was misnamed in Python 2 (corrected to be a "magic method" in Python 3) but belongs on the list nonetheless. It is the method of an iterator that returns the next item, and is core to (almost) any iterable object in Python (except sequences that can be sequentially indexed from zero).

dirkk0 13y ago
The author states that this article is on github and can be improved there.

Too 13y ago

Some comments on the first example. 1. Closing a file by relying on the, nondeterministic, GC seems like a cover-up solution for sloppy programming. I've seen quite a few C# code bases where people even, instead of closing, put the equivalent of assert(self.file.closed) inside the destructor to get an early warning when someone forgets to close an external handle. 2. Why call del self.file inside __del__? If the object itself is being destroyed that means anything self refers to will be useless already and the refcount should already be decremented.

recursive 13y ago
I thought the GC in most python implementations was deterministic. According to http://www.digi.com/wiki/developer/index.php/Python_Garbage_Collection http://www.digi.com/wiki/developer/index.php/Python_Garbage_... it does use a reference counting system, which is some inconclusive evidence. Does anyone know for sure?
- ggchappell 13y ago
  From the documentation of the Python 3.3 data model[1]: > CPython currently uses a reference-counting scheme with (optional) delayed detection of cyclically linked garbage, which collects most objects as soon as they become unreachable, .... Other implementations act differently and CPython may change. Do not depend on immediate finalization of objects when they become unreachable (ex: always close files). > object.__del__(self) Called when the instance is about to be destroyed. This is also called a destructor. ... It is not guaranteed that __del__() methods are called for objects that still exist when the interpreter exits. Summary: Don't count on it. Use "with". [1] http://docs.python.org/3.3/reference/datamodel.html http://docs.python.org/3.3/reference/datamodel.html
- ketralnis 13y ago
  Sort of. CPython's main GC strategy is is refcounting, which is deterministic. It also does have a cycle-detecting GC that's not deterministic But this is an implementation detail, so it could change in the future. And CPython isn't the only implementation, either. IronPython, pypy, and Jython all have different GC characteristics.

batbomb 13y ago

I love the way SQLAlchemy uses the magic methods SQL expressions.

goostavos 13y ago
I've never used SQLAlchemy. Do you have an example of how they use them?