6 ms·
Wield Python's super() like a Jedi
- lza 15y agoIt's a super() good overview of super(). Nice to see a post that uses a real example.
- mclin 15y agoHave to deal with enough languages as it is. Don't want to further add to my brain mess by learning Python 3 syntax just yet.
- raymondh 15y agosuper() was introduced in Python2.2 many years ago. Everything in the article works with very old versions of Python. Here's a link to the Python 2 version of the examples: http://code.activestate.com/recipes/577721-how-to-use-super-effectively-python-27-version/ http://code.activestate.com/recipes/577721-how-to-use-super-...
- kunley 15y agoPython 2.x doesn't have parameterless super() so apparently most of the article's code doesn't work with old versions.
- fuzzyman 15y agoWell, you just change to the Python 2 super call form and it works fine. The substitution hardly requires a master programmer...
- bendotc 15y agosuper() was introduced in CPython 2.2, but it changed in Python3: http://www.python.org/dev/peps/pep-3135/ http://www.python.org/dev/peps/pep-3135/. The link you posted uses super as "super(cls, instance)", but the main article uses Python 3's super, which can just be called as "super()", and it figures out the class and instance. So no, super really has changed, and the syntax in the article does not work with very old version of Python.
- bretthoerner 15y ago> So no, super really has changed, and the syntax in the article does not work with very old version of Python. Oh come on, he's saying if you do the incredibly minor syntax adjustment that all of the actual meat of the article still works in Python 2.
- nicpottier 15y agoI upvoted the parent, because that syntax distinction is actually fairly giant, always having to name the class and also changing that on renames etc, is a PITA. It is actually one of the things that annoys me most about Python, I guess this is the first thing that actually tempts me with Python 3. Damn.
- mclin 15y agosuper(myclass, self) was. Like I posted below, sometimes I have real trouble keeping the 5 or so languages I use straight. The other day I actually forgot what 'None' is in python because I hadn't used it in a few days, and None is different than in the others.
- steve-howard 15y agoPython 3 was introduced two and a half years ago. Its major syntactic changes, while frustrating if you're set in your ways, are hardly insurmountable.
- Wickk 15y agoThis. Most of the time I run into an issue with a change I wasn't aware of, a quick search through the 3.X docs just points me in the right direction.
- mclin 15y agoI think it's fine if you're doing python all day long, but I'm using Python, Java, PHP, Obj-C and Javascript each at least once a month on a variety of paid and personal projects. Like I said, I have enough trouble keeping those syntaxes straight.
- exDM69 15y agoAh, the "call super" antipattern revived.
- __rkaup__ 15y agoHow's it an antipattern?
- veyron 15y agohow does the runtime performance of super() compare to the hardcoding method? (from your example, using dict.__setitem__ as opposed to super().__setitem__ )
- alextp 15y agoI don't think it suffers much as the method resolution order (mro) is computed ahead-of-time when creating the class.
- veyron 15y agoso super() is computed ahead of time? That doesn't make sense when you look at his comment: "The calculation depends on both the class where super is called and on the instance’s tree of ancestors." I agree that the class's MRO can be computed ahead of time, but theoretically the instance's order could be different (if a class method changed the definition of __setitem__)
- alextp 15y agoThe method resolution order for instances of the same class is always the same for all methods. If the class doesn't implement that method, the interpreter just skips and goes to the next place in the call chain. All super does is call the next method in the chain. So if you have class A(object), class B(A), class C(A), and class D(B, C), the MRO of D's instances will be B C A object. Each time an instance of D calls super, it will call the next method in the chain, not necessarily B's or C's or A's, even if the call to super is in B's method. This is why super() receives the class as a parameter, so it can know from when in the object's mro to start looking up the next method.
- veyron 15y ago"This is why super() receives the class as a parameter" <-- this is not necessary in python 3. Your argument is valid in python2, but python3 allows a super() call without a type argument. From the docs: "super(type[, object-or-type])" <-- python2 doc, type is required "super([type[, object-or-type]])" <-- type is optional in python3
- kalelias 15y agotl;dr Just looks like super/parent like in other languages. Did i miss something from not reading this?
- masklinn 15y ago> Did i miss something from not reading this? Everything? Mostly the part where, Python being an MI language, a given class A has multiple parents. And these parents are a sequence, not a set, which define a Method Resolution Order, and that method resolution order usually isn't breadth-first either, so Python's super is a graph walker. And because it's explicit, any node in the graph can stop the walking. This means permuting two parent classes can have significant impact on the semantics of the graph walking, therefore the results of calling super.
- kalelias 15y agoThanks for the short explanation. I totally forget about python being MI.
- pdubroy 15y agoYes.
- Peristarkawan 15y agoYes, Python's super is more like Dylan's next-method than, say, C#'s super.
- silentbicycle 15y agoSome small steps in the direction of a meta-object protocol: more fine-grained control over call dispatch in a multiply-inheriting class hierarchy.
- smosher 15y agoYou didn't miss as much as others seem to think. Nothing here changes the fact that it subverts the object model and overuse causes kudzu.
- guygurari 15y agoIt seems to me that code that uses super() in these creative ways will be very difficult to maintain. You need to understand the subtleties of the MRO just to find out which method is being called. Pitty the poor Python programmer who stumbles into such code without being aware that super() may not call the class's direct base. I suppose similar criticism can be leveled against many other dynamic techniques, except that this one sacrifices more readability than I'm comfortable with.
- jessedhillon 15y agosuper() supports arguments which allow you to specify which class's implementation to lookup, which is an explicit way to resolve ambiguities. Although well-designed multiple-inheritance should try to avoid those ambiguities in the first place -- a class shouldn't inherit from multiple bases with conflicting implementations of a method. I think in a case like the one you describe, where the direct ancestor's implementation is circumvented to access a different ancestor's implementation, it should be done by calling super(Ancestor).someMethod() That should improve understanding.
- Peristarkawan 15y agoI disagree, that's extremely fragile. It might work for the particular class you are implementing. However, it might cause subclasses of that class to break, since those subclasses will have a different MRO than the current class, and you could end up skipping more of the MRO than you intended.
- deleted 15y ago[deleted]
- jessedhillon 15y agoHow would it cause a subclass to break? The ancestor is explicitly specified -- this could only break if the MRO was modified to exclude that particular ancestor.
- jamesli 15y agoAm I missing some points here? It seems that the examples are tightly coupled and favors more on inheritance than on composition.
- mrshoe 15y agoPersonally I'm not in favor of enforcing such strict conventions, especially in a dynamic language like Python. Also isn't this pattern he repeats throughout: class Shape: def __init__(self, **kwds): self.shapename = kwds.pop('shapename') super().__init__(**kwds) Better written like this? class Shape: def __init__(self, shapename=None, **kwds): self.shapename = shapename super().__init__(**kwds) It seems like there are some things about Python's which the author doesn't like, and he solves them by adding a lot of boiler plate to each of his classes. It's probably a better idea to just embrace the Python way or use a different language.
- teh 15y agoThe issue with your second version is In [1]: def init(shapename=None, **kwargs): ...: pass ...: In [2]: init(shapename='circle', **{'shapename': 'circle'}) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) /home/teh/Desktop/<ipython console> in <module>() TypeError: init() got multiple values for keyword argument 'shapename'
- Peristarkawan 15y agoThat's entirely the caller's fault, not the function's. Compare: >>> def init(**kwargs): ... pass ... >>> init(shapename='circle', **{'shapename': 'circle'}) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: init() got multiple values for keyword argument 'shapename'
- narm60 15y agoThese two aren't equivalent, the first example requires the shapename keyword arg, raising a KeyError when it wasn't specified
- Peristarkawan 15y agoThat's true. The second one might be better written as: def __init__(self, shapename, **kwds): Or, in Python 3: def __init__(self, *, shapename, **kwds): (making shapename a required keyword-only argument)
- Peristarkawan 15y agoIt seems to me that the example of combining built-in dictionary classes is naively optimistic. For starters, OrderedDict, as it happens, does not use super! It calls the dict super-class methods directly. Since dict happens to be the next class in the MRO, this doesn't really matter for the purpose of this example, but I can envision a scenario where some plucky programmer inherits from both OrderedCounter and some other dict subclass, and the result doesn't work because OrderedDict does the wrong thing. And OrderedDict isn't the only one. Maybe for some reason I would like to have an OrderedCounter where all the counts default to 42. So I do this: class DefaultOrderedCounter(defaultdict, OrderedCounter): pass doc = DefaultOrderedCounter(lambda: 42) doc.update('abracadabra') Which results in: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "c:\python32\lib\collections.py", line 507, in update _count_elements(self, iterable) File "c:\python32\lib\collections.py", line 63, in __setitem__ self.__map[key] = link = Link() AttributeError: 'DefaultOrderedCounter' object has no attribute '_OrderedDict__map' Whoops! Apparently defaultdict doesn't use super either. Of course a better way to do this would be to subclass DefaultOrderedCounter and just override the __missing__ method by hand, but that's not the point. The article goes into "How to Incorporate a Non-cooperative Class", which basically says "wrap it up in a proxy class". But that's not really going to work here, since the result would be two separate dicts, with the defaultdictwrapper methods operating on one dict, and the other methods operating on the other.
- Peristarkawan 15y agoOkay, I give. What's the markup for posting code snippets on this site?
- lmkg 15y agoPut two spaces at the beginning of a line for that line to be displayed monospaced. Example
- simcop2387 15y agowhy would you have to subclass DefaultOrderedCounter again in order to add in the missing method? Though I agree there's something annoying going on there with the multiple inheritence.
- lachenmayer 15y agoNice article, but one thing struck me when reading the code examples posted in the article: What's the use of using kwds in __init__? class Shape(Root): def __init__(self, **kwds): self.shapename = kwds.pop('shapename') super().__init__(**kwds) class ColoredShape(Shape): def __init__(self, **kwds): self.color = kwds.pop('color') super().__init__(**kwds) I would personally just have written this somewhat similar to this: class Shape(Root): def __init__(self, shapename): self.shapename = shapename super().__init__() class ColoredShape(Shape): def __init__(self, shapename, color): self.shapename = shapename self.color = color super().__init__(shapename) But then again I'm by no means what anyone could call a Python 'Jedi'. Can anyone explain to me why using kwds in this way would be better? Would it not lead to having to look up the declaration of the class for the exact names of arguments to pop() every time?
- Peristarkawan 15y agoThe reason for using keywords is that there is no guarantee that the next class in the MRO after ColoredShape will be Shape. When the instance is a ColoredShape, it will be, but if ColoredShape is subclassed with a more complex inheritance structure, then something else could end up in between them in the MRO. Thus, ColoredShape.__init__ can't simply pass on shapename to the super method, because that might not be the argument expected by the next __init__ method in the MRO. Instead, each method needs to gracefully accept arbitrary arguments and then pass on whatever it received (minus what it consumed).
- silentbicycle 15y agoThis is a particularly unnecessary title change.
- oinksoft 15y agoA true Jedi favors composition.
- uriel 15y agoOne of the reasons I'm starting to love Go more than Python. In Go something like super() is superfluous.
- jessedhillon 15y agoThe only reason why you don't need super in Go is because you cannot embed multiple interfaces except if their sets of methods are disjoint. This eliminates super's job of allowing programmers to differentiate between base classes with conflicting method implementations. Without this, Go requires you to design your class hierarchy in a way that is cleaner, but the super concept is still not superfluous -- because you do sometimes need to reference an ancestor's implementation in Go. You embed an interface and then explicitly reference the embedded interface by name. The Go interface type ReaderWriter interface { Reader Writer } is exactly like this Python class ReaderWriter(Reader, Writer): pass To get at a particular superclass implementation, you pass the class to super, like so: def read_and_write(self): super(Reader).read() super(Writer).write() which in Go is: func (rw *ReaderWriter) ReadAndWrite() { rw.Reader.read() rw.Writer.write() } So it's not superfluous -- there's an equivalent syntax to super which involves referencing embedded interfaces.
- Peristarkawan 15y agoSigh. No, that is not at all how super works in Python. The equivalent code to that Go function would be: def read_and_write(self): Reader.read(self) Writer.write(self) That is, if you want to get at a particular superclass implementation, then you just call it directly. super() is used when you're not calling a particular superclass, instead relying on Python to compute the correct "next method" for you. "super(Reader).read()", if you actually try it, will just result in an AttributeError. This is because super(Reader) returns an unbound super object, not a bound super object that you can actually dispatch from. Why this advanced usage might be useful is beyond the scope of this thread, but if you're curious then just google for python unbound super and hit the Feeling Lucky button.
- deleted 15y ago[deleted]
- draebek 15y agoI didn't see it linked in the article nor mentioned here already but several of the problems with `super` have been discussed in "Python's Super is nifty, but you can't use it"[1] for quite some time. [1]: http://fuhm.net/super-harmful/ http://fuhm.net/super-harmful/
- mikeklaas 15y agoThat article is about Python 2's super, which is a different beast. Lessons were learnt in the design of Python 3's version.
- Peristarkawan 15y agoNot really. The only thing that changed about super in Python 3 is that you can call it with no arguments, which is just a short-hand for the way it was already used in Python 2. Everything in that article applies equally to both Python versions.
- fferen 15y agoThe "kwds" technique is actually something I came up with independently to solve a different problem when I was using PyGame: ridiculously long argument lists when constructing sprites. In this case it significantly increased the flexibility and brevity of the code, although perhaps with a more complicated inheritance tree it wouldn't be worth it. Nice to see this pattern being acknowledged somewhere else; I thought I was the only one that did that.
- raymondh 15y agoIt's a great technique. I'm sure your PyGame code benefitted in a big way :-)