6 ms·

Python metaclasses

55 points by aldanor 11y ago

crocal 11y ago
Great tool, but not to be put in the wrong hands. The folks at Django got it right. We used it for unifying, simplifying and ensuring consistency of deeply nested configuration data entered using different means. However, in environment with less experience, I have seen "Akira-final-scene-tetsuo" monsters come to life with this. Another loosely related suggestion: if we could stay away from singleton as examples, I think we would save ourselves a lot of future troubles. ^^
- pandler 11y ago
  I enjoyed reading through the Django ORM code to understand how it worked. Metaclasses in python are fun, but I can rarely justify actually using them for anything other than messing around. Oh but how I wish I needed to use them more!
- ris 11y ago
  "Great tool, but not to be put in the wrong hands." I wouldn't quite say this - I wouldn't want to discourage someone who's going to learn them to go ahead, but I'd just say "Only use them when you really need to use them and if you're going to use them make sure you really understand them".
  - smegel 11y ago
    The problem with this is the "wrong hands" are exactly the kind of programmers who are convinced they really need complex abstractions when a much simpler option is available.
mrsuprawsm 11y ago
This is an interesting and fairly well explained post, but a little light on real world examples. Can anyone maybe suggest a few examples of where this could be used (or is used already?)
- pandler 11y ago
  The Django ORM or sqlalchemy good starting points. Metaclasses are what make them tick.
- akerl_ 11y ago
  The registry really does seem to be the most common use case: http://ivansmirnov.io/python-metaclasses/#metaclass-as-a-registry http://ivansmirnov.io/python-metaclasses/#metaclass-as-a-reg... This enables things like plugin systems, where you want the act of importing a plugin which defines some objects to load those objects into the available plugin list.
- bkeroack 11y ago
  I've used metaclasses to implement debug logging, where every class method logs its parameters and return values: https://github.com/bkeroack/elita/blob/27129ea64382275effa99d727f3dd7d9d2e1b47b/elita/dataservice/models.py#L33 https://github.com/bkeroack/elita/blob/27129ea64382275effa99... https://github.com/bkeroack/elita/blob/27129ea64382275effa99d727f3dd7d9d2e1b47b/elita/util/__init__.py#L20-L25 https://github.com/bkeroack/elita/blob/27129ea64382275effa99...
- crdoconnor 11y ago
  I've written a few code spikes using metaclasses and to be honest, it never worked out well for me. I always found a simpler, clearer way of achieving the same outcome without them and the code that I did write felt dirty. It's a very powerful language feature. Also very dangerous. It's a magnet for technical debt and obscure bugs.
- stuaxo 11y ago
  I've had fun using them to make quasi-orms for CSV and Excel spreadsheet data before.
- Retra 11y ago
  I've used it to simulate new language features. It sort of allows you to program the type system.
- aldanor 11y ago
aduitsis 11y ago
Tangential question, is there anything similar to Moose (Perl) in the Python world?
- StavrosK 11y ago
  What's moose?
  - aduitsis 11y ago
    http://moose.iinteractive.com/en/ http://moose.iinteractive.com/en/ TL;DR: Moose is an awesome OO framework for Perl.
    - StavrosK 11y ago
      I'm afraid that's not enough information for me to understand it, so I can't tell you whether there's something comparable for Python.
- bcostlow 11y ago
  Caveat: I haven't written any Perl except for an occasional one-liner in a decade. But I do a fair amount of Python, and a 20 minute cruise through the Moose docs leads me to believe that the most similar thing to Moose in the Python world is, well, Python.
  - gambogi 11y ago
    Not quite. One of the biggest features of Moose is it's concept of roles. They're in some senses similar to typeclasses in Haskell and Traits in Rust. Effectively, they give you access to parametric polymorphism in perl land. http://modernperlbooks.com/books/modern_perl_2014/07-object-oriented-perl.html#cm9sZXM http://modernperlbooks.com/books/modern_perl_2014/07-object-...
    - bcostlow 11y ago
      True. Rather than having the kind of parametric polymorphism you get with roles, Python prefers duck typing. And for the practical use of adding some composed, reusable, but orthogonal state and behavior to a class, Python provides multiple inheritance. For type-checking issues where duck typing is not sufficient, Python has Abstract Base Classes. Now, I am aware that roles are not the same thing as duck typing or multiple inheritance, or ABCs. Not having roles in Python is a design decision. http://legacy.python.org/dev/peps/pep-3133/ http://legacy.python.org/dev/peps/pep-3133/ But I still think in terms of the ability to use the language to write clean OOP code, Python is closer to Perl5 + Moose, than Perl5 alone is to Perl5 + Moose. (Not a knock on Perl5 or Moose, between 2002 and 2005, I did a bunch of web and systems programming in Perl, and I wish Moose existed then.)
skarap 11y ago
This topic has been beat to death. I guess there are more articles explaining metaclasses then people who understand them. There are 11K results for /understanding python metaclasses/ in google.
jamesdutc 11y ago
This seems to still miss the critical mechanism that metaclasses provide to the language. There are many ways to enforce a constraint from dervied classes to base classes. Metaclasses are the only* way to enforce a constraint from a base class to a derived class. (* there's actually another way, but don't do it.)
- pandler 11y ago
  Well, since you mentioned it... what is the other way? I'm curious. I promise I won't do it.
gamesbrainiac 11y ago
Metaclasses are the monads of python. Everyone seems to want to write about them at some point.
dkarapetyan 11y ago
I like Ruby's object model and uniform access principle much better. You don't have to worry about descriptors and meta-classes when doing metaprogramming. Just a few hook methods and `instance_eval` and you're good to go.

55 points by aldanor 11y ago

Great tool, but not to be put in the wrong hands. The folks at Django got it right. We used it for unifying, simplifying and ensuring consistency of deeply nested configuration data entered using different means. However, in environment with less experience, I have seen "Akira-final-scene-tetsuo" monsters come to life with this. Another loosely related suggestion: if we could stay away from singleton as examples, I think we would save ourselves a lot of future troubles. ^^

pandler 11y ago
I enjoyed reading through the Django ORM code to understand how it worked. Metaclasses in python are fun, but I can rarely justify actually using them for anything other than messing around. Oh but how I wish I needed to use them more!
ris 11y ago
"Great tool, but not to be put in the wrong hands." I wouldn't quite say this - I wouldn't want to discourage someone who's going to learn them to go ahead, but I'd just say "Only use them when you really need to use them and if you're going to use them make sure you really understand them".
- smegel 11y ago
  The problem with this is the "wrong hands" are exactly the kind of programmers who are convinced they really need complex abstractions when a much simpler option is available.

mrsuprawsm 11y ago

This is an interesting and fairly well explained post, but a little light on real world examples. Can anyone maybe suggest a few examples of where this could be used (or is used already?)

pandler 11y ago
The Django ORM or sqlalchemy good starting points. Metaclasses are what make them tick.
akerl_ 11y ago
The registry really does seem to be the most common use case: http://ivansmirnov.io/python-metaclasses/#metaclass-as-a-registry http://ivansmirnov.io/python-metaclasses/#metaclass-as-a-reg... This enables things like plugin systems, where you want the act of importing a plugin which defines some objects to load those objects into the available plugin list.
bkeroack 11y ago
I've used metaclasses to implement debug logging, where every class method logs its parameters and return values: https://github.com/bkeroack/elita/blob/27129ea64382275effa99d727f3dd7d9d2e1b47b/elita/dataservice/models.py#L33 https://github.com/bkeroack/elita/blob/27129ea64382275effa99... https://github.com/bkeroack/elita/blob/27129ea64382275effa99d727f3dd7d9d2e1b47b/elita/util/__init__.py#L20-L25 https://github.com/bkeroack/elita/blob/27129ea64382275effa99...
crdoconnor 11y ago
I've written a few code spikes using metaclasses and to be honest, it never worked out well for me. I always found a simpler, clearer way of achieving the same outcome without them and the code that I did write felt dirty. It's a very powerful language feature. Also very dangerous. It's a magnet for technical debt and obscure bugs.
stuaxo 11y ago
I've had fun using them to make quasi-orms for CSV and Excel spreadsheet data before.
Retra 11y ago
I've used it to simulate new language features. It sort of allows you to program the type system.
aldanor 11y ago

aduitsis 11y ago

Tangential question, is there anything similar to Moose (Perl) in the Python world?

StavrosK 11y ago
What's moose?
- aduitsis 11y ago
  http://moose.iinteractive.com/en/ http://moose.iinteractive.com/en/ TL;DR: Moose is an awesome OO framework for Perl.
  - StavrosK 11y ago
    I'm afraid that's not enough information for me to understand it, so I can't tell you whether there's something comparable for Python.
bcostlow 11y ago
Caveat: I haven't written any Perl except for an occasional one-liner in a decade. But I do a fair amount of Python, and a 20 minute cruise through the Moose docs leads me to believe that the most similar thing to Moose in the Python world is, well, Python.
- gambogi 11y ago
  Not quite. One of the biggest features of Moose is it's concept of roles. They're in some senses similar to typeclasses in Haskell and Traits in Rust. Effectively, they give you access to parametric polymorphism in perl land. http://modernperlbooks.com/books/modern_perl_2014/07-object-oriented-perl.html#cm9sZXM http://modernperlbooks.com/books/modern_perl_2014/07-object-...
  - bcostlow 11y ago
    True. Rather than having the kind of parametric polymorphism you get with roles, Python prefers duck typing. And for the practical use of adding some composed, reusable, but orthogonal state and behavior to a class, Python provides multiple inheritance. For type-checking issues where duck typing is not sufficient, Python has Abstract Base Classes. Now, I am aware that roles are not the same thing as duck typing or multiple inheritance, or ABCs. Not having roles in Python is a design decision. http://legacy.python.org/dev/peps/pep-3133/ http://legacy.python.org/dev/peps/pep-3133/ But I still think in terms of the ability to use the language to write clean OOP code, Python is closer to Perl5 + Moose, than Perl5 alone is to Perl5 + Moose. (Not a knock on Perl5 or Moose, between 2002 and 2005, I did a bunch of web and systems programming in Perl, and I wish Moose existed then.)

skarap 11y ago

This topic has been beat to death. I guess there are more articles explaining metaclasses then people who understand them. There are 11K results for /understanding python metaclasses/ in google.

jamesdutc 11y ago

This seems to still miss the critical mechanism that metaclasses provide to the language. There are many ways to enforce a constraint from dervied classes to base classes. Metaclasses are the only* way to enforce a constraint from a base class to a derived class. (* there's actually another way, but don't do it.)

pandler 11y ago
Well, since you mentioned it... what is the other way? I'm curious. I promise I won't do it.

gamesbrainiac 11y ago

Metaclasses are the monads of python. Everyone seems to want to write about them at some point.

dkarapetyan 11y ago

I like Ruby's object model and uniform access principle much better. You don't have to worry about descriptors and meta-classes when doing metaprogramming. Just a few hook methods and `instance_eval` and you're good to go.