4 ms·

Even using Python regularly I haven’t run into a need for meta classes. I’m sure there is a valid reason, but what is it?

tybug 5y ago
The most salient example I can think of is tracking all subclasses of a class. This is how django tracks `Model` subclasses to create a table for each model, for instance. I wrote a blog post on this topic: https://tybug.github.io/2021/11/06/tracking-all-subclasses-of-a-class.html https://tybug.github.io/2021/11/06/tracking-all-subclasses-o... I've also used metaclasses in the past to apply a decorator to certain methods of all subclasses, without needing to specify that decorator in the subclass declaration itself.
- dbrueck 5y ago
  Came here to say this. This is my most common use of metaclasses, and __init_subclass__ looks to be a much cleaner solution.
fdomingues 5y ago
Is useful to do code generation/transformation, for example in this utility[1] it uses the information provided on the declaration of the class to generate the methods required by the base class. The user of the utility write this: class Book(objects.Object): title: Optional[str] And the generated class code is: class Book(Base): __slots__ = ('title', ) title: typing.Optional[str] def __init__(self, title=None): self.title = title @classmethod def from_data(cls, data): title = data.get('title', None) return cls(title=title) def to_data(self): data = {} if self.title is not None: data['title'] = self.title return data @classmethod def from_oracle_object(cls, obj): return cls(title=obj.title) def to_oracle_object(self, connection): obj = connection.gettype('Book').newobject() obj.title = self.title return obj [1]https://github.com/domingues/oracle-object-mapping https://github.com/domingues/oracle-object-mapping
dragonwriter 5y ago
The need isn't common, but there are some real uses. The top answer of this SO thread (I didn't read the rest, so there may be more goodies) has some good discussion of use cases including comparison to non-metaclass approaches to the same problems. https://stackoverflow.com/questions/392160/what-are-some-concrete-use-cases-for-metaclasses https://stackoverflow.com/questions/392160/what-are-some-con...
enragedcacti 5y ago
More generically, Metaclasses are primarily useful when using (abusing?) the Python type system to represent other type systems. The most common example of this is ORMs like SQLAlchemy, where the Python type system is used to represent tables in a relational database. Another example is the builtin "Typing" module where the type system is used to provide a language for representing itself. "Union" itself is a type, and metaclasses allow you to set "__getitem__" on the "Union" type such that Union[Int|Str] is valid and represents a new type. Not that I don't know if metaclasses are actually used to build this system internally, but you could use it for many of the features implemented by Typing. There are alternative ways to go about doing that because Python is so flexible, but IMO metaclasses (or __init_subclass__ as seen in this post) are the least "magic" of the magic ways to tackle problems like these.
- contravariant 5y ago
  You got me curious so I looked it up. Apparently in Cpython 'Union' is not a class itself, but is a special object of type _SpecialForm, which is created by using _SpecialForm as a decorator on a function 'Union' which returns the actual union type given a set of parameters. Interestingly _SpecialForm uses a mixin _Final to prevent subclassing. This mixin is made using __init_subclass__
throwaway19937 5y ago
(https://www.linuxjournal.com/article/3882 https://www.linuxjournal.com/article/3882) has a dated (2000) example of metaclass usage in Python 2.
Sohcahtoa82 5y ago
I recently used them to create a framework for handling plugins to an IRC bot. By creating an IrcMessageHandler metaclass, I can easily subclass it to encapsulate functionality into different classes without having to add boilerplate to load each plugin. There's a Plugins directory with a __init__.py that automatically loads all .py files, and the main bot just has to do `from Plugins import *` to load them all. Each plugin script merely needs a `from ircbot import IrcMessageHandler`, then creates a subclass. So there's a Calc class that adds a "!calc" command, Seen class that adds "!seen", and more. Each of them merely needs to include an "on_message" function that parses IRC messages.
contravariant 5y ago
You basically need it when you have some very strong opinions on how objects should be created, and this isn't the same as the way that Python does it by default. To understand this fully you should try to understand how 'type' is the class of a class (which is also an object in python) and you should read how 'type' creates a new class, which is basically the job of your new metaclass should you create one. Usually you don't need it. Also it's finicky to get right. Something like __init_subclass__ has a more specific purpose, which I suppose is good. However it also sounds like the kind of thing you probably shouldn't be doing.

tybug 5y ago

The most salient example I can think of is tracking all subclasses of a class. This is how django tracks `Model` subclasses to create a table for each model, for instance. I wrote a blog post on this topic: https://tybug.github.io/2021/11/06/tracking-all-subclasses-of-a-class.html https://tybug.github.io/2021/11/06/tracking-all-subclasses-o... I've also used metaclasses in the past to apply a decorator to certain methods of all subclasses, without needing to specify that decorator in the subclass declaration itself.

dbrueck 5y ago
Came here to say this. This is my most common use of metaclasses, and __init_subclass__ looks to be a much cleaner solution.

fdomingues 5y ago

Is useful to do code generation/transformation, for example in this utility[1] it uses the information provided on the declaration of the class to generate the methods required by the base class. The user of the utility write this: class Book(objects.Object): title: Optional[str] And the generated class code is: class Book(Base): __slots__ = ('title', ) title: typing.Optional[str] def __init__(self, title=None): self.title = title @classmethod def from_data(cls, data): title = data.get('title', None) return cls(title=title) def to_data(self): data = {} if self.title is not None: data['title'] = self.title return data @classmethod def from_oracle_object(cls, obj): return cls(title=obj.title) def to_oracle_object(self, connection): obj = connection.gettype('Book').newobject() obj.title = self.title return obj [1]https://github.com/domingues/oracle-object-mapping https://github.com/domingues/oracle-object-mapping

dragonwriter 5y ago

The need isn't common, but there are some real uses. The top answer of this SO thread (I didn't read the rest, so there may be more goodies) has some good discussion of use cases including comparison to non-metaclass approaches to the same problems. https://stackoverflow.com/questions/392160/what-are-some-concrete-use-cases-for-metaclasses https://stackoverflow.com/questions/392160/what-are-some-con...

enragedcacti 5y ago

More generically, Metaclasses are primarily useful when using (abusing?) the Python type system to represent other type systems. The most common example of this is ORMs like SQLAlchemy, where the Python type system is used to represent tables in a relational database. Another example is the builtin "Typing" module where the type system is used to provide a language for representing itself. "Union" itself is a type, and metaclasses allow you to set "__getitem__" on the "Union" type such that Union[Int|Str] is valid and represents a new type. Not that I don't know if metaclasses are actually used to build this system internally, but you could use it for many of the features implemented by Typing. There are alternative ways to go about doing that because Python is so flexible, but IMO metaclasses (or __init_subclass__ as seen in this post) are the least "magic" of the magic ways to tackle problems like these.

contravariant 5y ago
You got me curious so I looked it up. Apparently in Cpython 'Union' is not a class itself, but is a special object of type _SpecialForm, which is created by using _SpecialForm as a decorator on a function 'Union' which returns the actual union type given a set of parameters. Interestingly _SpecialForm uses a mixin _Final to prevent subclassing. This mixin is made using __init_subclass__

throwaway19937 5y ago

(https://www.linuxjournal.com/article/3882 https://www.linuxjournal.com/article/3882) has a dated (2000) example of metaclass usage in Python 2.

Sohcahtoa82 5y ago

I recently used them to create a framework for handling plugins to an IRC bot. By creating an IrcMessageHandler metaclass, I can easily subclass it to encapsulate functionality into different classes without having to add boilerplate to load each plugin. There's a Plugins directory with a __init__.py that automatically loads all .py files, and the main bot just has to do `from Plugins import *` to load them all. Each plugin script merely needs a `from ircbot import IrcMessageHandler`, then creates a subclass. So there's a Calc class that adds a "!calc" command, Seen class that adds "!seen", and more. Each of them merely needs to include an "on_message" function that parses IRC messages.

contravariant 5y ago

You basically need it when you have some very strong opinions on how objects should be created, and this isn't the same as the way that Python does it by default. To understand this fully you should try to understand how 'type' is the class of a class (which is also an object in python) and you should read how 'type' creates a new class, which is basically the job of your new metaclass should you create one. Usually you don't need it. Also it's finicky to get right. Something like __init_subclass__ has a more specific purpose, which I suppose is good. However it also sounds like the kind of thing you probably shouldn't be doing.