6 ms·

Customizing class creation in Python

32 points by joeyespo 10y ago

andy_ppp 10y ago
I don't see why you'd need this sort of indirection. It's a very strange way of hiding behaviour from people that is currently clearly understood. Should metaclasses really be able to magic up decorators out of thin air and why is importing a decorator and using it bad? As for __init_subclass__() it literally allows you to mess with things when subclassing objects in totally unexpected ways! "Adjusting how classes are created can be very difficult to debug and so should only be used when you have a really legitimate use-case." I'd say never use it - if anyone has a good reason to do magic like this I'd like to see it!
- prashnts 10y ago
  While I agree that _this_ is an extreme demonstration, metaclasses are powerful tool for developing dev-friendly Base classes/APIs in your libraries. Django models, for example, use metaclasses so the fields you define are more expressive while creating instances.
  - paulddraper 10y ago
    That's okay as long as the magic holds. If it starts to fail, you have to peek behind the curtain, and you lose yourself among the meta.
    - andybak 10y ago
      Django famously had a "magic removal" refactor before 1.0 as they collectively decided they had gone too far. They've since plotted a more conservative path but the ORM and similar magic was deemed reasonable. I've found Django's metaclass usage pretty robust over the years. The curtain mostly remains unpeeked behind.
- joshuamorton 10y ago
  Take a look at SQLAlchemy/django's ORMs. Personally I have two uses, one is in proprietary code, but the idea is basically use python to generate json, but allow incoherence of json pieces via natural python flows. (metaclasses specifically allowed me to reverse the flow of creating objects, so that visually, code wouldn't look like inner1 = JsonObject(config) inner2 = JsonObject(config) outer = JsonObject(config, children=[inner1, inner2]) but instead class outer(JsonObject): inner1 = JsonObject(config) inner2 = JsonObject(config) _config = config ish. With this isn't not clear why this is important, but with highly nested structures, having the python code visually mirror the output in terms of nesting is really nice. The second is at https://github.com/gtagency/pyrostest/blob/master/pyrostest/rostest_utils.py https://github.com/gtagency/pyrostest/blob/master/pyrostest/.... This isn't strictly necessary, but "The documentation states that all test_* methods in a RosTest subclass spin up roscore" is a lot simpler and safer than "if you forget to add `RosTest.setUp()` and `RosTest.tearDown()` at the beginning and end of all of your setUp and tearDown methods, your tests will fail. (and further, future invocations of tests may fail until you run something like `killall -9 roscore && killall -9 rosmaster` in shell because you've unintentionally broken your environment." Generally speaking, Metaclasses allow you to make better user interfaces for developers, and avoid the repetitive kinds of things you encounter in "enterprisey" code. Specifically, they allow you to make interfaces much more declarative than you otherwise could (ie. the SQLAlchemy example of "this is a table and here are its columns" vs. "My table is a function that takes in some stuff and magic happens within it")
  - andy_ppp 10y ago
    Personally I find a language with Macros easier to understand and far more powerful for such things, and it's not a load of special cases that are still being added to (in this case the __init_subclass added in 3.6). Fine about meta classes for some things but you haven't covered why you'd use the two examples in the article.
    - joshuamorton 10y ago
      I actually wanted prepare in production, because it would have allowed me to override the class dict with an ordereddict, which was a more preferable solution to making output jsons deterministic than just sorting the components, but you do what you can. As for init subclass, if you're alright with a DSL that does weird things with the class line, it can be useful, as an example applied to SQLAlchemy, they could do this (in pseudo sqla): class MyTable(Table, primary_key='uid'): uid = Column(int) which would enforce via the api that there can only be 1 primary key.
luhn 10y ago
> aside: please don't abuse collections.namedtuple to make a simple Python object; the class is meant to help porting APIs that return a tuple to a more object-oriented one, so starting with namedtuple means you end up leaking a tuple API that you probably didn't want to begin with I haven't heard this before. I subclass namedtuples regularly—whenever I need a simple immutable object. I haven't had any problems personally. (Except when pickling them. I would not recommend that.) Can any experienced Pythonistas weigh in? Is this bad practice?
- TTPrograms 10y ago
  I don't really see how it's a problem that it's "leaking" a tuple API - it's not like there's private methods in Python. Everything can see everything. If anything having the state of an object available explicitly as a tuple seems like a good idea to me, as opposed to trying to figure out what's a stateful variable and what's a method etc.
- joshuamorton 10y ago
  You'll get surprises such as your subclass being indexable when you (probably) don't want that.
  - lgas 10y ago
    What problem would that cause? Wouldn't it only be a problem if you attempted to index into it, which you wouldn't do (on purpose) if you didn't think it was indexable? And if you did try to index into it thinking that it wasn't indexable, it would be a bug, which is the same as when it's indexable...
    - joshuamorton 10y ago
      Exactly. You probably don't want this thing that is essentially a namespace to be indexable, but you index into it, which is a bug, but it quacks like a tuple, so your mistake propagates silently, which is a bad thing. Its more permissive than one would expect. Although the bigger issue is what dragonne mentioned.
carapace 10y ago
> Adjusting how classes are created can be very difficult to debug and so should only be used when you have a really legitimate use-case. ...and you never have a really legitimate use-case. Seriously, please don't use this sort of magic in production code that you expect other people to use and depend on. I'm looking at you Django project. Bloddy cowboys.

32 points by joeyespo 10y ago

I don't see why you'd need this sort of indirection. It's a very strange way of hiding behaviour from people that is currently clearly understood. Should metaclasses really be able to magic up decorators out of thin air and why is importing a decorator and using it bad? As for __init_subclass__() it literally allows you to mess with things when subclassing objects in totally unexpected ways! "Adjusting how classes are created can be very difficult to debug and so should only be used when you have a really legitimate use-case." I'd say never use it - if anyone has a good reason to do magic like this I'd like to see it!

prashnts 10y ago
While I agree that _this_ is an extreme demonstration, metaclasses are powerful tool for developing dev-friendly Base classes/APIs in your libraries. Django models, for example, use metaclasses so the fields you define are more expressive while creating instances.
- paulddraper 10y ago
  That's okay as long as the magic holds. If it starts to fail, you have to peek behind the curtain, and you lose yourself among the meta.
  - andybak 10y ago
    Django famously had a "magic removal" refactor before 1.0 as they collectively decided they had gone too far. They've since plotted a more conservative path but the ORM and similar magic was deemed reasonable. I've found Django's metaclass usage pretty robust over the years. The curtain mostly remains unpeeked behind.
joshuamorton 10y ago
Take a look at SQLAlchemy/django's ORMs. Personally I have two uses, one is in proprietary code, but the idea is basically use python to generate json, but allow incoherence of json pieces via natural python flows. (metaclasses specifically allowed me to reverse the flow of creating objects, so that visually, code wouldn't look like inner1 = JsonObject(config) inner2 = JsonObject(config) outer = JsonObject(config, children=[inner1, inner2]) but instead class outer(JsonObject): inner1 = JsonObject(config) inner2 = JsonObject(config) _config = config ish. With this isn't not clear why this is important, but with highly nested structures, having the python code visually mirror the output in terms of nesting is really nice. The second is at https://github.com/gtagency/pyrostest/blob/master/pyrostest/rostest_utils.py https://github.com/gtagency/pyrostest/blob/master/pyrostest/.... This isn't strictly necessary, but "The documentation states that all test_* methods in a RosTest subclass spin up roscore" is a lot simpler and safer than "if you forget to add `RosTest.setUp()` and `RosTest.tearDown()` at the beginning and end of all of your setUp and tearDown methods, your tests will fail. (and further, future invocations of tests may fail until you run something like `killall -9 roscore && killall -9 rosmaster` in shell because you've unintentionally broken your environment." Generally speaking, Metaclasses allow you to make better user interfaces for developers, and avoid the repetitive kinds of things you encounter in "enterprisey" code. Specifically, they allow you to make interfaces much more declarative than you otherwise could (ie. the SQLAlchemy example of "this is a table and here are its columns" vs. "My table is a function that takes in some stuff and magic happens within it")
- andy_ppp 10y ago
  Personally I find a language with Macros easier to understand and far more powerful for such things, and it's not a load of special cases that are still being added to (in this case the __init_subclass added in 3.6). Fine about meta classes for some things but you haven't covered why you'd use the two examples in the article.
  - joshuamorton 10y ago
    I actually wanted prepare in production, because it would have allowed me to override the class dict with an ordereddict, which was a more preferable solution to making output jsons deterministic than just sorting the components, but you do what you can. As for init subclass, if you're alright with a DSL that does weird things with the class line, it can be useful, as an example applied to SQLAlchemy, they could do this (in pseudo sqla): class MyTable(Table, primary_key='uid'): uid = Column(int) which would enforce via the api that there can only be 1 primary key.

luhn 10y ago

> aside: please don't abuse collections.namedtuple to make a simple Python object; the class is meant to help porting APIs that return a tuple to a more object-oriented one, so starting with namedtuple means you end up leaking a tuple API that you probably didn't want to begin with I haven't heard this before. I subclass namedtuples regularly—whenever I need a simple immutable object. I haven't had any problems personally. (Except when pickling them. I would not recommend that.) Can any experienced Pythonistas weigh in? Is this bad practice?

TTPrograms 10y ago
I don't really see how it's a problem that it's "leaking" a tuple API - it's not like there's private methods in Python. Everything can see everything. If anything having the state of an object available explicitly as a tuple seems like a good idea to me, as opposed to trying to figure out what's a stateful variable and what's a method etc.
joshuamorton 10y ago
You'll get surprises such as your subclass being indexable when you (probably) don't want that.
- lgas 10y ago
  What problem would that cause? Wouldn't it only be a problem if you attempted to index into it, which you wouldn't do (on purpose) if you didn't think it was indexable? And if you did try to index into it thinking that it wasn't indexable, it would be a bug, which is the same as when it's indexable...
  - joshuamorton 10y ago
    Exactly. You probably don't want this thing that is essentially a namespace to be indexable, but you index into it, which is a bug, but it quacks like a tuple, so your mistake propagates silently, which is a bad thing. Its more permissive than one would expect. Although the bigger issue is what dragonne mentioned.

carapace 10y ago

> Adjusting how classes are created can be very difficult to debug and so should only be used when you have a really legitimate use-case. ...and you never have a really legitimate use-case. Seriously, please don't use this sort of magic in production code that you expect other people to use and depend on. I'm looking at you Django project. Bloddy cowboys.