16 ms·
Python 3.7: Introducing Data Classes
- hartator 9y agoIs Python becoming more statically typed then?
- craftyguy 9y agoIt certainly seems to be allowing the option to be. I welcome this reduction in boilerplate-ness.
- dragonwriter 9y agoPython, like many dynamic languages, is gaining increasing support for optional static type declarations and AOT type checking and is also leveraging the same structure to make code more concise.
- jernfrost 9y agoDynamic and static languages are fundamentally different. There is no more or less. I've tried to elaborate here why annotating variables with type doesn't make a dynamic language static: https://medium.com/@Jernfrost/dynamically-typed-languages-are-not-what-you-think-ac8d1392b803 https://medium.com/@Jernfrost/dynamically-typed-languages-ar... You can't look at the presence of type information to determine if a language is static or dynamic. What matters is when and how that type information is used. In static languages expression have type. In dynamic languages values have type. The implication is that you can't know the type of something in a dynamic language until the expression has been evaluated at run time. With a static language we can determine the type of every expression at compile time, which requires full knowledge of the whole program. It is why dependencies becomes much more problematic in static languages and why they are so poor as glue languages.
- RussianCow 9y ago> With a static language we can determine the type of every expression at compile time, which requires full knowledge of the whole program. It is why dependencies becomes much more problematic in static languages and why they are so poor as glue languages. But you have to have this information anyway, or at least most of it. Even in a dynamically typed language, you have to know what sort of arguments a function expects; otherwise, how can you write any code at all to operate on those values if you don't even know what they are? Static typing just forces you to be more explicit about encoding those constraints. Whether that's worth the tradeoff depends on a lot of factors, but its use as a "glue" language is certainly not one of them. Edit: Moreover, the line between statically typed and dynamically typed languages isn't as well-defined as you claim. For instance, TypeScript is a statically typed language, but it successfully compiles regular JavaScript code because, by default, every expression is given the type `any`. This means you can start out with dynamic typing and add static types progressively. And at that point, how is that any different than Python with mypy[0]? [0]: http://mypy-lang.org/ http://mypy-lang.org/
- goatlover 9y ago> Even in a dynamically typed language, you have to know what sort of arguments a function expects; otherwise, how can you write any code at all to operate on those values if you don't even know what they are? True, but the Smalltalk idea was that you send a message to an object, and that object decides how (or whether) to handle it. So in Ruby, if you send the + method to some kind of number object along with the objects you want added, and the receiving object decides how to perform that addition, assuming the objects you sent can be summed.
- RussianCow 9y agoSure, but even then, you still have to make assumptions about what messages an object accepts. In other words, you have to have some information about the interface of every object.
- UncleEntity 9y ago> Even in a dynamically typed language, you have to know what sort of arguments a function expects; otherwise, how can you write any code at all to operate on those values if you don't even know what they are? Not necessarily... If the value passed to function implements whatever functionality called then its type doesn't really matter. Really the whole theory behind duck typing. >>> i = 1 + "1" Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for +: 'int' and 'str' while javascript (apparently) will assign "11" to i.
- weberc2 9y agoYou seem to contradict yourself. > With a static language we can determine the type of every expression at compile time, which requires full knowledge of the whole program. > Dynamic and static languages are fundamentally different. There is no more or less. For many Python programs, every expression can be determined statically. While this may not be true for all expressions in all programs, surely this means the OP's claim that "Python is becoming more statically typed" is true and your claim that "There is no more or less" is false, no? Besides, there are languages like Go which have dynamic types (interface{}) and which are considered statically typed languages. You might say that the static type is interface{}, but then you could say that every type in an un-annotated Python program is also interface{} (or whatever the equivalent Python type would be). It seems like your arguments hinge on semantic games.
- coldtea 9y ago>Besides, there are languages like Go which have dynamic types (interface{}) and which are considered statically typed languages. And C# has a var type (dynamic) and Dart has gradual typing.
- jakear 9y agovar is type inference, not dynamic typing. Unless something has changed since I last C#’d. The docs seem to say this still holds: https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/var https://docs.microsoft.com/en-us/dotnet/csharp/language-refe...
- coldtea 9y agoI meant this variable declaration: C# Dynamic Type: C# 4.0 (.NET 4.5) introduced a new type that avoids compile time type checking. You have learned about the implicitly typed variable- var in the previous section where the compiler assigns a specific type based on the value of the expression. ... A dynamic type can be defined using the dynamic keyword.Jan 1, 2014
- 9y ago
- staticassertion 9y ago> In static languages expression have type. In dynamic languages values have type. I don't think this is true. In a dynamic language all values, including expressions, share a single type - a union of all possible types. In a static language you can limit which possible types a value may hold. A dynamic language is like if you cast every single type to Any in a static language. In Python this is entirely possible with mypy - a static type system for the Python language, which works through annotations. > in static languages they are used to prevent the compilation of programs containing expressions where types don’t match up. This is already the case with mypy in Python. It seems very much static. It's like everything is statically determined to be Any, then you specify in certain areas a more limited type. In your case with Julia, the type annotations are static. Even if by default Julia is dynamic, it allows for static annotations. I disagree that there is no in-between. I think Julia is actually a great example of an in-between. And, in fact, it has a name - gradual typing.
- spiralx 9y ago> In a dynamic language all values, including expressions, share a single type - a union of all possible types. In a static language you can limit which possible types a value may hold. A dynamic language is like if you cast every single type to Any in a static language. No, you're thinking of a weakly- versus strongly-typed language. Python is dynamically and strongly typed - values have a definite type, but a variable can be assigned a value of any type.
- RussianCow 9y agoI think you misread the OP: they are saying that Python is like a statically typed language in which every value has the type Any. That has nothing to do with strong/weak typing.
- staticassertion 9y agoAs RussianCow said, I think you're misreading my post. A Python variable, from a type perspective, is a variable with a type Any/ a type that is a union of all possible types. Using annotations you can then specify the type. Nothing to do with weak/ strong, which merely imply some level of implicit casting.
- joshuamorton 9y agoSo what this comes down to is that python is not statically typed, but mypy+ python is. And given how mypy works, you can enforce types over only parts of a program. That, to me, sounds like the python ecosystem is becoming more statically typed.
- jjuel 9y agoWell considering they are type hints that aren't actually checked then the answer is no.
- jonathanpoulter 9y agoThat was my take on this too, why do the type hints need to be there for this to work, they don't seem to be involved at all?
- dragonwriter 9y ago> That was my take on this too, why do the type hints need to be there for this to work Because (1) you need to declare the fields, and (2) it's good to do that in a way that will work with typecheckers even though they are external, and (3) there are actually two particular type declarations that are used by the implementation, even aside from external typecheckers.
- mixmastamyk 9y agoThere are a few options to use them without declared types, but for some reason there was a noticeable reluctance to document them, perhaps due to types coming back into fashion. For example you could set the type to object, typing.Any, ... (Ellipsis), or None, etc.
- jernfrost 9y agoLooks like how a lot of languages already work out of the box. E.g. whenever I create a data type in Julia I automatically have such a constructor. Static languages such as Go and C already essentially let you do this through initialization through braces.
- fleetfox 9y agoWhile 3.7 is not here, back-port: https://github.com/ericvsmith/dataclasses https://github.com/ericvsmith/dataclasses
- mixmastamyk 9y agoAlso on PyPI.
- metalliqaz 9y agoRaymond Hettinger had a pretty good presentation on Data Classes and how they relate to things like named tuples and a few recipes/patterns. It was linked on Reddit[0] but it looks like the video has been removed from YouTube. His slides are online[1], though. [0] https://www.reddit.com/r/Python/comments/7tnbny/raymond_hettinger_python_37s_new_data_classes/ https://www.reddit.com/r/Python/comments/7tnbny/raymond_hett... [1] https://twitter.com/i/web/status/959358630377091072 https://twitter.com/i/web/status/959358630377091072
- reaperhulk 9y agoAs noted in the PEP data classes is a less fully-featured stdlib implementation of what attrs already provides. Unless you’re constrained to the stdlib (as those who write CPython itself of course are) you should consider taking a look at attrs first. http://www.attrs.org/en/stable/ http://www.attrs.org/en/stable/
- oblio 9y agoThis library? http://www.attrs.org/en/stable/ http://www.attrs.org/en/stable/
- reaperhulk 9y agoYep, I’ll add the URL in my comment, sorry about that.
- shoyer 9y agoLess fully-featured, sure, but also a bit more cleanly designed (in my opinion). Are there features from attrs that you would miss with dataclasses?
- metalliqaz 9y agoThis is spot on. The design of attrs remines me a little bit of the syntax from a declarative ORM, for example. I'm sure it can do very powerful things that I've not had occasion to use, but it is heavy. The @dataclass format is very clean and seems more like the syntactic sugar that I expect from Python. One of the prime uses of a dataclass is to be a mutable namedtuple. And the syntax can be almost identical: Part = make_dataclass('Part', ['part_num', 'description', 'quantity']) (from Raymond Hettinger's twitter) This has the added benefit of not requiring type hinting, if you don't want to bother with such things.
- froztbyte 9y agoin a freshly created venv, with attrs and ipython (and deps, of course): (attrs) % ipython Python 2.7.14 (default, Oct 19 2017, 23:28:49) Type "copyright", "credits" or "license" for more information. IPython 5.6.0 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. In [1]: import attr In [2]: Part = attr.make_class("Part", ["part_num", "description", "quantity"]) In [3]: Part? Init signature: Part(self, part_num, description, quantity) Docstring: <no docstring> Type: type Outside of the name of the thing that creates the item, the syntax is identical to what you have here. edit: removed triple backticks, indented with 2sp according to https://news.ycombinator.com/formatdoc https://news.ycombinator.com/formatdoc rules
- swalsh 9y agoLooks great, too bad my project will forever be in 2.7
- fermigier 9y agoYou an use attrs then.
- sigjuice 9y agoWhy too bad? Sounds like you have a useful and realistic project and you don't plan to waste time on 2 -> 3 busy work.
- theptip 9y agoPresumably (hopefully?) not in 2020, when it stops getting security fixes? https://legacy.python.org/dev/peps/pep-0373/ https://legacy.python.org/dev/peps/pep-0373/
- zenhack 9y agoIt will probably be a fair bit longer than that before 2.7 stops being relevant. 2.7 is still the default system Python on a lot of Linux distros, which will be in vendor support for longer. It doesn't really matter if the fixes are coming from the PSF or someone else - nice thing about Foss. RHEL/CentOS 6 is still reasonably widely used, and the system Python there is 2.6.
- kstrauser 9y ago> 2.7 is still the default system Python on a lot of Linux distros I will never understand why that matters. "ed" is the default system editor, but I'm only "{apt-get,yum} install {vim,emacs}" away from having something I actually want to use. That's the whole point of a distro. You don't have to use Python 2.ancient just because /usr/bin/paleolithic is written with it.
- jstarfish 9y agoNot all environments have unfettered access to the internet to download whatever arbitrary packages the user decides they need that day. Sometimes you're stuck with whatever the distro shipped with.
- ianamartin 9y agoI'm piqued by one thing in this article. Using an object as a dictionary key. What's the use for that? I don't think it's ever occurred to me to do that.
- cakebrewery 9y agoAvoids the extra step of hashing the object to make the key by simply passing the object itself.
- talltimtom 9y agoAnything hashable can be a dict key, so if you have some mapping between one thing that’s hashable and other things, you can use a dict directly rather than having to come up with an index and keep track of that association.
- chubot 9y agoI'm not sure what part of the article you refer to, but: - If the objects are value types (as dataclasses are, since they respect equality): it makes sense to use them as dict keys, just as it makes sense to use tuples as dict keys. Changing a tuple to a namedtuple is natural. - If the objects have reference semantics rather than value semantics, you can still use them as dict keys in Python. The default __hash__ is the id/address of the object (the default __eq__ uses id() too). I don't particularly like this style but I have seen it used. It feels error-prone to me and not "algebraic".
- dagw 9y agoUsing an object as a dictionary key. What's the use for that? I've used tuples has dict keys several times, so I don't see why I conceivably wouldn't use a Data Class containing the same data as a dict key.
- comnetxr 9y agomemoization is a common use case. If you are doing an expensive computation that gives the same answer for the the same inputs, you can build a lookup table for cases you have already encountered to avoid recomputing. you can use an object to encapsulate the inputs (or the inputs might be objects already).
- Kwpolska 9y agoattrs does a pretty good job, while also supporting older Python versions: http://www.attrs.org/en/stable/ http://www.attrs.org/en/stable/
- foxhop 9y agoI have to be honest, coming from Python 2.3 (2004ish), I don't recognize "new" Python anymore. I think it's mostly regarding type definitions.
- prlin 9y agoAfter seeing the huge improvements that JavaScript has gone through over the years I'm all for language updates. Same with Java and C++ (although not as much for Java and I don't know C++ but I always hear C++11 is "new").
- Deinumite 9y agoJava 8 (and 9 more so) bring a lot of changes but more focused on core libraries than the language itself. Not as drastic as the changes JS has seen. C++11 though is quite a bit different. Probably the biggest change being that raw pointers * for the most part should not be used anymore.
- toopsss 9y agoIt’s not too bad I think, it’s just an evolution really. You can probably grok the basics of the type annotations in a short sit down. I can’t even remember when decorators were introduced but that even more greatly changed how python was written. I’ve been using python since 1.6 and I always thought the amount of repetition was ridiculous. I bet I’m not the only one that has written a “dsl” of what attrs and this pep does 1000 times using the facilities python had at the time: metaclasses, then decorators. Of course all these implementations were rushed, half assed and barely production quality. Despite any warts attrs is a pleasure to use. Type annotations boost IntelliJ/pycharm already quite clever assistance. One lingering thing is attrs named_attrs that while syntactically the best approach in my mind doesn’t work well with IntelliJ. So hopefully this will address it.
- gdwatson 9y agoIt's relatively recent. IMHO Python 3.5 to 3.7 feel like the language is going in a different direction than it did before -- type hints and the handling of asynchrony in particular.
- breatheoften 9y agoWould be convenient if it also supporting generating a to dict method with dunder hooks for customizing the translation ...
- SatvikBeri 9y agoFinally! Attrs is great, but I'm glad to see this in the standard library.
- Osmium 9y agoLink to the PEP: https://www.python.org/dev/peps/pep-0557/ https://www.python.org/dev/peps/pep-0557/
- weberc2 9y agoI'm happy to see data classes. I think something like this exists in 3.6: class Person(typing.NamedTuple): name: str age: int But I don't think it supports the __post_init__; however, constructors have no business doing parsing like this anyway, so unless I'm missing something, deriving from `typing.NamedTuple` seems strictly better than `@dataclass` insofar as it seems less likely to be abused.
- mixmastamyk 9y agoTuples are read-only.
- weberc2 9y agoAh, of course. Good point. I tend to write things in an immutable style, so I don't usually pay attention to this.
- silviogutierrez 9y agoI see immutability as a feature. There are uses cases for mutability, but it should be opt-in, not opt-out. So I'm not loving the fact that "frozen" (the mutability param for data classes) defaults to False.
- weberc2 9y agoI agree, which is part of the reason I think I'd be inclined to derive from NamedTuple and forego data classes.
- icebraining 9y agoPython is very much not immutable by default, it'd be weird to subvert it in a specific case.
- caioariede 9y agoAlso, AFAIK you cannot add methods to NamedTuple, making it much less flexible than data classes.
- std_throwaway 9y agoComing from C++ it feels really weird that you can simply assign instance.new_name = value from anywhere without properly declaring it beforehand. You also never really know what you get or if somebody modified your instance members from the outside.
- gtaylor 9y agoIf you run a linter, the cases where you are doing this outside of __init__ will usually be pointed out. You can silence the warning/error on a case by case basis if you really need to do it.
- craigds 9y agoWhich linter does that? flake8 doesn't AFAIK.
- a_bored_husky 9y agoPylint does: http://pylint-messages.wikidot.com/messages:w0201 http://pylint-messages.wikidot.com/messages:w0201
- kstrauser 9y agopylint does.
- mixmastamyk 9y agoDoesn't happen maliciously in practice, also can be very handy when you need to attach a little extra data for the ride. If you need extra assurance there are techniques to make the instance "very" read-only.
- goatlover 9y agoJS & PHP let you do this as well. One advantage is that you don't have to adhere to a rigid class structure and be forced to refactor or create a new class every-time you need add a new property or method. And sometimes you want a property/method for just that particular instance, and not all members. As with most things, there are trade-offs.
- tommikaikkonen 9y agoI love using attrs, like the idea of bringing something similar to the standard library, but strongly disagree with the dataclasses API. It treats untyped Python as a second class citizen. This is what I'd prefer from dataclasses import dataclass, field @dataclass class MyClass: x = field() but it produces an error because fields need to be declared with a type annotation. This is the GvR recommended way to get around it: @dataclass class MyClass: x: object You could use the typing.Any type instead of object, but then you need to import a whole typing library to use untyped dataclasses. I highly prefer the former code block. There's a big thread discussing the issue on python-dev somewhere. Also some discussion in https://github.com/ericvsmith/dataclasses/issues/2#issuecomment-353903594 https://github.com/ericvsmith/dataclasses/issues/2#issuecomm... Anyway, it's not a huge issue—attrs is great and there's no reason not to use it instead for untyped Python.
- sleavey 9y agoYeah, it seems strange to force people to use type hints when it has had such a mixed reception. I really tried to use type hints with a new project a few months ago, but ended up stripping it all out again because it's just so damn ugly. I wish it were possible to fully define type hints in a separate file for linters, and not mix it in with production code. It's kind of possible to do it, but not fully [1], and mixing type hints inline and in separate files is in my opinion even worse than one or the other. [1] https://stackoverflow.com/questions/47350570/is-it-possible-to-separate-all-type-hinting-checking-infrastructure-in-python https://stackoverflow.com/questions/47350570/is-it-possible-...
- rezistik 9y agoI've always wanted a programming UI similar to RapGenius's UI. With annotations and docs being opened in a form panel.
- david-cako 9y agoThis __post_init__ thing is shit IMO. Why does it make sense for the language feature to allow you to completely disregard the type hint and fix it later? I have to imagine that the type checkers would factor in anything that occurs in __post_init__ when evaluating whether the class conforms to the type hints, but it still feels like this python static typing stuff is drifting in the wrong direction.
- cpburns2009 9y agoI don't like `__post_init__` either. Wouldn't it make more sense to just override `__init__` similar to: @dataclass class DataObj: some_date: datetime def __init__(self, **kw): kw['some_date'] = dateutil.parser.parse(kw['some_date']) super().__init__(**kw)
- dragonwriter 9y agoTo allow you to override init that way, @dataclass would have to create an additional superclass with the generated init method, rather than generating the init method in the class intended for use, which seems to have more overhead.
- cpburns2009 9y agoYeah, good point. I didn't think that through very much.
- ptx 9y agoYou could attach the __init__ method to dataclass (as if it were a class) and call it like this, couldn't you? dataclass.__init__(self, **kw) Like in the olden days before super().
- dragonwriter 9y agoFrom the description, the dataclass decorator generates an __init__ method for the decorated class, which seems different than using a common generic __init__ method for all classes with the decorator.
- Rotareti 9y agoIt's great that we have simple/clean declarations for NamedTuples an (Data)classes now. But I wonder why they chose two different styles for creating them. This for NamedTuples: from typing import NamedTuple class Foo(NamedTuple): bar: str baz: int and this for DataClasses: @dataclass class Foo: bar: str baz: int
- metalliqaz 9y agoWhen you write it that way it makes me wonder why there isn't a DataClass type
- joshuamorton 9y agoThe short answer is that the only way to do what dataclasses do as a base class is via python metaclasses, and you can only have one metaclass. So this way, you can dataclassify something that inherits from a metaclass.
- deleted 9y ago[deleted]
- aserafini 9y agoI love that Peter Norvig left an improvement to the __post_init__ method in the comments section of the JetBrains blog. I wonder if he uses PyCharm?
- czardoz 9y agoLooking forward to having ORMs support this way of defining models
- no_wizard 9y agoI just hope the `__post_init__` method catches on and becomes a regular python dundermethod. I actually find it to be a very good quality in this new implementation. Example from the PEP 557: @dataclass class C: a: float b: float c: float = field(init=False) def __post_init__(self): self.c = self.a + self.b https://www.python.org/dev/peps/pep-0557/#post-init-processing https://www.python.org/dev/peps/pep-0557/#post-init-processi... I could use this all over the place.
- UncleEntity 9y agoWhere would it be useful outside of an automagically generated __init__ method?
- noobermin 9y agoI have used dictionaries for this for the longest time. The thing is I sort of rely on the type being flexible, so what can I do?
- kalessin 9y agoIf you rely on your type being "flexible" I would argue that you have a design problem.
- noobermin 9y agoMy working code beats your ideological purity.
- kalessin 9y agoThis was really coming from practical experience, but no offense taken.
- preordained 9y agoSeconded...I swear to God "that's a symptom of bad design" and its ilk are the most hackneyed type of response to a situation that has the bonus of making you seem "above it all". It absolves you of even having to bother to look into the situation to see if maybe there is nuance that you can't just throw platitudes at.
- 9y ago
- kuon 9y agoI think it's because I'm working with elm right now, but this kind of thing scares me: created: datetime But if type(self.created) is str: self.created = dateutil.parser.parse(self.created) So basically, the type annotation cannot be trusted.
- owaislone 9y agoNo, they can't be but Python doesn't pretend that they do. They are just that, annotations. They don't ensure correctness. It's up to other programs to analyze (like mypy) and warn during development. That's it.
- bunderbunder 9y agoIn Python, type annotations are (expressly) just a special type of comment that's been given a regularized format for the sake of both human and computer readability. I don't know that it should scare you, but, like with many Python features, it's something that should be approached in a sensible manner. The Python philosophy is to leave you with the freedom to monkey around, and leave it to you to decide whether you want to abuse that ability. (Look at me, all talking like someone who didn't spend 2 hours diagnosing a type error that a static language would have found and forced me to fix within half a second. :P )
- AlexCoventry 9y agoReminds me a lot of the attrs module.
- axzak 9y agoIt will be nice to have this but for data deserialization, which is the case in the example, I would still use drf serializers or marshmallow schemas.
- aldanor 9y agoRe: serialization, worth mentioning cattrs project (work with attrs classes, but would work the same way with stdlib dataclass): https://github.com/Tinche/cattrs https://github.com/Tinche/cattrs