3 ms·
Painpoint with type annotations: not being able to reuse "shapes" of data, e.g. struct-like fields such as TypedDict, NamedTuple, dataclasses.dataclass, and soo
by tony 4y ago
Painpoint with type annotations: not being able to reuse "shapes" of data, e.g. struct-like fields such as TypedDict, NamedTuple, dataclasses.dataclass, and soon *kwargs (PEP 692 [1]) via TypedDict.
Right now, there isn't a way to load up a JSON / YAML / TOML into a dictionary, upcast it via a `TypedGuard`, and pass it into a TypedDict / NamedTuple / dataclass.
dataclasses.asdict() or dataclasses.astuple() return naïve / untyped tuples and dicts. Also the factory functions will not work with TypedDict or NamedTuple, respectively, even if you duplicate the fields by hand [2].
Standard library doesn't have runtime validation (e.g. pydantic [3]). If I make a typed NamedTuple/TypedDict/dataclass with `apples: int`, nothing is raised in runtime when a string is passed.
Other issues you may run into using mypy:
- pytest fixtures are hard. It's repetitious needing to re-annotate them every test.
- Django is hard. PEP 681 [4] may not be a saving grace either [5]. Projects like django-stubs don't give you completions, it'd be a dream to see reverse relations in django models.
- Some projects out there have very odd packaging and metaprogramming that make typing and completions impossible: faker, FactoryBoy.
[1] https://peps.python.org/pep-0692/ https://peps.python.org/pep-0692/
[2] https://github.com/python/typeshed/issues/8580 https://github.com/python/typeshed/issues/8580
[3] https://github.com/pydantic/pydantic https://github.com/pydantic/pydantic
[4] https://peps.python.org/pep-0681/ https://peps.python.org/pep-0681/
[5] https://github.com/microsoft/pyright/blob/8a1932b/specs/dataclass_transforms.md#django https://github.com/microsoft/pyright/blob/8a1932b/specs/data...
- charliermarsh 4y agoThe "shapes of data" thing resonates a lot especially coming from TypeScript. The runtime validation stuff has been interesting to watch. Do you use Pydantic? It's very popular but I have a hard time getting over its willingness to cast / coerce implicitly (if I mark a field as an int, and pass in a str, I want an error -- is that weird of me?).
- stevesimmons 4y agoYou can type it as StrictInt etc to stop this implicit type conversion. https://pydantic-docs.helpmanual.io/usage/types/#strict-types https://pydantic-docs.helpmanual.io/usage/types/#strict-type...
- tony 4y agoI haven't used pydantic yet. I'm conservative when adding (non-dev) dependencies since I am maintaining library packages. For other projects, it's a possibility. > if I mark a field as an int, and pass in a str, I want an error -- is that weird of me That is perfectly fine. And the question is why don't TypedDict, NamedTuple, dataclass raise during construction - since if they don't - we have to play it safe and do isinstance checking since we can't trust the field's types at runtime. I suppose the idea the `typing` module [1] is to be unobtrusive: not to be involved in runtime checks. What I want is something that does what pydantic does in standard library. I think it's a sensible request and would be indispensable for anyone wrangling data. [1] https://docs.python.org/3/library/typing.html https://docs.python.org/3/library/typing.html
- kortex 4y ago> pytest fixtures are hard. It's repetitious needing to re-annotate them every test. Yeah, this drives me nuts as well. But Pycharm has recently started inferring types and jump-to-declaration works, so I believe obtaining that information must be possible. Celery is another library in the group like django/pytest in that everything about it is suuuper dynamic and *kwargs-y. It drives me up a wall that I can't readily decouple task functions from their interfaces in a typesafe way. Also I don't know how to decouple the tasks from a Celery app with a pre-defined broker uri without resorting to config files/env vars - that approach simply does not unit test well.
- rzimmerman 4y agoYes anything that dynamic is super confusing. I honestly usually don’t bother type checking unit tests. There’s definitely some value but it’s really hard to justify on existing codebases. One thing I love about type hints (rather than compiler checks) is that you can do things like that.
- leni536 4y agohttps://pypi.org/project/jsonschema-typed-v2/ https://pypi.org/project/jsonschema-typed-v2/ While not without caveats, but I started experimenting this, and it is quite useful.
- danjac 4y agoI have yet to get a moderately-complex Django project working with django-stubs. It seems to break with basic idioms like custom QuerySets, and I immediately run into new bugs every time I try it. Usually I give up and use dialled-down mypy settings.
- noitpmeder 4y agoYou should look into the 'dacite' library! It solved most of our load-json-into-typed-dataclass woes.