5 ms·
Even before talking about the typing system itself, annotations existed by themselves and are not attached to any behavior in Python: they simply hold metadata
by BiteCode_dev 4y ago
Even before talking about the typing system itself, annotations existed by themselves and are not attached to any behavior in Python: they simply hold metadata.
When Python 3 came out, annotations were indeed added to the language, but no use case was decided for them. It was an experiment to see what the community would do with them instead of forcing a use case base on what the core devs would think would be useful.
In order to allow this, annotations where permitted to be any legal python expression.
Nowadays, type hints have won: annotations are widely used to add static type checking in Python. But the way the language treat them have not changed much appart from becomming lazy. While mypy cannot understand dynamic Python expressions in them, those are nevertheless legal syntax, and will result in correct metadata at runtime.
Therefore, were the typing system not turing complete, the annotations would have been anyway by nature of being basically an exec().
- arjvik 4y agoWhat other use cases did the community come up with?
- Too 4y agoCreating argument parsers for CLIs: https://pypi.org/project/argtyped/ https://pypi.org/project/argtyped/ I've seen similar (in-house) to generate HTML forms also.
- ok123456 4y agoThings like pydantic and fastapi where the type annotations are used for run time argument validation and interface generation.
- deleted 4y ago[deleted]
- whimsicalism 4y agoDoes this still work with Python 3.10 changes removing runtime evaluation of type annotations? I seem to recall this screwed me when I was writing something for runtime usage of types.
- BiteCode_dev 4y agoThere was a heated debate because it broke things at the time, but now it works, yes.
- patrick91 4y agowe still don't know what's going to happen with type hints in future though, but I'm sure things won't break as they might have broken if they changed the default behaviour
- whimsicalism 4y agoOh it looks like they actually pushed back the PEP 563 changes.. From this thread [0] it does not look open and shut that it will work once these changes go into effect. [0]: https://github.com/pydantic/pydantic/issues/2678 https://github.com/pydantic/pydantic/issues/2678
- LtWorf 4y agoThey postponed that change. It remained enabled with a future and while it kinda works, it doesn't work in all cases. Non top level definitions can never be retrieved basically.
- BiteCode_dev 4y agoDependency injections, foreign-language bridges, data validation and parsing, etc. But thanks to how we can introspect annotations, we could have a specialized syntax for types that would be also usable for those things, while the other way around would be harder. So it's nice we went the type hints road, because you can now define types, and yet have wonderful libs like pydantic, fastapi, typer, taichi and so on.
- arjvik 4y agoCool, any examples of libraries that use it for dependency injection or FFI or validation?
- GalaxySnail 4y agoFor FFI, there is a pure python mode in cython: https://cython.readthedocs.io/en/latest/src/tutorial/external.html
- BiteCode_dev 4y agodjango-ninja and fastapi does dep injection, parsing and validation based on annotations. taichi does ffi with their own dsl, cython with c. I you haven't tried pydantic or sister projects (django-ninja, fastapi and typer), this is a good place to start. They bring a breath of fresh air and are quite fun to use.
- kylebarron 4y agoSpecifically it's the underlying pydantic that does validation based on annotations https://pydantic-docs.helpmanual.io/ https://pydantic-docs.helpmanual.io/
- BiteCode_dev 4y agoIndeed. The dep injection is fastapi and typer code though, and quite tied to it. So it's worth mentioning that somebody is attempting (quite successfully from the look of it) to create a generic lib of dep injection using annotation named DI, inspired by those libs: https://github.com/adriangb/di https://github.com/adriangb/di
- patrick91 4y agoI've created strawberry graphql, it uses type hints to generate GraphQL schemas
- Waterluvian 4y agoI used them at runtime to define the types for my (absolutely stupid) library that takes a Python function and replaces its contents with a C call. You define a Python function interface but populate it with a string of C code.
- kortex 4y agoWait, so like this? def add(x: int, y: int) -> int: """ // c code int x, y return x + y """ And it compiles it and turns it into a C extension? That's...horrifying yet amazing.
- eru 4y agoThat might be a worthwhile addition to the suggestion at https://github.com/faster-cpython/ideas/issues/453 https://github.com/faster-cpython/ideas/issues/453
- BiteCode_dev 4y agoCombined with the cog (https://nedbatchelder.com/code/cog https://nedbatchelder.com/code/cog) to have the c declaration in a proper separate c file, but included in the docstring, you could have the best of both worlds.
- BiteCode_dev 4y agoThat's a neat hack!
- _dain_ 4y agoThere's a library for simulating the Excel calculation engine in Python. It uses annotations to hint functions, but according to Excel's type system, not Python's. Then at runtime, those hints are used to do conversion and promotion based on how Excel does it, so you can do e.g. the equivalent of "1" + 1 and get 2. It's either xlcalculator or pycel, one of the two. I forgot which one does it that way.
- arjvik 4y agoLooks like it's xlcalculator: https://github.com/bradbase/xlcalculator#addingregistering-excel-functions https://github.com/bradbase/xlcalculator#addingregistering-e...
- anderskaseorg 4y agoThat’s not at all what’s going on here. While the Python interpreter permits and executes arbitrary expressions as annotations, Python static type checkers (mypy, Pytype, Pyre, Pyright) do not. This isn’t just an implementation limitation; the typing specifications (PEP 484 and successors) don’t allow it. For example, although these two definitions evaluate to the same result at runtime: def f() -> int: return 1 def f() -> (lambda: int)(): return 1 only the first is correctly typed by PEP 484. mypy rejects the second with “error: Invalid type comment or annotation”. That’s the whole idea of static type checking: in return for restricting yourself to a controlled subset of the language that isn’t “too dynamic”, you gain the ability to type-check your code without running it. The result of this paper is that one can nonetheless abuse subtyping to perform Turing-complete computation in the type checker. The computation doesn’t happen when your code is executed (annotations have no effect on the Python interpreter at runtime unless your program chooses to inspect them), it happens when you run mypy on it. This is generally considered undesirable, because we want type checking to be decidable, and Turing-complete computation is not.
- eru 4y ago> That’s the whole idea of static type checking: in return for restricting yourself to a controlled subset of the language that isn’t “too dynamic”, you gain the ability to type-check your code without running it. Eh, you are mixing up distinctions. All that's required of _static_ typing is that your static type specification finishes running before you start your program. Whether that static type specification language is Turing complete or not is a completely separate issue. And, yes, you are right that it's generally useful for that language to be decidable. But see eg Haskell's UndecidableInstances.
- BiteCode_dev 4y agoThat's why my comment litterally starts with "even before talking about the typing system itself".
- anderskaseorg 4y agoI’m sure that you understood the important differences, but there are likely many readers who didn’t (many more programmers are familiar with Python than with mypy). When you write “even if we don’t have P, we’d have Q anyway”, the reader is left with the impression that P and Q are somehow interchangeable or equivalent or comparable—that P is unimportant in the presence of Q. I think it’s instructive to point out that this impression would be mistaken, and the paper’s result of P is interesting and consequential.