14 ms·
Python Type Hints – *args and **kwargs (2021)
- assbuttbuttass 3y agoThis does restrict all of your keyword arguments to the same type. If you have keyword arguments of different types, you're right back to no type safety.
- masklinn 3y agoThat seems obvious? If you want a variable number of arguments of arbitrary type you have to specify the common supertype, commonly top itself. To do otherwise would require some form of vararg generics which is uncommon.
- IshKebab 3y agoIt's extremely common for Python programmers to write code with kwargs of different types. Look at subprocess.run() for example.
- actualwitch 3y agoWell, if you want to type your kwargs and use newer versions of python, you can use Unpack with typed dicts to achieve that. But the footgun there is that you can't redefine fields when extending them, so no Partial<SomeType> for you.
- zbentley 3y agoTrue, but there are a couple of mitigations available: you can express the types of selected kwargs (by leaving them out of the * residual), and you can use typing.Union/| to express product types for values in the residual as well.
- PartiallyTyped 3y agoAlternatively, use an `@overload` in a `.pyi` file and specify your types there. This means that you will have 2^N combinations and doubling every time you accept a new argument. If that is not good enough, then simply use a `TypedDict` with everything optional instead of `**kwargs`. Your call will then become `foo(SomeTypedDict(p1=p2,...))`.
- vorticalbox 3y agoWhy do people not just type everything they want passed? def variable(n:str, nn:str, nnn:str, *, a:int, b:int, c:int) Anything after,*, is a kwarg.
- masklinn 3y agoYour signature requires exactly 3 positional[0] and 3 keyword arguments. The OP allows any number of either. [0] actually 3 positional-or-keyword which is even more widely divergent
- vorticalbox 3y agoBut why would you want that doesn't that make for a more confusing api? Would it not be better to just have everything as a kwarg? You would get better types that way
- masklinn 3y agoI genuinely don’t understand what you are asking.
- zbentley 3y agoI think what GP is saying is that with explicit kwargs you can't express variadic signatures, i.e. "this function takes one int positional, and then any number of key/value pairs where the values are lists". The variable length is the important bit. It's certainly debatable whether doing that is better than passing a single argument whose value is a dict with that same type, but many people do prefer the variadic args/kwargs style.
- Znafon 3y agoIt is used when the number of argument can vary, like: def sum(*args: int) -> int: if len(args) == 0: return 0 return args[0] + sum(*args[1:])
- zoomablemind 3y ago
- akasakahakada 3y agoAlthough these two comes in handly, people have been using them wrong. Often in scientific open source package, they slap *kwargs in function definition without documentation. How am I suppose to know what to pass in? https://qiskit.org/ecosystem/aer/stubs/qiskit_aer.primitives.Estimator.set_options.html#qiskit_aer.primitives.Estimator.set_options https://qiskit.org/ecosystem/aer/stubs/qiskit_aer.primitives...
- toxik 3y agoSadly a problem with any wrapper function is that it nullifies this kind of information. Use functools.wraps.
- akasakahakada 3y agoMy question is that can @warps warp more than 1 function? Maybe in some use case people need to merge 2 functions into 1, I don't know if it can handle this situation.
- zbentley 3y agoI'm not sure what it means to "merge two functions into one", can you elaborate? If you are referring to a type signature for a function that passes through it's arguments to one of two inner functions, each of which has different signatures, such that the outer signature accepts the union of the two inner signatures, well ... you could achieve that with ParamSpecs or similar, but it would be pretty hard to read and indirected. Better, I'd say, to manually express appropriate typing.Union (|) annotations on the outer function, even if that is a little less DRY.
- cbarrick 3y ago> I'm not sure what it means to "merge two functions into one", can you elaborate? I'm not OP, but I see this pattern often enough: def foo(**kwargs): pass der bar(**kwargs): pass def wrapper(**kwargs): foo(**kwargs) bar(**kwargs)
- SushiHippie 3y agoFor typing **kwargs there are TypedDicts https://peps.python.org/pep-0692/ https://peps.python.org/pep-0692/ If your function just wraps another you can use the same type hints as the other function with functools.wraps https://docs.python.org/3/library/functools.html#functools.wraps https://docs.python.org/3/library/functools.html#functools.w...
- zbentley 3y agoWhile functools.wraps does propagate __annotations__ by default, be aware that not all IDE-integrated type checkers handle that properly. It's easy in PyCharm, for example, to use functools.wraps such that the wrapper function is treated by the IDE as untyped. Underneath, this is because many (most?) type checkers for Python aren't actually running the code in order to access annotation information, and are instead parsing it "from the outside" using complex and fallible techniques of variable reliability. That said, it's a testament to JetBrains' excellent work that PyCharm's checker works as well as it does, given how crazily metaprogrammed even simple Python often turns out to be.
- veber-alex 3y agoPycharm has the worst type checker that exists today. It may have been the best a few years back but others have suppressed it considerably. I recently switched from Pycharm to vscode which uses pyright and it's night and day on the amount of type errors it catches, it considerably improved the quality of my code and confidence during refactoring. And to add insult to injury Pycharm doesn't even have a pyright plugin and the mypy plugin is extremely slow and buggy.
- no_wizard 3y agoThere definitely seems to be a few areas since the Fleet announcement that have given me pause on JetBrains. Their Python support has t kept up with other tools as noted. I’ve see a similar decline in the ability for for them to keep up to date with things like Svelte, Vue, Astro etc too. They need to embrace the LSP
- 3y ago
- refactor_master 3y agoThe ability of **kwargs to leave behind no proper documentation and silently swallow any invalid arguments has made us remove them entirely from our codebase. They're almost entirely redundant when you have dataclasses.
- liquidpele 3y agoYea, really only useful imho for proxy functions that then just pass the arguments along to something that DOES properly type every arg.
- roland35 3y agoI agree - it is convenient to use at first but it sure makes it hard to use an unfamiliar codebase!
- zbentley 3y agoWhat about decorators, or wrappers around third-party code whose contracts change frequently (or even second party code when interacting with functions provided by teams that don't follow explicit argument typing guidelines, if you have that sort of culture)?
- refactor_master 3y agoUsually the solutions range from a culture of “just don’t” to tests/mypy that have become increasingly stricter over the years, every time we’ve come a step further up the ladder. But I admit, it has taken quite some bridging to get there. Moving to static Python in most places has dramatically improved the code and language.
- blibble 3y agonow try typing a decorator https://stackoverflow.com/questions/47060133/python-3-type-hinting-for-decorator https://stackoverflow.com/questions/47060133/python-3-type-h... what a disaster
- amethyst 3y agoPEP 612 made this much better FWIW. https://peps.python.org/pep-0612/ https://peps.python.org/pep-0612/
- hk__2 3y ago> In the function body, args will be a tuple, and kwargs a dict with string keys. This always bugs me: why is `args` immutable (tuple) but `kwargs` mutable (dict)? In my experience it’s much more common to have to extend or modify `kwargs` rather than `args`, but I would find more natural having an immutable dict for `kwargs`.
- adamchainz 3y agoYeah, that is odd. Python still has no immutable dict type, except it kinda does: https://adamj.eu/tech/2022/01/05/how-to-make-immutable-dict-in-python/ https://adamj.eu/tech/2022/01/05/how-to-make-immutable-dict-...
- dragonwriter 3y ago> This always bugs me: why is `args` immutable (tuple) but `kwargs` mutable (dict)? Because python didn’t (still doesn’t, but at this point even if it did backward compatibility would mean it wouldn’t be used for this purpose) have a basic immutable mapping type to use. (Note, yes, MappingProxyType exists, but that’s a proxy without mutation operations, not a basic type, so it costs a level of indirection.)
- throwaway2037 3y agoIn Python, except for mutability, is there any difference between tuple and list? In my experience: Pure Python people get so excited about tuples ("oh, it's so Pythonic"); others: much less.
- hk__2 3y ago> In Python, except for mutability, is there any difference between tuple and list? In my experience: Pure Python people get so excited about tuples ("oh, it's so Pythonic"); others: much less. In my experience, people who don’t care about tuples are people who don’t understand them. It’s not much about being Pythonic or not (they exists in other languages) but rather about choosing the right data structure for solving your problem. Tuples are much more than a way of making immutable lists, they offer a type-safe and serializable representation of pairs and triplets; something you can’t have with a list. If you don’t use them yet, I really encourage you to document yourself (and again-- in general, not just in Python) because you’re missing something.
- dfee 3y agoI actually created a library for this! Forge: forge (python signatures) for fun and profit https://python-forge.readthedocs.io/ https://python-forge.readthedocs.io/ https://github.com/dfee/forge https://github.com/dfee/forge
- itissid 3y ago#TIL. Also cool to know is pydantic's @validate decorator: https://docs.pydantic.dev/latest/usage/validation_decorator/#function-signatures https://docs.pydantic.dev/latest/usage/validation_decorator/... and in case you were thinking its not superflous to mypy(https://docs.pydantic.dev/latest/usage/validation_decorator/#usage-with-mypy https://docs.pydantic.dev/latest/usage/validation_decorator/...).
- m3047 3y agoThat article promulgates a misunderstanding about immutability. For my way of thinking, python is already an interpreted language and I can enforce tropes in code more cleanly and effectively than people taking something five levels up at face value and trying to figure out what sticks when they throw it against the wall: no wonder they end up frustrated, and it's a frustrating situation. Given: def foo(*args): print(args) return class Thing(object): def __init__(self,a,b): self.a = a self.b = b return def foo_style(self): return (self.a, self.b) args is not required to refer to a tuple: >>> foo(*[31,42]) (31, 42) I can have objects construct parameters conforming to the specifications for a signature: >>> foo(*Thing(3,91).foo_style()) (3, 91) Consider that a counterexample.
- deleted 3y ago[deleted]
- adamchainz 3y agoWithin the function, args is a tuple, as your output demonstrates.
- curiousgal 3y agoIs it just me or are Python type hints like..goofy?
- b5n 3y agoCall me crazy, but I just use a statically typed language where static types are required.
- lijok 3y agoBig time. Getting better very quickly however
- PheonixPharts 3y agoAs someone who has written Python for nearly 20 years now, and also has plenty of experience with strongly and statically typed languages (including a fair bit of Haskell), I think type hints in Python should at most remain just that, hints. A language being statically typed or dynamically typed is a design decision with implications for what the language can do. There are benefits to each method of programming. Trying to strap type checking on to Python is born out of some misplaced belief that static type is just better. Using Python as a dynamically typed language allows you for certain programming patterns that cannot be done in a statically typed language. There are some great examples in SICP of Scheme programs that could not exist (at least with as much elegance) in a typed language. Dynamic typing allows a language to do things that you can't do (as easily/elegantly) in a statically typed language. Some may argue that these type of programming patterns are bad for production systems. For most of these arguments I strongly agree. But that means for those systems Python is probably a poor choice. I also think metaprogramming is very powerful, but also a real potential footgun. It would be ridiculous to attempt to strip metaprogramming from Ruby to make it "better", just use a language that depends less on metaprogramming if you don't like it. This is extra frustrating because in the last decade we've seen the options for well designed, statically typed languages explode. It's no longer Python vs Java/C++. TypeScript and Go exist, have great support and are well suited for most of the domains that Python is. Want types? Use those languages.
- mirsadm 3y ago
- amelius 3y agoWhat if the second argument is a float?
- throwaway2037 3y agoNothing. Type hints are only paint on the bike shed. By default, there is no function in the language, e.g., raise exception for incorrect type.