10 ms·
Type checking in Python
- fx5 12y agoAppreciate that it has a "logging only" mode. I wish there was something like this, but parsing the docstrings with the same format as pycharm: http://www.jetbrains.com/pycharm/webhelp/type-hinting-in-pycharm.html http://www.jetbrains.com/pycharm/webhelp/type-hinting-in-pyc...
- senko 12y agoI'm currently entertaining the idea of it outputting docstrings (adding :param:, :rtype: and other applicable tags if they're not already there).
- artursapek 12y agoThis is neat, but why not just use Go!
- skj 12y agoI am an avid gopher, but this is not a useful response. Believe it or not, there are many places and reasons where Python works out better than Go, and making it a bit easier to use Python in those situations is valuable.
- pekk 12y agoGo is neat, but why not just use Haskell? (etc.)
- clubhi 12y agoHaskell is ok, but you should check out the dynamic nature of Python
- jbeja 12y agoI think that the dynamic nature of python is why some folks prefer haskell over it.
- Dewie 12y agoYou should check out the dynamic nature of Typeable. "The module Data.Dynamic uses Typeable for an implementation of dynamics." http://hackage.haskell.org/package/base-4.7.0.0/docs/Data-Typeable.html http://hackage.haskell.org/package/base-4.7.0.0/docs/Data-Ty...
- raverbashing 12y agoMonads Or more specifically, how non-intuitive it is to use the IO stuff I don't care what it uses, or what is it called, I care about being able to use it with what I know So yeah, I'll go for Go instead of Haskell
- Peaker 12y agoThe IO stuff has almost nothing to do with monads. It is unfamiliar, but if you use static typing, at least reap the benefits of better error checking, no runtime null dereferences, etc.
- frowaway001 12y agoYeah, because why would we ever need the compiler to help us identifying operations which make it harder to reason about our programs!
- uniclaude 12y agoThe case of a large, already existing Python codebase comes to mind.
- smnrchrds 12y agoYou make Python sound like FORTRAN. :)
- yen223 12y agoYou're getting downvoted, but I think you have a point. Any attempts to enact type-checking in a language that doesn't require it is doomed to failure. You'll get all the safety of a dynamic language, with the flexibility of a typed language.
- frowaway001 12y ago> You'll get all the safety of a dynamic language, with the flexibility of a typed language. That sounds a pretty good description of Go, so I'm wondering what his point is in the first place.
- _random_ 12y agoStatic-typed languages are not very good for scripting and "gluing" systems.
- ant_sz 12y agoerr...It's a tradeoff. I think when the first dynamic programing language was invented, the loose typing system must be thought as a big advantage. Type checking is necessary in some cases, but I really enjoy dynamic typing a lot. So I think a bit tradeoff like this is acceptable. After all, we have to tradeoff everywhere when it comes to computer science such as the time-space tradeoff of an algorithm.
- Peaker 12y agoI think people who enjoy dynamic typing have had bad experiences with bad static type systems earlier (C, C++, Java). Once you use a good static type system (ML, OCaml, Haskell) you see that dynamic typing isn't responsible for the joy, but the ability to express rich ideas without repeating redundant type declarations.
- hcarvalhoalves 12y agoI would like to know why in every post about Python on HN there's someone shouting "just use Go"!
- Dewie 12y agoIt seems like quite a few people who are used to dynamic languages have found out that static typing and verbosity aren't strictly synonyms by using Go.
- csirac2 12y agoThis is the single biggest thing I miss coming from modern Perl. Pythonistas seem very anti-type validation/checking, but with Moose classes the type stuff simplifies a lot, is declarative, self documenting, and vastly reduces certain classes of bugs. The closest I've seen is the traits thing from Enthought, but there doesn't seem to be much buy-in from python users.
- hershel 12y agoThrees also the mypy project which offers optional static typing in python, if you're interested in such stuff.
- pekk 12y agoIf you write APIs which only accept this one type, then you are explicitly disallowing people from using types that are functionally equivalent as far as the API is concerned. Why should you do that?
- csirac2 12y agoThat's not how these trait/type systems work.
- doctoboggan 12y agoFormer Enthought employee here, I always liked programming with Traits. Although nowadays I use the similar but much for light weight atom[0], which was created by another former Enthoughter. [0]: https://github.com/nucleic/atom https://github.com/nucleic/atom
- csirac2 12y agoAwesome pointer, thanks!
- willvarfar 12y agoI've been checking types in Python dynamically for years now. My library, Obiwan https://pypi.python.org/pypi/obiwan/1.0.2 https://pypi.python.org/pypi/obiwan/1.0.2 uses type annotations, which I think nicer than decorators.
- andybak 12y agoAs the article mentions - the big advantage of decorators is Python 2.x compatibility. (not that I want to start another discussion of THAT topic) EDIT - looks like obiwan will allow decorators as well as annotations. I do prefer the syntax of typedecorator though. It seems less cluttered.
- andreasvc 12y agoIt's a matter of taste. I think the decorator approach looks more bolted-on and ad hoc. I prefer C style declarations over annotations, though, because they have the least amount of noise.
- jbeja 12y agoThis actually looks pretty good.
- xiaq 12y agoThis module doesn't seem to make it possible to type-annotate map and filter, which are Python builtins. I believe that dynamic typing results from frustration over generics, rather than over static typing in general. (Type stuttering is another problem, but modern languages solve it with local type inference.) I mean, annotating non-generic functions is always straightforward doesn't introduce much complexity: # s should be str # returns nothing def print(s): ... # just becomes (using Py3 annotation syntax) def print(s: str) -> None: ... However, it's much harder to write a type annotation of generic functions. Worse, remember that Python's map is variadic. # for any types t0, t1, t2, ... tn: # fn is a function that takes arguments of types t0, t1, ... t(n-1) # and returns a value of type tn; # seqs are iterables of types t0, t1, ... t(n-1); # returns a list of type tn. def map(fn, *seqs): ... # How to write this signature is non-obvious Many mature static typing systems would allow you to express such types (usually called parameterized types). But any one of them would require more than those trivial notations in print. That's when the dynamic typing people get annoyed and go "fxxk static typing systems, I can handle this in my mind". Which is about 65% (totally random estimation) the point of dynamic typing in my opinion. Any type checker for dynamic typed languages that doesn't seriously try to solve the generics problem is not genuinely interesting.
- pekk 12y agoThere is nothing special about map and filter, they are not idiomatic Python because there are better ways to do the same things. Python 3 annotations are already not used in any specific predetermined way, so if you create a system to do something with type annotations then you can implement List(str) or whatever it is you want an annotation to work like.
- xiaq 12y ago> There is nothing special about map and filter, they are not idiomatic Python because there are better ways to do the same things. True, list comprehension is often used instead. But a complete typing system should be able to type-check that too. The point is not changed. Also, I only used map and filter as an example. Python programmers tend to write a lot of generics functions without actually knowing it. def f(g, *args): ... r = g(*args) ... return r This function is actually quite generic, akin to map. And we are not yet getting into the realm of "trait" or "concept" or "typeclass" or whatever. def sum(*nums): s = 0 for n in nums: s += n return s Remember, although s is definitely an int, s + n may yield some completely unrelated type if n implements __radd__. The function here involves potentially infinite number of types, and their __add__ or __radd__ traits (reusing the name of the Python magic method). To fully express its type without unnecessary constraints is an interesting exercise. Since Python allows dynamic parameter list building (like sum(*nums)), dependent typing may be necessary for that purpose.
- petrounias 12y agoUnion types would be an incredibly useful extension of this.
- illumen 12y agoIndeed! pysonar2 uses them with success.
- bsaul 12y agoDidn't know Guido thought of static type checking such a long time ago. I'm thinking now that one solution to two problems (this one, and slow python 3 adoption) would have been to include optional static typing (like in dart) in python 3.
- markbnj 12y agoAs a somewhat new python programmer (~2 years) coming from statically compiled languages like C#, C++ and Pascal this is something I have thought about a lot. My initial impulse was to wish I had static type checking. But I have come to the conclusion that I just hadn't fully appreciated the differences between interpreted and compiled languages. The article says that the lack of type checking "lets through" a certain class of errors. However, let's be specific here: there is no build-time static analysis phase in the transformation of python source to executable code. So "lets through" means the same thing whether you have strong typing or not: the error is going to be found at runtime. What we're really talking about, then, is converting AttributeError ("YourObject has no attribute 'startswith'") into something more specific ("Hey, this is supposed to be a string!"). Honestly, that seems to me to be a pretty minor increase in diagnostic information. So the bottom line for me is that I feel I didn't really understand duck typing. I used to joke that the term actually meant "no typing," and that is in some respects true. But what it really means is "the thing can do what the method expects it to be able to do." If the thing can't serve in the expected role, then the existing errors that result are sufficiently explanatory, imo.
- Peaker 12y agoThe issue is about when the error is discovered. You can have "duck typing" in Haskell, for example, using type-classes. Instead of "letting the errors through" to run-time, you detect them at compile-time.
- markbnj 12y agoYes, but I am talking about python as it is now, not what it might be if it combined the features of other languages. I don't know anything beyond the most basic characteristics of Haskell, so I can't really comment further.
- dbpatterson 12y agoObviously there are some differences, but type classes are pretty similar to the way python normally works - type classes specify methods that a type has to have. So in the case of 'print', the type specifies that print can be called on anything that implements the method that turns a value into a string. The neat thing, of course, is that if you try to call it on something that doesn't implement that method, you get a compile error (and with proper tooling, I actually see red squiggles in my editor if I make this mistake, which is about as good as it can get).
- cabalamat 12y agoI wrote a similar python typechecker some time ago (https://github.com/cabalamat/ulib/blob/master/debugdec.md https://github.com/cabalamat/ulib/blob/master/debugdec.md). My syntax is somewhat less verbose so this: @returns(int) @params(a=int, b=int) def add(a, b): return a + b becomes: @typ(int, int, ret=int) def add(a, b): return a + b I was inspired to do it like that by Haskell's very clean type syntax. Senko's version has the advantage that you can compose types in it, e.g. {str:int} is a dictionary whose keys are strings and values are integers.
- raymondh 12y agoHaving seen many type-checking initiatives for Python come and go over the years, I've come to believe that type-checking is like security in that it is difficult to add after-the-fact and works best when designed-in from day one. When function annotations were introduced, I ended up removing all their uses from the standard library because every early attempt to use it was too simplistic and failed to support any type-system use case except for extra documentation.
- illumen 12y agoI recently wrote this post "statically type checking python" http://renesd.blogspot.de/2014/05/statically-checking-python.html http://renesd.blogspot.de/2014/05/statically-checking-python... Turns out with modern tools you _can_ statically type check python :)
- nox_ 12y agoToo bad that this isn't type checking in the traditional sense though. That's just runtime tag checking.
- ak217 12y agoFor Python 3, I've added optional type checking based on function annotations and decorators to the Ensure library: https://github.com/kislyuk/ensure#enforcing-function-annotations https://github.com/kislyuk/ensure#enforcing-function-annotat.... @ensure_annotations def f(x: int, y: float) -> float: return x+y f(1, 2.3) >>> 3.3 f(1, 2) >>> ensure.EnsureError: Argument y to <function f at 0x109b7c710> does not match annotation type <class 'float'> I think it works better than other approaches mentioned here because it (1) doesn't repeat the contents of the function signature, (2) doesn't install any magic system-wide hooks (and the attendant performance issues), and (3) is completely optional.
- anon4 12y agoThere's one thing that doesn't sit well with me about adding type-checking to the CPython implementation of the Python language. Type-checking is generally used at compile-time to check that your program won't do nasty things, and then run-time checks are done only when you do something overly dynamic that the compiler couldn't check (i.e. casting an Object to some concrete class in Java). However, all type-checking implementations for CPython work purely at runtime! And type-checks are invoked each time your function is called! This seems not only wasteful, but also defeats the purpose of type-checking before running your program. I would rather see a more generic pre/post-condition contract system, to be used in select top-level functions, that gives a lot more flexibility in expressing what is supposed to happen (i.e. I expect to be given a not-None value with a __str__ method that doesn't throw and will return an iterator yielding consecutive not-empty, not-None strings), including what assumptions you're making when you call a function that you got from a random object (i.e. here I'm calling a function that I got somehow as a parameter and I'm going to assume it can take a string of the format "ipv4 address:port" and returns an object that is a database connection), together with a mechanism for recovering from broken contracts - we are doing runtime checks, so there's no reason to restrict ourselves to Java-style type constraints, which don't even work for Python in general. Or, alternatively, a lighter system that can be checked at import time once before the program is started "for real". Preferably both of those.