5 ms·

Comparing Python Type Checkers: Typing Spec Conformance

119 points by ocamoss 7mo ago

persedes 7mo ago
Article is a nice write up of https://htmlpreview.github.io/?https://github.com/python/typing/blob/main/conformance/results/results.html https://htmlpreview.github.io/?https://github.com/python/typ... (glad they include ty now)
Pay08 7mo ago
I still can't get over the utter idiocy in Python's type hints being decorative. In what world does x: int = "thing" not give someone in the standardisation process pause?
- dcreager 7mo ago
  Can you elaborate what you mean by decorative? If you run a type checker like ty or pyright they're not decorative — you'll get clear diagnostics for that particular example [1], and any other type errors you might have. You can set up CI so that e.g. blocks PRs from being merged, just like any other test failure. If you mean types not being checked at runtime, the consensus is that most users don't want to pay the cost of the checks every time the program is run. It's more cost-effective to do those checks at development/test/CI time using a type checker, as described above. But if you _do_ want that, you can opt in to that using something like beartype [2]. [1] https://play.ty.dev/905db656-e271-4a3a-b27d-18a4dd45f5da https://play.ty.dev/905db656-e271-4a3a-b27d-18a4dd45f5da [2] https://github.com/beartype/beartype/ https://github.com/beartype/beartype/
  - Pay08 7mo ago
    Exactly my point. If I need to run 300 external tools for a language feature to be worth a damn, why is it a language feature?
    - wiseowise 7mo ago
      ‘uvx ty check’ or ‘uvx pyrefly check’. That’s hardly 300 external tools.
- badlibrarian 7mo ago
  It's a community that delayed progress for a decade while they waited for everyone to put parenthesis on the print statement. Give 'em enough time and they'll figure out best practices.
- Spivak 7mo ago
  In C-ish languages the statement int x = "thing" is perfectly valid. It means reserve a spot for a 32 bit int and then shove the pointer to the string "thing" at the address of x. It will do the wrong thing and also overflow memory but you could generate code for it. The type checker is what stops you. It's the same in Python, if you make type checking a build breaker then the annotations mean something. Types aren't checked at runtime but C doesn't check them either.
  - lefra 7mo ago
    In C, int may be as small as 16 bits You may get 32 bits (or more) but it's not guaranteed. I don't see how you get a memory overflow though? I'd be surprised if a compiler with -Wall -Werror accepts to compile this. Trying to cast back the int to a char* might work if the pointers are the same size as int on the target platform, but it's actually Undefined Behaviour IIRC.
    - Pay08 7mo ago
      I guess an overflow would be possible if the size of a point and int differs.
  - Pay08 7mo ago
    It's valid in C, due to semantics around pointers. Try that in Java and you'll quickly find that it's not valid in "C-ish languages". C absolutely checks types, it's just weakly typed. Python doesn't check types at all, which I wouldn't have a problem with, if the language didn't have type annotations that sure look like they'll do something.
  - tialaramex 7mo ago
    It won't "overflow memory". This says there will be an immutable array of six bytes, with the ASCII letters for "thing" in the first five and then the sixth is zero, this array can be coerced to the pointer type char* (a pointer to bytes) and then (though a modern C compiler will tell you this is a terrible idea) coerced to the signed integer type int. The six byte array will end up in the "read only data" section of the executable, it doesn't "overflow memory" and isn't stored in the x. Even if you gave x a more sensible type "char*" that word "thing" isn't somehow stored in your variable, it's a pointer. So, this isn't the same at all and you don't understand C as well as you thought you did. Edited: fix escaping bold markers
- Daishiman 7mo ago
  It's the complete opposite. The objective of type hints is that they're optional precisely because type hints narrow the functionality of the language. And evidenced by the fact that different type checks have different heuristics for determining what is a valid typed program and what isn't, it seems that the decision is correct. No type system will allow for the dynamism that Python supports. It's not a question of how you annotate types, it's about how you resolve types.
  - hrmtst93837 7mo ago
    [flagged]
  - Pay08 7mo ago
    Nobody is saying they are mandatory, and I'm actually a big fan of gradual typing. My point is that they do nothing. However, type hints reducing the functionality of the language isn't true either.
martinky24 7mo ago
I've been using ty on some previously untyped codebases at work. It does a good job of being fast and easy to use while catching many issues without being overly draconian. My teammates who were writing untyped Python previously don't seem to mind it. It's a good addition to the ecosystem!
- tfrancisl 7mo ago
  And it makes it infinitely easier for them to get with the times and start typing their code!
- rirze 7mo ago
  I am worried about the false negatives/positive rate however. Hope it improves.
  - notatallshaw 7mo ago
    My understand is Astral's focus for ty has been on making a good experience for common issues, whereas they plan for very high compliance but difficult or rare edge cases aren't are prioritized. Compliance suite numbers are biased towards edge cases and not the common path because that's where a lot of the tests need to be added. My advise is to see how each type checker runs against your own codebase and if the output/performance is something you are happy with.
    - dcreager 7mo ago
      > My understand is Astral's focus for ty has been on making a good experience for common issues, whereas they plan for very high compliance but difficult or rare edge cases aren't are prioritized. I would say that's true in terms of prioritization (there's a lot to do!), but not in terms of the final user experience that we are aiming for. We're not planning on punting on anything in the conformance suite, for instance.
ddxv 7mo ago
I've used mypy forever and never even tried these others. Looking at them though it looks like it's worth trying out Zuban or Pyright? Is there a noticeable benefit when switching between different checkers?
- x187463 7mo ago
  Speed, especially in larger codebases.
- rirze 7mo ago
  If you care about correctness, unless you pick pyright, don't bother at the moment. If you're creating a new project and looking for a promise for better faster typing, then pick one of Zuban, Pyrefly, or ty.
  - fastasucan 7mo ago
    Isnt there correctness with Zuban?
- winrid 7mo ago
  Mypy still best for Django
  - lastofus 7mo ago
    As a long time Django user that wants to start using typing, can you elaborate on why mypy is still the way to go?
    - vesselapi 7mo ago
      [dead]
    - infamia 7mo ago
      There's a really nice typing plug-in for mypy that's been around a long time: https://github.com/typeddjango/django-stubs https://github.com/typeddjango/django-stubs It is very disappointing that these new type checkers don't support plug-ins, so things like django-stubs aren't possible. That means you're stuck with whatever is delivered with these new type checkers. It must be really difficult since none of them support plug-ins. Some of these newer type checkers promise support for Django, but you're stuck with what they (will) have on offer. Also, you'll likely want typing for other libs you might use.
Scene_Cast2 7mo ago
Are there any good static (i.e. not runtime) type checkers for arrays and tensors? E.g. "16x64x256 fp16" in numpy, pytorch, jax, cupy, or whatever framework. Would be pretty useful for ML work.
- dcreager 7mo ago
  There have been some early proposals to add something like that, but none of them have made it very far yet. As you might imagine, it's a hard problem!
- ocamoss 7mo ago
  We're working on statically checking Jaxtyping annotations in Pyrefly, but it's incomplete and not ready to use yet :)
  - ainch 7mo ago
    This would be an insta-switch feature for me! Jaxtyping is a great idea, but the runtime-only aspect kills it for me - I just resort to shape assertions + comments, but it's a pretty poor solution. A follow-up question: Google's old `tensor_annotations` library (RIP) could statically analyse operations - eg. `reduce_sum(Tensor[Time, Batch], axis=0) -> Tensor[Batch]`. I guess that wouldn't come with static analysis for jaxtyping?
- deleted 7mo ago
  [deleted]
- westurner 7mo ago
  - /?hnlog pycontract icontract https://westurner.github.io/hnlog/ https://westurner.github.io/hnlog/ : From https://news.ycombinator.com/item?id=14246095 https://news.ycombinator.com/item?id=14246095 (2017) : > PyContracts supports runtime type-checking and value constraints/assertions (as @contract decorators, annotations, and docstrings). > Unfortunately, there's yet no unifying syntax between PyContracts and the newer python type annotations which MyPy checks at compile-type. Or beartype. Pycontracts has: https://andreacensi.github.io/contracts/ https://andreacensi.github.io/contracts/ : @contract def my_function(a : 'int,>0', b : 'list[N],N>0') -> 'list[N]': @contract(image='array[HxWx3](uint8),H>10,W>10') def recolor(image): For icontract, there's icontract-hyothesis. parquery/icontract: https://github.com/Parquery/icontract https://github.com/Parquery/icontract : > There exist a couple of contract libraries. However, at the time of this writing (September 2018), they all required the programmer either to learn a new syntax (PyContracts) or to write redundant condition descriptions ( e.g., contracts, covenant, deal, dpcontracts, pyadbc and pcd). @icontract.require(lambda x: x > 3, "x must not be small") def some_func(x: int, y: int = 5) -> None: icontract with numpy array types: @icontract.require(lambda arr: isinstance(arr, np.ndarray)) @icontract.require(lambda arr: arr.shape == (3, 3)) @icontract.require(lambda arr: np.all(arr >= 0), "All elements must be non-negative") def process_matrix(arr: np.ndarray): return np.sum(arr) invalid_matrix = np.array([[1, -2, 3], [4, 5, 6], [7, 8, 9]]) process_matrix(invalid_matrix) # Raises icontract.ViolationError
extr 7mo ago
Wow, quite surprising results. I have been working on a personal project with the astral stack (uv, ruff, ty) that's using extremely strict lint/type checking settings, you could call it an experiment in setting up a python codebase to work well with AI. I was not aware that ty's gaps were significant. I just tried with zuban + pyright. Both catch a half dozen issues that ty is ignoring. Zuban has one FP and one FN, pyright is 100% correct. Looks like I will be converting to pyright. No disrespect to the astral team, I think they have been pretty careful to note that ty is still in early days. I'm sure I will return to it at some point - uv and ruff are excellent.
- roflcopter69 7mo ago
  This is the way. For now, pyright it's also 100% pyright for me. I can recommend turning on reportMatchNotExhaustive if you're into Python's match statements but would love the exhaustiveness check you get in Rust. Eric Traut has done a marvellous job working on pyright, what a legend! But don't get me wrong, I made an entry in my calendar to remind me of checking out ty in half a year. I'm quite optimistic they will get there.
  - p1necone 7mo ago
    Say what you will about Microsoft, but their programming language people consistently seem to make very solid decisions.
    - zdimension 7mo ago
      Microsoft started as a programming language company (MS-BASIC) and they never stopped delivering serious quality software there. VB (classic), for all its flaws, was an amazing RAD dev product. .NET, especially since the move to open-source, is a great platform to work with. C# and TS are very well-designed languages. Though they still haven't managed to produce a UI toolkit that is both reliable, fast, and easy to use.
  - athoscouto 7mo ago
    For big codebases pyright can be pretty slow and memory hungry. Even though ty is still a WIP, I'm adopting it at work because of how fast it is and some other goodies (e.g. https://docs.astral.sh/ty/features/type-system/#intersection-types https://docs.astral.sh/ty/features/type-system/#intersection...)
pgwalsh 7mo ago
Using VSCodium I was having issues with python type checkers for quite a while. I did the basedpyright thing for a while but that was painful. It's a bit too based for me, and I'm not sure i'd call it based. Right now I have uv, ruff, and ty and I'm happy with it. It's super easy to update and super fast. I didn't realize the coverage wasn't as good as some others but I still like it. I may have to try pyrefly. Never heard of it until this post, so thank you.
IshKebab 7mo ago
Interesting. This is the first I've heard of Zuban. The fact that Mypy fails so badly matches my experience. It would be interesting to see exactly where Pyright "fails". It's been so reliable to me I wouldn't be 100% surprised if these are deliberate deviations from the spec, where it is dumb.
- ocamoss 7mo ago
  Pyright had 100% conformance until recently. There was a new PEP that was accepted which required some new conformance tests, which no type checker has added support for yet.
- networked 7mo ago
  I think Zuban's license limited its exposure. I knew about Zuban but didn't pay attention to it. A proprietary development dependency was a no-go for my FLOSS projects, and I didn't want to adopt a separate tool just for proprietary code. I see that Zuban went AGPL in September 2025 (with exceptions available). This makes it a lot more interesting.
refactor_master 7mo ago
How does Zuban manage to be developed by what appears to be a single person without megacorp backing, yet be mere inches behind pyright at this stage?
Neywiny 7mo ago
This is great and I'll try out pyright ASAP on my current codebase. The people who wrote it evidently didn't have any type checking running (despite I think 3+ linters??) so it's a nightmare of > "well the checker accurately reports it will be type X in an error case not Y" > "but we never get type X" > "Then we don't have good enough coverage" It's so easy in vscode, but it isn't on by default like the c/c++ one I guess because too much legacy code would cause infinite errors. And the age old problem of .pyi files lying about types.
nas 7mo ago
Just an FYI, for people looking at the low pass rates for mypy and ty and concluding they must not be very useful. These test suites are checking many odd corners of the typing spec. For "normal" Python code, I find mypy does pretty good. Certainly I find it helpful, especially on a large code base and when working with other developers of various experience levels. The reason I prefer pyrefly over mypy is mostly because of speed. Better accuracy is nice but speed it the killer feature. Given the quality of uv and ruff and the experience of the team working on ty, I'm quite confident it's going to be great in that respect as well.
- bko 7mo ago
  I dont know, I've seen some truly simple things check out as ok in mypy Example def foo(bar: bool) -> bool: if bar: m = True return m No error that m is defined conditionally? What's going on?
  - maleldil 7mo ago
    ty and zuban also don't give an error. pyright and pyrefly do.
    - _carljm 7mo ago
      We provide this diagnostic in ty (https://docs.astral.sh/ty/reference/rules/#possibly-unresolved-reference https://docs.astral.sh/ty/reference/rules/#possibly-unresolv...), but it's disabled by default because it can have false positives in many scenarios where the code works at runtime (this is true also in the type checkers that enable it by default). A typical example is code like def _(flag: bool): if flag: x = True ... if flag: read(x) But it's easy to turn the rule on if you want it: [tool.ty.rules] possibly-unresolved-reference = "error"
- ocamoss 7mo ago
  > people looking at the low pass rates for mypy and ty and concluding they must not be very useful Yeah, that would be the wrong takeaway from this blog. The point of the blog was to add context to what the conformance results mean and clarify their limitations, since I saw quite a few people sharing links to the tracker online w/o context.
JackSlateur 7mo ago
My favorite typing-related stuff is typeguard (https://pypi.org/project/typeguard/ https://pypi.org/project/typeguard/), with its pytest plugin

119 points by ocamoss 7mo ago

Article is a nice write up of https://htmlpreview.github.io/?https://github.com/python/typing/blob/main/conformance/results/results.html https://htmlpreview.github.io/?https://github.com/python/typ... (glad they include ty now)

Pay08 7mo ago

I still can't get over the utter idiocy in Python's type hints being decorative. In what world does x: int = "thing" not give someone in the standardisation process pause?

dcreager 7mo ago
Can you elaborate what you mean by decorative? If you run a type checker like ty or pyright they're not decorative — you'll get clear diagnostics for that particular example [1], and any other type errors you might have. You can set up CI so that e.g. blocks PRs from being merged, just like any other test failure. If you mean types not being checked at runtime, the consensus is that most users don't want to pay the cost of the checks every time the program is run. It's more cost-effective to do those checks at development/test/CI time using a type checker, as described above. But if you _do_ want that, you can opt in to that using something like beartype [2]. [1] https://play.ty.dev/905db656-e271-4a3a-b27d-18a4dd45f5da https://play.ty.dev/905db656-e271-4a3a-b27d-18a4dd45f5da [2] https://github.com/beartype/beartype/ https://github.com/beartype/beartype/
- Pay08 7mo ago
  Exactly my point. If I need to run 300 external tools for a language feature to be worth a damn, why is it a language feature?
  - wiseowise 7mo ago
    ‘uvx ty check’ or ‘uvx pyrefly check’. That’s hardly 300 external tools.
badlibrarian 7mo ago
It's a community that delayed progress for a decade while they waited for everyone to put parenthesis on the print statement. Give 'em enough time and they'll figure out best practices.
Spivak 7mo ago
In C-ish languages the statement int x = "thing" is perfectly valid. It means reserve a spot for a 32 bit int and then shove the pointer to the string "thing" at the address of x. It will do the wrong thing and also overflow memory but you could generate code for it. The type checker is what stops you. It's the same in Python, if you make type checking a build breaker then the annotations mean something. Types aren't checked at runtime but C doesn't check them either.
- lefra 7mo ago
  In C, int may be as small as 16 bits You may get 32 bits (or more) but it's not guaranteed. I don't see how you get a memory overflow though? I'd be surprised if a compiler with -Wall -Werror accepts to compile this. Trying to cast back the int to a char* might work if the pointers are the same size as int on the target platform, but it's actually Undefined Behaviour IIRC.
  - Pay08 7mo ago
    I guess an overflow would be possible if the size of a point and int differs.
- Pay08 7mo ago
  It's valid in C, due to semantics around pointers. Try that in Java and you'll quickly find that it's not valid in "C-ish languages". C absolutely checks types, it's just weakly typed. Python doesn't check types at all, which I wouldn't have a problem with, if the language didn't have type annotations that sure look like they'll do something.
- tialaramex 7mo ago
  It won't "overflow memory". This says there will be an immutable array of six bytes, with the ASCII letters for "thing" in the first five and then the sixth is zero, this array can be coerced to the pointer type char* (a pointer to bytes) and then (though a modern C compiler will tell you this is a terrible idea) coerced to the signed integer type int. The six byte array will end up in the "read only data" section of the executable, it doesn't "overflow memory" and isn't stored in the x. Even if you gave x a more sensible type "char*" that word "thing" isn't somehow stored in your variable, it's a pointer. So, this isn't the same at all and you don't understand C as well as you thought you did. Edited: fix escaping bold markers
Daishiman 7mo ago
It's the complete opposite. The objective of type hints is that they're optional precisely because type hints narrow the functionality of the language. And evidenced by the fact that different type checks have different heuristics for determining what is a valid typed program and what isn't, it seems that the decision is correct. No type system will allow for the dynamism that Python supports. It's not a question of how you annotate types, it's about how you resolve types.
- hrmtst93837 7mo ago
  [flagged]
- Pay08 7mo ago
  Nobody is saying they are mandatory, and I'm actually a big fan of gradual typing. My point is that they do nothing. However, type hints reducing the functionality of the language isn't true either.

martinky24 7mo ago

I've been using ty on some previously untyped codebases at work. It does a good job of being fast and easy to use while catching many issues without being overly draconian. My teammates who were writing untyped Python previously don't seem to mind it. It's a good addition to the ecosystem!

tfrancisl 7mo ago
And it makes it infinitely easier for them to get with the times and start typing their code!
rirze 7mo ago
I am worried about the false negatives/positive rate however. Hope it improves.
- notatallshaw 7mo ago
  My understand is Astral's focus for ty has been on making a good experience for common issues, whereas they plan for very high compliance but difficult or rare edge cases aren't are prioritized. Compliance suite numbers are biased towards edge cases and not the common path because that's where a lot of the tests need to be added. My advise is to see how each type checker runs against your own codebase and if the output/performance is something you are happy with.
  - dcreager 7mo ago
    > My understand is Astral's focus for ty has been on making a good experience for common issues, whereas they plan for very high compliance but difficult or rare edge cases aren't are prioritized. I would say that's true in terms of prioritization (there's a lot to do!), but not in terms of the final user experience that we are aiming for. We're not planning on punting on anything in the conformance suite, for instance.

ddxv 7mo ago

I've used mypy forever and never even tried these others. Looking at them though it looks like it's worth trying out Zuban or Pyright? Is there a noticeable benefit when switching between different checkers?

x187463 7mo ago
Speed, especially in larger codebases.
rirze 7mo ago
If you care about correctness, unless you pick pyright, don't bother at the moment. If you're creating a new project and looking for a promise for better faster typing, then pick one of Zuban, Pyrefly, or ty.
- fastasucan 7mo ago
  Isnt there correctness with Zuban?
winrid 7mo ago
Mypy still best for Django
- lastofus 7mo ago
  As a long time Django user that wants to start using typing, can you elaborate on why mypy is still the way to go?
  - vesselapi 7mo ago
    [dead]
  - infamia 7mo ago
    There's a really nice typing plug-in for mypy that's been around a long time: https://github.com/typeddjango/django-stubs https://github.com/typeddjango/django-stubs It is very disappointing that these new type checkers don't support plug-ins, so things like django-stubs aren't possible. That means you're stuck with whatever is delivered with these new type checkers. It must be really difficult since none of them support plug-ins. Some of these newer type checkers promise support for Django, but you're stuck with what they (will) have on offer. Also, you'll likely want typing for other libs you might use.

Scene_Cast2 7mo ago

Are there any good static (i.e. not runtime) type checkers for arrays and tensors? E.g. "16x64x256 fp16" in numpy, pytorch, jax, cupy, or whatever framework. Would be pretty useful for ML work.

dcreager 7mo ago
There have been some early proposals to add something like that, but none of them have made it very far yet. As you might imagine, it's a hard problem!
ocamoss 7mo ago
We're working on statically checking Jaxtyping annotations in Pyrefly, but it's incomplete and not ready to use yet :)
- ainch 7mo ago
  This would be an insta-switch feature for me! Jaxtyping is a great idea, but the runtime-only aspect kills it for me - I just resort to shape assertions + comments, but it's a pretty poor solution. A follow-up question: Google's old `tensor_annotations` library (RIP) could statically analyse operations - eg. `reduce_sum(Tensor[Time, Batch], axis=0) -> Tensor[Batch]`. I guess that wouldn't come with static analysis for jaxtyping?
deleted 7mo ago
[deleted]
westurner 7mo ago
- /?hnlog pycontract icontract https://westurner.github.io/hnlog/ https://westurner.github.io/hnlog/ : From https://news.ycombinator.com/item?id=14246095 https://news.ycombinator.com/item?id=14246095 (2017) : > PyContracts supports runtime type-checking and value constraints/assertions (as @contract decorators, annotations, and docstrings). > Unfortunately, there's yet no unifying syntax between PyContracts and the newer python type annotations which MyPy checks at compile-type. Or beartype. Pycontracts has: https://andreacensi.github.io/contracts/ https://andreacensi.github.io/contracts/ : @contract def my_function(a : 'int,>0', b : 'list[N],N>0') -> 'list[N]': @contract(image='array[HxWx3](uint8),H>10,W>10') def recolor(image): For icontract, there's icontract-hyothesis. parquery/icontract: https://github.com/Parquery/icontract https://github.com/Parquery/icontract : > There exist a couple of contract libraries. However, at the time of this writing (September 2018), they all required the programmer either to learn a new syntax (PyContracts) or to write redundant condition descriptions ( e.g., contracts, covenant, deal, dpcontracts, pyadbc and pcd). @icontract.require(lambda x: x > 3, "x must not be small") def some_func(x: int, y: int = 5) -> None: icontract with numpy array types: @icontract.require(lambda arr: isinstance(arr, np.ndarray)) @icontract.require(lambda arr: arr.shape == (3, 3)) @icontract.require(lambda arr: np.all(arr >= 0), "All elements must be non-negative") def process_matrix(arr: np.ndarray): return np.sum(arr) invalid_matrix = np.array([[1, -2, 3], [4, 5, 6], [7, 8, 9]]) process_matrix(invalid_matrix) # Raises icontract.ViolationError

extr 7mo ago

Wow, quite surprising results. I have been working on a personal project with the astral stack (uv, ruff, ty) that's using extremely strict lint/type checking settings, you could call it an experiment in setting up a python codebase to work well with AI. I was not aware that ty's gaps were significant. I just tried with zuban + pyright. Both catch a half dozen issues that ty is ignoring. Zuban has one FP and one FN, pyright is 100% correct. Looks like I will be converting to pyright. No disrespect to the astral team, I think they have been pretty careful to note that ty is still in early days. I'm sure I will return to it at some point - uv and ruff are excellent.

roflcopter69 7mo ago
This is the way. For now, pyright it's also 100% pyright for me. I can recommend turning on reportMatchNotExhaustive if you're into Python's match statements but would love the exhaustiveness check you get in Rust. Eric Traut has done a marvellous job working on pyright, what a legend! But don't get me wrong, I made an entry in my calendar to remind me of checking out ty in half a year. I'm quite optimistic they will get there.
- p1necone 7mo ago
  Say what you will about Microsoft, but their programming language people consistently seem to make very solid decisions.
  - zdimension 7mo ago
    Microsoft started as a programming language company (MS-BASIC) and they never stopped delivering serious quality software there. VB (classic), for all its flaws, was an amazing RAD dev product. .NET, especially since the move to open-source, is a great platform to work with. C# and TS are very well-designed languages. Though they still haven't managed to produce a UI toolkit that is both reliable, fast, and easy to use.
- athoscouto 7mo ago
  For big codebases pyright can be pretty slow and memory hungry. Even though ty is still a WIP, I'm adopting it at work because of how fast it is and some other goodies (e.g. https://docs.astral.sh/ty/features/type-system/#intersection-types https://docs.astral.sh/ty/features/type-system/#intersection...)

pgwalsh 7mo ago

Using VSCodium I was having issues with python type checkers for quite a while. I did the basedpyright thing for a while but that was painful. It's a bit too based for me, and I'm not sure i'd call it based. Right now I have uv, ruff, and ty and I'm happy with it. It's super easy to update and super fast. I didn't realize the coverage wasn't as good as some others but I still like it. I may have to try pyrefly. Never heard of it until this post, so thank you.

IshKebab 7mo ago

Interesting. This is the first I've heard of Zuban. The fact that Mypy fails so badly matches my experience. It would be interesting to see exactly where Pyright "fails". It's been so reliable to me I wouldn't be 100% surprised if these are deliberate deviations from the spec, where it is dumb.

ocamoss 7mo ago
Pyright had 100% conformance until recently. There was a new PEP that was accepted which required some new conformance tests, which no type checker has added support for yet.
networked 7mo ago
I think Zuban's license limited its exposure. I knew about Zuban but didn't pay attention to it. A proprietary development dependency was a no-go for my FLOSS projects, and I didn't want to adopt a separate tool just for proprietary code. I see that Zuban went AGPL in September 2025 (with exceptions available). This makes it a lot more interesting.

refactor_master 7mo ago

How does Zuban manage to be developed by what appears to be a single person without megacorp backing, yet be mere inches behind pyright at this stage?

Neywiny 7mo ago

This is great and I'll try out pyright ASAP on my current codebase. The people who wrote it evidently didn't have any type checking running (despite I think 3+ linters??) so it's a nightmare of > "well the checker accurately reports it will be type X in an error case not Y" > "but we never get type X" > "Then we don't have good enough coverage" It's so easy in vscode, but it isn't on by default like the c/c++ one I guess because too much legacy code would cause infinite errors. And the age old problem of .pyi files lying about types.

nas 7mo ago

Just an FYI, for people looking at the low pass rates for mypy and ty and concluding they must not be very useful. These test suites are checking many odd corners of the typing spec. For "normal" Python code, I find mypy does pretty good. Certainly I find it helpful, especially on a large code base and when working with other developers of various experience levels. The reason I prefer pyrefly over mypy is mostly because of speed. Better accuracy is nice but speed it the killer feature. Given the quality of uv and ruff and the experience of the team working on ty, I'm quite confident it's going to be great in that respect as well.

bko 7mo ago
I dont know, I've seen some truly simple things check out as ok in mypy Example def foo(bar: bool) -> bool: if bar: m = True return m No error that m is defined conditionally? What's going on?
- maleldil 7mo ago
  ty and zuban also don't give an error. pyright and pyrefly do.
  - _carljm 7mo ago
    We provide this diagnostic in ty (https://docs.astral.sh/ty/reference/rules/#possibly-unresolved-reference https://docs.astral.sh/ty/reference/rules/#possibly-unresolv...), but it's disabled by default because it can have false positives in many scenarios where the code works at runtime (this is true also in the type checkers that enable it by default). A typical example is code like def _(flag: bool): if flag: x = True ... if flag: read(x) But it's easy to turn the rule on if you want it: [tool.ty.rules] possibly-unresolved-reference = "error"
ocamoss 7mo ago
> people looking at the low pass rates for mypy and ty and concluding they must not be very useful Yeah, that would be the wrong takeaway from this blog. The point of the blog was to add context to what the conformance results mean and clarify their limitations, since I saw quite a few people sharing links to the tracker online w/o context.

JackSlateur 7mo ago

My favorite typing-related stuff is typeguard (https://pypi.org/project/typeguard/ https://pypi.org/project/typeguard/), with its pytest plugin