8 ms·

Pyrefly vs. Ty: Comparing Python's two new Rust-based type checkers

• senkora 1y ago
  > ty, on the other hand, follows a different mantra: the gradual guarantee. The principal idea is that in a well-typed program, removing a type annotation should not cause a type error. In other words: you shouldn’t need to add new types to working code to resolve type errors. The gradual guarantee that Ty offers is intriguing. I’m considering giving it a try based on that. With a language like Python with existing dynamic codebases, it seems like the right way to do gradual typing.
  • yoyohello13 1y ago
    Unless you're doing greenfield, gradual typing is really the only way. I've incorporated type hinting in several legacy Python code bases with mypy and really the only sensible way is to "opt-in" one module at a time. If pyrefly doesn’t support that I think its use will be pretty limited. Unless maybe they are going for the llm code gen angle. I could see a very fast and strict type checker being useful for llm generating python scripts.
  • RandomBK 1y ago
    It reminds me of the early days of Typescript rollout, which similarly focused on a smooth on-boarding path for existing large projects. More restrictive requirements (ie `noImplicitAny`) could be turned on one at a time before eventually flipping the `strict` switch to opt in to all the checks.
  • rendaw 1y ago
    Gradual typing means that an implicit "any" (unknown type) anywhere in your code base is not an error or even a warning. Even in critical code you thought was fully typed. Where you mistakenly introduce a type bug and due to some syntax or inference limits the type checker unexpectedly loses the plot and tells you confidently "no problems in this file!" I get where they're coming from, but the endgame was a huge issue when I tried mypy - there was no way to actually guarantee that you were getting any protection from types. A way to assert "no graduality to this file, it's fully typed!" is critical, but gradual typing is not just about migrating but also about the crazy things you can do in dynamic languages and being terrified of false positives scaring away the people who didn't value static typing in the first place. Maybe calling it "soft" typing would be clearer. I think gradual typing is an anti-pattern at this point.
    • mmoskal 1y ago
      As mentioned in other comments - in TypeScript which follows this gradual typing there is a number of flags to disable it (gradually so to speak). No reason ty wouldn't do it.
    • belmont_sup 1y ago
      Responding to your gradual typing anti-pattern bit: Agree that dynamic language behaviors can be extreme but it’s also easy to get into crazy type land. Putting aside a discussion of type systems, teams can always add runtime checks like pydantic to ensure your types match reality. Sorbet (Ruby typechecker) does this where it introduces a runtime checks on signatures. Similarly in ts, we have zod.
      • MeetingsBrowser 1y ago
        > teams can always add runtime checks like pydantic to ensure your types match reality. That's the problem with bugs though, there's always something that could have been done to avoid it =) Pydantic works great in specific places, like validating user supplied data, but runtime checks as a replacement for static type checkers are not really feasible. Every caller would need to check that the function is being called correctly (number and position of args, kwarg names, etc) and every callee would need to manually validate that each arg passed matches some expected type.
        • eternityforest 1y ago
          Pydantic also takes CPU time and doesn't do anything till runtime. Type checking is real time in the IDE and lets you fix stuff before you waste fifteen minutes actually running it.
        • Spivak 1y ago
          pydantic does have @validate_call for this use-case.
          • guappa 1y ago
  • tialaramex 1y ago
    Although I'm paid to write (among other things) Python and not Rust, I would think of myself as a Rust programmer and to me the gradual guarantee also makes most sense.
  • IshKebab 1y ago
    This is a big turnoff for me. Half the point of adding type annotations to Python is to tame its error-prone dynamic typing. I want to know when I've done something stupid, even if it is technically allowed by Python itself. Hopefully they'll add some kind of no-implicit-any or "strict" mode for people who care about having working code...
• paddy_m 1y ago
  I hope some typechecker starts doing serious supported notebook integration. And integration for live coding, not just a batch script to statically check your notebook. Finding errors with typing before running a 1-60 minute cell is a huge win.
  • thelastbender12 1y ago
    Do you use Jupyter notebooks in VSCode? It uses the same pylance as regular python files, which actually gets annoying when I want to write throwaway code.
    • ZeroCool2u 1y ago
      Anyone reading this, if you're like me and prefer the open source version of VSCode where Microsoft disables Pylance, I'd encourage you to try BasedPyright instead.
      • baggiponte 1y ago
        BasedPyright is much better than pyright! Must use.
  • tomtom1337 1y ago
    I echo the other response here. You absolutely should switch to using notebooks in VSCode with their static typenchecker. Language Servers do exactly what you are wanting, with both notebook integration and «live coding».
• deleted 1y ago
  [deleted]
• dgroshev 1y ago
  I really wish they get first class Django support. Sadly, its ORM architecture is impossible to type and impossible to change now. Django is one of the most important use cases for Python, having fast full type checking with Django is a dream, but it does require some special casing from the type checker.
  • catlover76 1y ago
    Isn't that largely on the shoulders of Django maintainers?
  • MeetingsBrowser 1y ago
    What makes the Django ORM impossible to type check?
    • zbentley 1y ago
      It uses a huge amount of what I’m terming “getattr bullshit”: many/most fields of ORM objects are only determined at runtime (they’re technically determinABLE at early runtime during Django initialization, but in practice are often not actually visible via reflection until they are first used due to lazy caching). What fields are present and what types they have is extremely non uniform: it depends heavily on ORM objects’ internal configuration and the way a given model class relates to other models, including circular dependencies. (And when I say “fields” here, I’m not only referring to data fields; even simple models include many, many computed method-like fields, complex lazily-evaluatable and parametrizable query objects, fields whose types and behavior change temporally or in response to far-distant settings, and more). Some of this complexity is inherent to what ORMs are as a class of tool—many ORMs in all sorts of languages provide developer affordances in the form of highly dynamic, metaprogramming-based DSL-ish APIs—but Django really leans in to that pattern more than most. Add to that a very strong community tendency to lazily (as in diligence, not caching) subclass ORM objects in ways that shadow computed fields—and often sloppily override the logic used to compute what fields are available and how they act—and you have a very thorny problem space for type checkers. I also want to emphasize that this isn’t some rare Django power-user functionality that is seldom used, nor is it considered deprecated or questionable—these computed fields are the core API of the Django ORM, so not only are they a moving target that changes with Django (and Django extension module) releases, but they’re also such a common kind of code that even minor errors in attempts to type-check them will be extremely visible and frustrating to a wide range of users. None of that should be taken as an indictment of the Django ORM’s design (for the most part I find it quite good, and most of my major issues with it have little to do with type checking). Just trying to answer the question as directly as possible.
• deleted 1y ago
  [deleted]
• kuratkull 1y ago
  > my_list = [1, 2, 3] > pyrefly, mypy, and pyright all assume that my_list.append("foo") is a typing error, even though it is technically allowed (Python collections can have multiple types of objects!) > If this is the intended behavior, ty is the only checker that implicitly allows this without requiring additional explicit typing on my_list. EDIT: I didn't intend my comment to be this sharp, I am actually rooting for ty to succeed :) ORIGINAL: I am strongly against ty behaviour here. In production code you almost always have single type lists and it is critical that the typechecker assumes this, especially if the list already has same-type _literal_ items. The fact that Python allows this has no bearing at all. To me having list[int | str] implicitly allowed by the typechecker seems like optimizing for beginner-level code.
  • zem 1y ago
    I am strongly for ty's behaviour here. working python code should not raise type errors unless the user explicitly opts in to a more static subset of the language by adding type annotations.
  • ameliaquining 1y ago
    I don't think it's optimizing for beginner-level code, I think it's optimizing for legacy code. Introducing a type checker to a large existing untyped codebase is a big lift, but becomes less of one if almost all existing code is accepted.
    • luoc 1y ago
      Well then support an option to enable that kind behaviour? Make it an explicit decision by the devs. I think running in a type error and then adding an exception to your config is safer than silently pass and only learn about the mixed types in a production bug
      • fastasucan 1y ago
        The tool doesn't have a version number yet, its in preview. Chill.
      • 1y ago
• melodyogonna 1y ago
  I think I prefer Pyrefly's stronger type inference. It can be a pain on Projects with a lot of dynamism, but I'll personally make the tradeoff
• wejick 1y ago
  Ty will have lower barrier of entry, similar to the early days of Typescript. I'm curious to see which way the community will lean to.
• kombine 1y ago
  Are any of these already useful as LSPs for code editors such as Neovim? I run pyright in my Neovim config but I could certainly use something faster.
  • dcreager 1y ago
    ty is definitely not ready to be a pyright replacement yet. But it is usable as an LSP for simple things like go to definition, and deeper LSP features are on the roadmap for the eventual beta and GA releases. https://github.com/astral-sh/ty/blob/main/docs/README.md#other-editors https://github.com/astral-sh/ty/blob/main/docs/README.md#oth...
    • boomer_joe 1y ago
      What's the plan for ruff? Will it be part of ty one day?
      • dcreager 1y ago
        The current plan is that they will remain separate tools, but will work together nicely if you are using both. For instance, we want to add type-aware and multi-file lints to ruff at some point down the line.
  • alfalfasprout 1y ago
    Not yet tbh. I would realistically expect at least a year or more before you can expect any kind of parity with eg; pyright/basedpyright. Type checkers are hard and have a long tail of functionality that must be implemented.
  • djrj477dhsnv 1y ago
    What kind of codebases cause performance problems? I've been using pyright with neovim for years and have never experienced any kind of noticeable lag.
• muglug 1y ago
  I’ve built a few typecheckers (in different languages) that hew closer to Pyrefly’s behaviour than Ty’s behaviour. If you have a large codebase that you want to be typesafe, Pyrefly’s approach means writing far fewer type annotations overall, even if the initial lift is much steeper. Ty effectively has noImplicitAny set to false.
• dcreager 1y ago
  [ty developer here] We are happy with the attention that ty is starting to receive, but it's important to call out that both ty and pyrefly are still incomplete! (OP mentions this, but it's worth emphasizing again here.) There are definitely examples cropping up that hit features that are not yet implemented. So when you encounter something where you think what we're doing is daft, please recognize that we might have just not gotten around to that yet. Python is a big language!
  • zem 1y ago
    surfacing revealed types as `@TODO` made me laugh, but thinking about it it's actually a pretty neat touch!
    • dcreager 1y ago
      It really helps in our mdtests, because then we can assert that not-implemented things are currently wrong but for the right reasons!
  • echelon 1y ago
    Totally orthogonal question, but since you're deep in that side of Rust dev - The subject of a "scripting language for Rust" has come up a few times [1]. A language that fits nicely with the syntax of Rust, can compile right alongside rust, can natively import Rust types, but can compile/run/hot reload quickly. Do you know of anyone in your network working on that? And modulus the syntax piece, do you think Python could ever fill that gap? [1] https://news.ycombinator.com/item?id=44050222 https://news.ycombinator.com/item?id=44050222
    • mdaniel 1y ago
      > And modulus the syntax piece, do you think Python could ever fill that gap? I would never ever want a full fledged programming language to build type checking plugins, and doubly so in cases where one expects the tool to run in a read-write context I am not saying that Skylark is the solution, but it's sandboxed mental model aligns with what I'd want for such a solution I get the impression the wasm-adjacent libraries could also help this due to the WASI boundary already limiting what mutations it is allowed
• NeutralForest 1y ago
  Nice, I'm using basedpyright right now, both as a type checker in my IDE as well as in GitHub actions. It's good and mainly does what I want. I'm not very fond of mypy as it struggles even with simple typing at times.
• modeless 1y ago
  As they're described here, Pyrefly's design choices make more sense to me. I like the way Typescript does type inference and it seems like Pyrefly is closer to that. Module-level incrementalism also seems like a good tradeoff. Fine-grained incrementalism on a function level seems like overkill. Performance should be good enough that it's not required.
• vintagedave 1y ago
  What was not immediately obvious to me (but should have been) is that these are dev-time type checkers -- I think. (I think, both from from the github descriptions which focus heavily on editing, and from the article.) That's really useful because type inference is lacking, to me, in-editor. I tend to ask Copilot: 'add type annotations'. So in complement to this can I share my favorite _run-time_ type checker? Beartype: this reads your type annotations (ie I see this is where Pyrefly and Ty come in), and enforces the types at runtime. It is blazingly fast, as in, incredibly fast. I use it for all my deployed code. https://beartype.readthedocs.io/en/latest/ https://beartype.readthedocs.io/en/latest/ I suspect either of Pyrefly or Ty will be highly complementary with Beartype in terms of editor additions, and then runtime requirements. The docs also have a great sense of humour.
  • maleldil 1y ago
    beartype is great, but I only find it useful at the edges. Runtime checks aren't needed if you have strict typing throughout the project. On a gradually-typed codebase, you can use beartype (e.g. is_bearable) to ensure the data you're ingesting has the proper type. I usually use it when I'm dealing with JSON types.
    • vintagedave 1y ago
      Why isn’t it necessary? Do you mean that with edit-time type checking, you can catch all errors, so no need for runtime verification the edit-time type decls match? What about interacting with other libraries?
      • maleldil 1y ago
        If you have strict static type checking, type errors can't creep in, so you don't need runtime checking. pyright (type checker) will tell you when a runtime check is redundant. For example, if you already have `var` annotated (or inferred) as `str`, then `if isinstance(var, str)` is statically guaranteed to be true. Of course, that's only if you trust all the types in your code. You still have escape hatches, such as Any and cast, that can break this guarantee. There are lints (from ruff) and pyright options to help with this. Concerning external libraries, I either use libraries that are 100% typed (which is common these days), or write my own type-safe wrappers around the others.
• suspended_state 1y ago
  I am not well versed in python programming, this is just my opinion as an outsider. For anyone interested in using these tools, I suggest reading the following: https://www.reddit.com/r/Python/comments/10zdidm/why_type_hinting_sucks/ https://www.reddit.com/r/Python/comments/10zdidm/why_type_hi... That post should probably be taken lightly, but I think that the goal there is to understand that even with the best typing tools, you will have troubles, unless you start by establishing good practices. For example, Django is large code base, and if you look at it, you will observe that the code is consistent in which features of python are used and how; this project passes the stricter type checking test without troubles. Likewise, Meta certainly has a very large code base (why develop a type checker otherwise?), and they must have figured out that they cannot let their programmers write code however they like; I guess their type checker is the stricter one for that reason. Python, AFAIK, has many features, a very permissive runtime, and perhaps (not unlike C++) only some limited subset should be used at any time to ensure that the code is manageable. Unfortunately, that subset is probably different depending on who you ask, and what you aim to do. (Interestingly, the Reddit post somehow reminded me of the hurdles Rust people have getting the Linux kernel guys to accept their practice: C has a much simpler and carefree type system, but Rust being much more strict rubs those C guys the wrong way).
  • mhh__ 1y ago
    At this point I'm fairly convinced that the effort one would spend trying to typecheck a python program is better spent migrating away from python into a language that has a proper type system, then using interop so you can still have the bits/people that need python be in python. Obviously that isn't always possible but you can spend far too long trying to make python work.
    • davedx 1y ago
      Unfortunately with us being in the middle of the AI hype cycle, everyone and their dog is currently busy migrating to python.
      • hu3 1y ago
• merksoftworks 1y ago
  I'd like to experiment with writing a reflection based dynamic type annotator for one of these. Just as a toy. Imagine a module which monkey patches every function, class in the parent module it is loaded into, then reflects on the arguments at run time slowly building up a database of expected type signatures. The user would then pick through the deltas approving what made sense.
  • LoganDark 1y ago
    This is a neat idea, but for large systems it could take a long time to fully exercise every code path with all possible input data. (If this sounds way too excessive, that's because normal people don't do this.) Unless you mean something like record prod for a few weeks to months, similar to how Netflix uses eBPF (except they run it all the time).
    • merksoftworks 1y ago
      I think my main use case would be to run it on unit tests, and then to provide a human-in-the-loop way to slowly hydrate small to medium sized code bases with types. Maybe progressive refactor tools aren't something anyone actually wants to use or maintain, because I just don't see them around that much.
• whyho 1y ago
  Astral tooling is great and brings new energy into python land but what is the long game of all astral projects? Integrate them into python natively? Be gone in 5 years and leave unmaintained tooling behind? Rug pull all of us with a subscription?
  • move-on-by 1y ago
    I don’t think any of these questions are specific to Astral and can be applied to pretty much any project. ‘Be gone in 5 years and leave unmaintained tooling’ seems particularly plausible with regard to Facebook’s tooling. Use any of them at your own risk I suppose.
  • ijustlovemath 1y ago
    They'll most likely pursue some sort of business source licensing, where you will not be allowed to deploy apps in production using their tooling without the business paying some kind of subscription. I understand that none of their existing products fit this use case, but it will probably be a similar approach. VCs are not charities.
  • belmont_sup 1y ago
    As a Redditor said: > The standard VC business model is to invest in stuff that FAANG will buy from them one day. The standard approach is to invest in stuff that's enough of a threat to FAANG that they'll buy it to kill it, but this seems more like they're gambling on an acqui-hire in the future.
    • ipsum2 1y ago
      I have never seen a FAANG company buy a pure programming-language based tooling startup.
      • tjalfi 1y ago
        Facebook acquired Monoidics in 2013; they were the startup that created Infer[0]. [0] https://en.wikipedia.org/wiki/Infer_Static_Analyzer https://en.wikipedia.org/wiki/Infer_Static_Analyzer
• claytonjy 1y ago
  for decades, big tech contributed relatively little in the way of python ecosystem tooling. There’s Facebooks Pyre, but that’s about it. Nothing for package/dependency management, linting, formatting, so folks like those at Astral have stepped up to fill the gap. why is type checking the exception? with google and facebook and astral all writing their own mypy replacements, i’m curious why this space is suddenly so busy
  • flakes 1y ago
    Probably because a large amount of AIs are churning out Python code, and they need type-checkers to sanitize/validate that output quickly. Dynamic languages are hard enough for people to make sense of half the time, and I bet AI agents are struggling even more.
  • emptysea 1y ago
    Instagram built a linter with the ability to fix errors which is an improvement over flake8 & pylint: https://github.com/Instagram/Fixit https://github.com/Instagram/Fixit But Ruff is an even greater improvement over that
  • ckwalsh 1y ago
    Coming from a Meta background (not speaking on behalf of Meta): "package/dependency management" - Everything is checked into a monorepo, and built with [Buck2](https://buck2.build/ https://buck2.build/). There's tooling to import/update packages, but no need to reinvent pip or other package managers. Btw, Buck2 is pretty awesome and supports a ton of languages beyond python, but hasn't gotten a ton of traction outside of Meta. "linting, formatting" - [Black](https://github.com/psf/black https://github.com/psf/black) and other public ecosystem tooling is great, no need to develop internally. "why is type checking the exception" - Don't know about Astral, but for Meta / Google, most everyone else doesn't design for the scale of their monorepos. Meta moved from SVN to Git to Mercurial, then forked Mercurial into [Sapling](https://sapling-scm.com/ https://sapling-scm.com/) because simple operations were too slow for the number of files in their repo, and how frequently they receive diffs. There are obvious safety benefits to type checking, but with how much Python code Meta has, mypy is not an option - it would take far too much time / memory to provide any value.
• librasteve 1y ago
  it is sad to see how painful typed python is … i prefer a language where (gradual) types were designed infrom the get go … https://raku.org https://raku.org
  • flanked-evergl 1y ago
    As someone who has added type hints to two huge code bases that had nine. It's not that painful. Something much more painful is realizing the code base you are adding type hints to is irreconcilably bugged by design, which would not have been possible had type checking been used.
• thom 1y ago
  Any progress in the Python ecosystem on static checking for things like tensors and data frames? As I understand it, the comments at the bottom of this FAQ still apply: https://docs.kidger.site/jaxtyping/faq/ https://docs.kidger.site/jaxtyping/faq/
• N1H1L 1y ago
  I have a problem with Python's `Optional` type. For example for this following code: from typing import Optional, Union def square( a: Union[int, float], b: Optional[int] = 2 ) -> float: c = a**b return c Many type checkers throw an error because `Optional[int]` actually means `int | None` and you cannot square an `int` or a `float` with a `None`. Is there any plans for *ty* around this?
  • dcreager 1y ago
    I think that's a genuine error, since as you say, `None` is a possible value for `b` according to your signature. To handle this you would need to use "narrowing" to separately handle the case where `b` is `None`, and the case where it is not: def square(a: Union[int, float], b: Optional[int] = 2) -> float: if b is None: return 0 else: c = a**b return c https://play.ty.dev/97fe4a09-d988-4cc3-9937-8822e292f8d1 https://play.ty.dev/97fe4a09-d988-4cc3-9937-8822e292f8d1 (This is not specific to ty, that narrowing check should work in most Python type checkers.)