9 ms·
Pyrefly: A new type checker and IDE experience for Python
- homarp 1y agothis is the official announcement, but pyrefly was previously discuted a few weeks ago: https://news.ycombinator.com/item?id=43831524 https://news.ycombinator.com/item?id=43831524 "Today we’re releasing Pyrefly as an alpha. At the same time, we’re busy burning down the long-tail of bugs and features aiming to remove the alpha label this Summer. "
- globalnode 1y agoIts probably cool n' all but fb isnt getting any of my attention. They'd need to come up with AGI for that to happen, and even then I'd shrug it off.
- frogperson 1y agoI agree. I simply can't support anything Mark Zuckerberg does at this point.
- surajrmal 1y agoThis is an open source developer tooling project, not a product which Zuckerberg had anything to do with. What's the point of ignoring it?
- voidfunc 1y agoGuilt by association is very in vogue these days
- doug_durham 1y agoThese talented engineers could take their skills elsewhere. That's the message.
- globalnode 1y ago> Why we built Pyrefly: Back in 2017, we embarked on a mission to create a type checker that could handle Instagram’s massive codebase of typed Python They're saying this on fb.com. How does it not have anything to do with fb? The feedback section takes you to fb's github.
- insane_dreamer 1y agowhat if they pull off the metaverse? ;)
- globalnode 1y agook that would be impressive, " no no we're not interested in AGI -- we want to become god "
- kingkongjaffa 1y agoThis is very cool but why wouldn’t they just contribute to uv and ruff and ty https://github.com/astral-sh/ty https://github.com/astral-sh/ty
- ldng 1y agoNIH + copyright
- trostaft 1y agoIsn't both this and ty MIT license?
- djinnish 1y agoI think astral and meta were both working on their own type-checkers independently. My current understanding is that meta released so they could preempt the initial release of ty. It seems like they're a bit further ahead in development. Not sure if there are going to be any real differences between the two down the line.
- bsimpson 1y agoSounds a lot like TypeScript and Flow.
- surajrmal 1y agoTypescript was Microsoft though. Meta might have the edge here based on brand awareness, but who knows for sure.
- bsimpson 1y agoRight, and Flow was FB
- singhrac 1y agoSure, but in this case they are both implementations of a spec defined by PEPs, so a bit more like gcc vs clang (less tightly bound than those, of course, in design decisions). Neither company is trying to invent a new language here.
- ThePhysicist 1y agoSeems there are at least three Rust-based competitors for type checkers in Python now (Microsoft, Facebook, Astral), and of course there's still mypy.
- morkalork 1y agoThey're all static type checkers right? None for runtime?
- Yossarrian22 1y agoYes. If you want runtime validation of data you’re taking in people recommended pydantic. If you’re looking for runtime validation within your own code I’ve seen people use beartype, though to be honest I don’t personally understand the value added from it
- rationably 1y ago...or Marshmallow, which allows one to do many complex validations in a relatively trivial manner.
- morkalork 1y agoOn one hand, I feel like I've been in a coma since covid because I've just been coasting along with Marshmallow and jsonschema, but on the other hand it's like a lot of the major advances have been in the past couple years. Apparently pydantic got a big version update in 2023? And now all these competing static type checkers?
- WD-42 1y agoPydantic got the re write in rust treatment so de/serialization is crazy fast now.
- notpushkin 1y ago
- phlakaton 1y agoI'm curious to know more about the Pyre to Pyrefly transition, specifically the rewrite in Rust. Was that merely a case of trading in a lesser-known language for the language du jour? Were there specific advantages they wanted to get out of Rust?
- team_pyrefly 1y agoHi! We address this question in our FAQ and probably could do a longer blog post about our experience after we are further along: https://pyrefly.org/en/docs/pyrefly-faq/#why-rust https://pyrefly.org/en/docs/pyrefly-faq/#why-rust
- adsharma 1y ago> Not only is Pyrefly written in a new language (Rust instead of OCaml), but its design deviates in a major way from Pyre. I'm sure you had reasons to do it this way. But given sufficient time to market, implementing the algorithm in pyre and then tooling/llm assisted conversion to pyrefly would've been preferable. May be you'd have had some humans in the loop initially. But that tech is getting better and aligned with the direction Meta and the rest of the industry are taking. Yes, I'm biased :)
- wiseowise 1y ago> tooling/llm assisted conversion to pyrefly would've been preferable Is this how it is now? Not the first time I hear this brainrot idea.
- phlakaton 1y agoThanks! But my real question is why the departure from OCaml. I'm guessing performance wasn't the rationale there.
- vivzkestrel 1y agohow does this compare to mypy
- deleted 1y ago[deleted]
- aleksanb 1y agoTo repeat an earlier comment of mine from the launch of uv on hn (tl; dr: these new type checkers never support django): The way these type checkers get fast is usually by not supporting the crazy rich reality of realworld python code. The reason we're stuck on mypy at work is because it's the only type checker that has a plugin for Django that properly manages to type check its crazy runtime generated methods. I wish more python tooling took the TS approach of "what's in the wild IS the language", as opposed to a "we only typecheck the constructs we think you SHOULD be using".
- fulafel 1y agoTS has the luxury of being its own distinct language, defining its own semantics and having its own compiler. You could have something like that targeting Python.
- tasn 1y ago1. Maybe it's time to drop the crazy runtime generation and have something statically discoverable, or at least a way to annotate the typing statically. 2. Astral indicated already they plan to just add direct support for Django and other popular languages. 3. As people replied to similar comments on the previous threads (maybe to you?): that's not why ty is fast and why mypy is slow. It's also an easy claim to disprove: run both without plugins and you'll see ty is still 100x+ faster.
- WhyNotHugo 1y ago> 1. Maybe it's time to drop the crazy runtime generation and have something statically discoverable, or at least a way to annotate the typing statically. That, and duck typing, are one of the biggest things that make Python what it is. If I have to drop all that for type checking and rewrite my code, why would I rewrite it in Python?
- dontlaugh 1y agoHaving used Python for many years, it’s the least interesting aspect of the language. Almost all such tricks can be done with compile time meta programming, often even without API changes.
- rw2 1y agonon AI IDEs are going to have a hard time in the future.
- bpshaver 1y agoUnsure how this comment is relevant
- lacker 1y agoI'm a little worried on behalf of the "Python Language Tooling Team" at Meta, because uv has been so popular, and I wouldn't be surprised if ty wins out in this space. So watch out, or this will become like Atom or Flow, an internal competitor of a technology that is surpassed by the more popular external open source version, leaving the directors/vps muttering to themselves "It's too bad that this team exists at all. Could we get rid of them and just switch to the open source stuff?" Perhaps just something for the manager (Aaron Pollack?) to keep an eye on....
- 90s_dev 1y agoJSX is my favorite thing to come out of Facebook (also the only good thing).
- owebmaster 1y agoI feel bad for that people that love JSX and don't know about lit-html yet.
- 90s_dev 1y agoJSX supports * Autocompletion * Type checking * Syntax highlighting * Lack of runtime string parsing Tagged template literals don't.
- owebmaster 1y agoJSX does not support any of these, it was coded by others. tagged template literals can have all of these, some already exist¹ and doesn't need a build step. 1. https://marketplace.visualstudio.com/items?itemName=bierner.lit-html https://marketplace.visualstudio.com/items?itemName=bierner....
- 90s_dev 1y agoMatt, who works pretty high level on VS Code, has written some amazing plugins. But they still don't have all the first-class features of JSX. I don't think this plugin works with type checking for instance.
- doug_durham 1y agoWhy is "written in Rust" a feature to be mentioned? Who cares? So my type checker has memory protection and is compiled. I'm not running my type checker on an embedded system or in a mission critical service. It seems kind of like "written in Erlang". I'd prefer to have non-performance critical code for Python written in Python. That way the broader community can support and extend it.
- tomrod 1y agoShortcut for "noticeably fast." Open source Rust is still review able.
- beeb 1y agoIt makes it easy to find performant and quality software by searching for "[insert tool description] rust", I personally don't mind! Seeing how 95% of the tools I use on the daily are written in Rust, I love finding new ones and am rarely disappointed.
- mcbuilder 1y agoI feel like 70% of open source projects on GitHub say written in the language that they were written in
- bpshaver 1y agoI feel like the likelihood that a project will say what language it is written in is much higher if that language is Rust. I like Rust but I do find this trend a little annoying. (Even though I acknowledge that "written in Rust" probably means the tool is relatively new, not buggy, and easy to use.)
- amelius 1y agoSounds like a project that is trying to solve too many things at once ...
- melodyogonna 1y agoThe Rust code written here is so easy to follow but all these new Python tooling being written in Rust worries me, it adds yes another vector to the N-language problem. I hope Mojo can offer something here
- nine_k 1y agoFor the Python ecosystem, it's natural to use Python where Python can cope, and a high-performance language where it cannot. There are two such languages in wide use around Python: Rust, and, inevitably, C. So N = 3. (C, to my mind, should be eventually phased out from application programming altogether, so N would be 2, but it's a loooooong process; Python may become a legacy language before it converges.)
- fluidcruft 1y agoYes but the idea is that by slightly upgrading python code to mojo (which is a controlled superset of python), you get complied very high performance code. So for example if it were possible to convert mypy to mojo it could be as fast as rust but pythonic.
- Hasnep 1y agoBy the way, Mojo seems to have given up on being a strict superset of python, I think they're aiming for more of a C++ to C relationship where there are some small differences, but the majority of code will work without modification.
- melodyogonna 1y agoIt has not. They decided to change the messaging to reflect what the language is to today because people kept thinking it is already a superset
- Hasnep 1y agoThanks for the explanation, that makes more sense than them giving up.
- team_pyrefly 1y agoHi folks, I work on the Pyrefly team at Meta. Our FAQ covers a good number of the questions raised here: https://pyrefly.org/en/docs/pyrefly-faq/ https://pyrefly.org/en/docs/pyrefly-faq/. I can also try to answer some of your questions. Thanks for taking a look!
- muglug 1y agoAt Slack we have an internal Rust-based Hack typechecker that’s about 20% faster than the OCaml one (we use both) Did you see better speedups over Pyre? Aka did I leave stuff on the table…
- samwgoldman 1y agoHack is quite a bit more optimized than Pyre was, but Pyrefly is at least 10x faster than Pyre on the IG codebase. I didn’t know about the Rust-based Hack checker— that’s really cool!
- muglug 1y agoWhere do those speed ups come from? 10x seems extraordinary for a task that’s pretty IO-bound. Was Pyre ineffectively parallelised or something?
- CoolCold 1y agoIO bound, how you define that? From my perspective, modern consumer grade SSDs/NVMes producing 7GB/s-8GB/s (yep, Giga Byte per second) for reads gives me 1/7-1/8 of second to load that data in RAM the rest of the 1 second is for CPU.
- muglug 1y ago"pretty" was doing too much work, but there's the twin cost of reading and parsing the file, which usually needs to be done twice (once for static reflection, and once for analysis). The reading and parsing eats up about 16% of the analysis phase, and slightly more overall — maybe 20% of the total runtime.
- oellegaard 1y agoI lost all interest when I saw VS Code. I don’t get why people consider this a suitable IDE for python when you can have a real IDE like PyCharm.
- artemisart 1y agopyrefly is not tied to vscode? Also please try to be more considerate of people preferences, and pycharm is not strictly better. Remote dev on vscode is very convenient for me, should I go on the Internet saying that pycharm is trash? No
- oellegaard 1y agoI’m not saying VS Code is trash but I think it’s closer to a text editor than an IDE. I even use it for some things non python but I remain curious to the fact why people use it for python. It might not be tied to VS Code but the title clearly says “New […] IDE experience for…” which is why I commented. I had hoped to see something for PyCharm or even a new IDE.
- fastasucan 1y agoThats what you think, other people think something else.
- bobflorian 1y agoWhat can you do in PyCharm that you cannot do in VS Code? I recently switched from PyCharm to VS Code to maintain a project with 250k LoC in Python (Django) and VS Code has been like a breath of fresh air. While you may need to install some plugins to get it "just right", it's more extensible. PyCharm is more "batteries included", and maybe that's the rub here.
- zem 1y agofortunately the LSP protocol means you can use pyrefly with whatever ide or editor you like!
- xwowsersx 1y agoHappy to see instructions for integrating into Vim/Neovim: https://pyrefly.org/en/docs/IDE/#other-editors https://pyrefly.org/en/docs/IDE/#other-editors
- weekendcode 1y agoofcourse it has it rewritten in rust.
- lucas_membrane 1y agoI just tried pyrefly on a project that really needed it. It complained about an assignment of a new value to a global int variable within a function, even though the function contained the 'global' statement that should have made that OK, I think. I know that globals and assigning to them here and there are problematic for real good software, but I am surprised that Pyrefly is stricter than python on something that I don't see as a type- checking issue. But it did find a decent list of other problems that I haven't finished working my way through. I had gotten so messed up trying to put together a quicky hobby-type program to create a data structure of perhaps a hundred data items in various overlapping and inter-related hierarchies, tuples, dicts, and lists akimbo, that I gave up on it about 10 days ago. I hypothecated that bondage and discipline might be the way to control the confusion, so I'm rewriting, using SQLite for the dataflow from function to function, lots of little tables and no optional fields. Can anyone opine on whether that is a sensible option?
- samwgoldman 1y agoThanks for trying it out! If you run into any blockers, please let us know by filing a GitHub issue or sending us a message on Discord. Pyrefly is still alpha software, so bugs are expected, but your feedback is extremely valuable as we work to squash them.
- gitroom 1y ago[dead]
- liotier 1y agoHow close to a statically-typed language does such addition make Python ?