8 ms·

Python really needs to take the Typescript approach of "all valid Python4 is valid Python3". And then add value types so we can have int64 etc. And allow obje

• panzi 7mo ago
  Isn't rpython doing that, allowing changes on startup and then it's basically statically typed? Does it still exist? Was it ever production ready? I only once read a paper about it decades ago.
  • mattclarkdotnet 7mo ago
    RPython is great, but it changes semantics in all sorts of ways. No sets for example. WTF? The native Set type is one of the best features of Python. Tuples also get mangled in RPython.
  • zahlman 7mo ago
    It exists in the sense that PyPy exists. As far as I can tell, it only ever existed to make PyPy possible, and was only defined/specified in terms of PyPy's needs.
• mattclarkdotnet 7mo ago
  To clarify, it is nuts that in an object method, there is a performance enhancement through caching a member value. class SomeClass def init(self) self.x = 0 def SomeMethod(self) q = self.x ## do stuff with q, because otherwise you're dereferencing self.x all the damn time
  • mathisfun123 7mo ago
    > it is nuts that in an object method, there is a performance enhancement through caching a member value i don't understand what you think is nuts about this. it's an interpreted language and the word `self` is not special in any way (it's just convention - you can call the first param to a method anything you want). so there's no way for the interpreter/compiler/runtime to know you're accessing a field of the class itself (let alone that that field isn't a computed property or something like that). lots of hottakes that people have (like this one) are rooted in just a fundamental misunderstanding of the language and programming languages in general <shrugs>.
    • bmitc 7mo ago
      I don't think it's a hot take to say much of Python's design is nuts. It's a very strange language.
    • mattclarkdotnet 7mo ago
      What's nuts is that the language doesn't guarantee that successive references to the same member value within the same function body are stable. You can look it up once, go off and do something else, and look it up again and it's changed. It's dynamism taken to an unnecessary extreme. Nobody in the real world expects this behaviour. Making it just a bit less dynamic wouldn't change the fundamentals of the language but it would make it a lot more tractable.
      • rtpg 7mo ago
        In Python attribute access aren't stable! `self.x` where `x` is a property is not guaranteed to refer to the same thing. And getting rid of descriptors would be a _fundamental change to the language_. An immeense one. Loads of features are built off of descriptors or descriptor-like things. And what you're complaining about is also not true in Javascript world either... I believe you can build descriptor-like things in JS now as well. _But_ if you want that you can use stuff like mypyc + annotations to get that for you. There are tools that let you get to where you want. Just not out of the box because Python isn't that language. Remember, this is a scripting language, not a compiled language. Every optimization for things you talk about would be paid on program load (you have pyc stuff but still..) Gotta show up with proof that what you're saying is verifiable and works well. Up until ~6 or 7 years ago CPython had a concept of being easy to onboard onto. Dataflow analyses make the codebase harder to deal with. Having said all of that.... would be nice to just inline RPython-y code and have it all work nicely. I don't need it on everything and proving safety is probably non-trivial but I feel like we've got to be closer to doing this than in the past. I ... think in theory the JIT can solve for that too. In theory
  • dekelpilli 7mo ago
    Java also has a performance cost to accessing class fields, as exampled by this (now-replaced) code in the JDK itself - https://github.com/openjdk/jdk/blob/jdk8-b120/jdk/src/share/classes/java/lang/String.java#L2855 https://github.com/openjdk/jdk/blob/jdk8-b120/jdk/src/share/...
    • anematode 7mo ago
      Any decent JIT compiler (and HotSpot's is world class) will optimize this out. Likely this was done very early on in development, or was just to reduce bytecode size to promote inlining heuristics that use it
      • LtWorf 7mo ago
        But what if whatever you call is also accessing and changing the attribute?
        • anematode 7mo ago
          If what you call gets inlined, then the compiler can see that it either does or doesn't modify the attribute and optimize it accordingly. Even virtual calls can often be inlined via, e.g., class hierarchy analysis and inline caches. If these analyses don't apply and the callee could do anything, then of course the compiler can't keep the value hoisted. But a function call has to occur anyway, so the hoisted value will be pushed/popped from the stack and you might as well reload it from the object's field anyway, rather than waste a stack slot.
          • LtWorf 7mo ago
            Another thread can access it and do that, how could the compiler possibly know about it?
            • bmm6o 7mo ago
              There are documented ways to ensure that changes are visible across threads (e.g. locks). If these are not used, the compiler is within its rights to not go out of its way to pull changes from another thread.
  • duskdozer 7mo ago
    You mean even if x is not a property?
  • 1718627440 7mo ago
    This is not just a performance concern, this describes completely different behaviour. You forgot that self.x is just Class.__getattr__(self, 'x') and that you can implement __getattr__ how you like. There is no object identity across the values returned by __getattr__.
    • dundarious 7mo ago
      This level of dynamism is commonly forgotten/omitted because it is most often not at all needed. "There is no object identity across the values [retrieved by self.x]" is a very curious choice to many.
      • 1718627440 7mo ago
        It's very Pythonic to expose e.g. state via the existence of attributes. This also makes it possible to dynamically expose foreign language interfaces. You can really craft the interface you like, because the interface exposal is also normal code that returns strings and objects. You are right that it is not needed often, but there is often somewhere a part in the library stack that does exactly this, to expose a nice interface.
        • xenadu02 7mo ago
          This is just an analogy but in Swift String is such a commonly used hot path the type is designed to accommodate different backing representations in a performant way. The type has bits in its layout that indicate the backing storage. eg a constant string is just a pointer to the bytes in the binary and unless the String escapes or mutates incurs no heap allocation at all - it is just a stack allocation and a pointer. Javascript implementations do their own magic since most objects aren't constantly mutating their prototypes or doing other fun things. They effectively fast-path property accesses and fallback if that assumption proves incorrect. Couldn't python tag objects that don't need such dynamism (the vast majority) so it can take the fast path on them?
          • adrian17 7mo ago
• mattclarkdotnet 7mo ago
  Oh, and while we're at it, fix the "empty array is instantiated at parse time so all your functions with a default empty array argument share the same object" bullshit.
  • Izkata 7mo ago
    Execution time, not parse time. It's a side effect of function declarations being statements that are executed, not the list/dict itself. It would happen with any object.
    • mattclarkdotnet 7mo ago
      Let's not get started on the cached shared object refs for small integers....
      • zahlman 7mo ago
        What realistic use case do you have for caring about whether two integers of the same value are distinct objects? Modern versions of Python warn about doing unpredicatble things with `is` exactly because you are not supposed to do those things. Valid use cases for `is` at all are rare.
        • thaumasiotes 7mo ago
          > Valid use cases for `is` at all are rare. There might not be that many of them, depending on how you count, but they're not rare in the slightest. For example, you have to use `is` in the common case where you want the default value of a function argument to be an empty list.
          • fwip 7mo ago
            Could you expand on this? For example, this works just fine: def silly_append(item, orig=[]): return orig + [item] Edit: Oh, I think you probably mean in cases where you're mutating the input list.
• bloppe 7mo ago
  But that's just not what python is for. Move your performance-critical logic into a native module.
  • mattclarkdotnet 7mo ago
    Performance is one part of the discussion, but cleanliness is another. A Python4 that actually used typing in the interpreter, had value types, had a comptime phase to allow most metaprogramming to work (like monkey patching for tests) would be great! It would be faster, cleaner, easier to reason about, and still retain the great syntax and flexibility of the language.
    • mechsy 7mo ago
      I too see potential in this - it started feeling a bit weird in recent years switching between Go, Python and Rust codebases with Python code looking more and more like a traditional statically typed language and not getting the performance benefits. I know I know, there are libraries and frameworks which make heavy use of fun stuff you can do with strings (leading to the breakdown of even the latest and greatest IDE tooling and red squiggly lines all over you code) and don’t get me started on async etc. Funnily enough I’ve found Python to be excellent for modelling my problem domain with Pydantic (so far basically unparalleled, open for suggestions in Go/Rust), while the language also gets out of my way when I get creative with list expressions and the like. So overall, still it is extremely productive for the work I’m doing, I just need to spin up more containers in prod.
    • BerislavLopac 7mo ago
      > A Python4 that actually used typing in the interpreter, had value types, had a comptime phase to allow most metaprogramming to work (like monkey patching for tests) would be great! It would be faster, cleaner, easier to reason about, and still retain the great syntax and flexibility of the language. And what prevents someone from designing such a language?
      • LtWorf 7mo ago
• rich_sasha 7mo ago
  I think sadly a lot of Python in the wild relies heavily, somewhere, on the crazy unoptimisable stuff. For example pytest monkey patches everything everywhere all the time. You could make this clean break and call it Python 4 but frankly I fear it won't be Python anymore.
  • mattclarkdotnet 7mo ago
    Allowing metaprogramming at module import (or another defined phase) would cover most monkey patching use cases. From __future__ import python4 would allow developers to declare their code optimisable.
  • fyrn_ 7mo ago
    As a person who has spent a lot of time with pytest, I'm ready for testing framework that doesn't do any of that non-obvious stuff. Generally use unittest as much as I can these days, so much less _wierd_ about how it does things. Like jeeze pytest, do you _really_ need to stress test every obscure language feature? Your job is to call tests.
    • zahlman 7mo ago
      Yeah, I've been thinking about how I'd do it from scratch, honestly. (One of the reasons Pytest could catch on is that it supported standard library `unittest` classes, and still does. But the standard library option is already ugly as sin, being essentially an ancient port of JUnit.) I think it's not so much that Pytest is using obscure language features (decorators are cool and the obvious choice for a lot of this kind of stuff) but that it wants too much magic to happen in terms of how the "fixtures" automatically connect together. I would think that "Explicit is better than implicit" and "Simple is better than complex" go double for tests. But things like `pytest.mark.parametrize` are extremely useful.
  • NetMageSCW 7mo ago
    Perl 6 showed what happens when you do something like that.
  • germandiago 7mo ago
• musicale 7mo ago
  > Python really needs to take the Typescript approach of "all valid Python4 is valid Python3 Great idea, but I'm not convinced that they learned anything from the Python 2 to 3 transition, so I wouldn't hold my breath. If you want a language system without contempt for backward compatibility, you're probably better off with Java/C++/JavaScript/etc. (though using JS libraries is like building on quicksand.) Bit of a shame since I want to like Python/Rust/Swift/other modern-ish languages, but it turns out that formal language specifications were actually a pretty good idea. API stability is another.
  • musicale 7mo ago
    is that you, python core dev team? ;-)
    • musicale 7mo ago
      I'll take that as a yes.
• stabbles 7mo ago
  That was how the Mojo language started. And then soon after the hype they said that being a superset of Python was no longer the goal. Probably because being a superset of Python is not a guarantee for performance either.
  • Hendrikto 7mo ago
    Being a superset would mean all valid Python 3 is valid Python 4. A valuable property for sure, but not what OP suggested. In fact, it is the exact opposite.
• wolvesechoes 7mo ago
  > Python really needs to take the Typescript approach of "all valid Python4 is valid Python3" It is called type hints, and is already there. TS typing doesn't bring any perf benefits over plain JS.
  • stabbles 7mo ago
    You really need dedicated types for `int64` and something like `final`. Consider: class Foo: __slots__ = ("a", "b") a: int b: float there are multiple issues with Python that prevent optimizations: * a user can define subtype `class my_int(int)`, so you cannot optimize the layout of `class Foo` * the builtin `int` and `float` are big-int like numbers, so operations on them are branchy and allocating. and the fact that Foo is mutable and that `id(foo.a)` has to produce something complicates things further.
    • wolvesechoes 7mo ago
      Maybe, but I quoted specific part I was replying to. TS has no impact on runtime performance of JS. Type hints in Python have no impact on runtime performance of Python (unless you try things like mypyc etc; actually, mypy provides `from mypy_extensions import i64`) Therefore Python has no use for TS-like superset, because it already has facilities for static analysis with no bearing on runtime, which is what TS provides.
      • wiseowise 7mo ago
        What OP means is that they need to: 1) Add TS like language on top of Python in backwards compatible way 2) Introduce frozen/final runtime types 3) Use 1 and 2 to drive runtime optimizations
        • wolvesechoes 7mo ago
          Still makes no sense. OP demands introduction of different runtime semantics, but this doesn't require adding more language constructs (TS-like superset). Current type hints provide all necessary info on the language level, and it is a matter of implementation to use them or not. From all posts it looks like what OP wants is a different language that looks somewhat like Python syntax-wise, so calling for "backwards-compatible" superset is pointless, because stuff that is being demanded would break compatibility by necessity.
• BiteCode_dev 7mo ago
  There will be not Python 4, and 3.X policy requires forward compat, so we are already there.
• dobremeno 7mo ago
  SPy [1] is a new attempt at something like this. TL;DR: SPy is a variant of Python specifically designed to be statically compilable while retaining a lot of the "useful" dynamic parts of Python. The effort is led by Antonio Cuni, Principal Software Engineer at Anaconda. Still very early days but it seems promising to me. [1] https://github.com/spylang/spy https://github.com/spylang/spy
  • mattclarkdotnet 7mo ago
    Thank you! Spy looks brilliant, especially the comptime-like freezing after import.
• fermigier 7mo ago
  I have made some experiments with P2W, my experimental Python (subset) to WASM compiler. Initial figures are encouraging (5x speedup, on specific programs). https://github.com/abilian/p2w https://github.com/abilian/p2w NB: some preliminary results: p2w is 4.03x SLOWER than gcc (geometric mean) p2w is 5.50x FASTER than cpython (geometric mean) p2w is 1.24x FASTER than pypy (geometric mean)
• BerislavLopac 7mo ago
  > python code could be so much faster if it didn't have to assume everything could change at any time Definitely, but then it wouldn't be Python. One of the core principles of Python's design is to be extremely dynamic, and that anything can change at any time. There are many other, pretty good, strictly dynamically typed languages which work just as well if not better than Python, for many purposes.
  • oblio 7mo ago
    I feel that this excuse is being trotted out too much. Most engineers never get to choose the programming language used for 90% of their professional projects. And when Python is a mainstream language on top of which large, globally known websites, AI tools, core system utilities, etc are built, we should give up the purity angle and be practical. Even the new performance push in Python land is a reflection of this. A long time ago some optimizations were refused in order to not complicate the default Python implementation.
    • drob518 7mo ago
      You’re always free to create your own Python-like language that caters more toward your goals. No excuses, then.
      • oblio 7mo ago
        If you're a contributor to Python, my apologies.
        • drob518 7mo ago
          I’m not a Python contributor, so no need to apologize to me. But if you have strong ideas about what Python should be, perhaps you should step up and contribute that code rather than saying that others are offering excuses for why they won’t deliver what you want. I have worked on other open source projects where users were very entitled, to the point of demanding that the project team deliver them certain features. It’s not fun. It’s ironic that open source often brings out both the best and the worst in people. Suggesting changes and new features is fine, even critical to a strong roadmap. But we all need to realize that maintainers may have other goals and there’s no obligation on their part to implement anything. The beauty of open source is that you can customize or fork as much as you want to match your goals. But then you’re responsible for doing the work and if your changes are public you may have your own set of users demanding their own favorite changes.
• giancarlostoro 7mo ago
  I went sort of this route in an experiment with Claude.. I really want Python for .NET but I said, damn the expense, prioritize .NET compatibility, remove anything that isn't supported feasably. It means 0 python libs, but all of NuGet is supported. The rules are all signatures need types, and if you declare a type, it is that type, no exceptions, just like in C# (if you squint when looking at var in a funny way). I wound up with reasonable results, just a huge trade of the entire Python ecosystem for .NET with an insanely Python esque syntax. Still churning on it, will probably publish it and do a proper blog post once I've built something interesting with the language itself.
  • coredog64 7mo ago
    IronPython -> TitaniumPython?
• phkahler 7mo ago
  >> Sensible type-annotated python code could be so much faster if it didn't have to assume everything could change at any time. Then it wouldn't be Python any more.
  • germandiago 7mo ago
    I share your view. Python's flexibility is central to Python. Even type annotations, though useful, can get in the way for certain tasks.Betting on things like these to speed up things would be a mistake, since it would kind of force you to follow that style. Anything that accelearates things should rely on run-time data, not on type annotations that won't change.
  • 0xffff2 7mo ago
    Fine by me. I don't particularly like Python, but it's the defacto standard in my field so I have to use it (admittedly this is an improvement over a decade ago, when MATLAB was the defacto standard). I don't care about preserving the spirit of Python, I just care that the thing that bears the name Python meets my needs.
    • 7bit 7mo ago
      If that's fine by you, maybe just pick another language instead of having a unqualified opinion.