25 ms·

Python 3.12

652 points by qsort 3y ago

paganel 3y ago
What's with the politicising at the end? I agree that both the EU (from where I'm from) and the US have chosen to take a very dark and authoritarian view on immigration, but I think that that discussion doesn't belong in a Python release announcement.
- nomdep 3y ago
  It's a US cultural thing
- brobinson 3y ago
  Luxury beliefs (see: NYC right now)
nickdrozd 3y ago
Running my project's computation-intensive test suite takes 83 seconds under 3.11 and 67 seconds under 3.12. That's a pretty good improvement!
- freediver 3y ago
  I was not able to reproduce and speed improvements over 3.11.
georgehotelling 3y ago
I'm just happy for itertools.batched for chunking iterables: https://docs.python.org/3.12/library/itertools.html#itertools.batched https://docs.python.org/3.12/library/itertools.html#itertool...
- kastden 3y ago
  This is the greatest addition since f-strings!
- philshem 3y ago
  Yes. I’ve written explicit code that needed this 100s of times.
  - intalentive 3y ago
    Yeah I've memorized it by now: for i in range(len(lst) // batch_size + 1): batch = lst[i * batch_size : (i + 1) * batch_size]
    - hddqsb 3y ago
      You have a minor bug -- when len(lst) is a multiple of batch_size, this will have an extra iteration at the end with an empty batch. The fixed version is `range((len(lst) + batch_size - 1) // batch_size)`, which emulates `ceil(len(lst) / batch_size)`. Yet more proof that this should be part of stdlib :) Personally I think I'd actually write it like this: for i in range(0, len(lst), batch_size): batch = lst[i:i+batch_size] The docs give another pretty nice implementation using iter() and islice() in a loop (but it uses the walrus operator `:=` so it requires Python 3.8+ as written).
      - rossant 3y ago
        Blatant reason why a native solution was long overdue.
      - deleted 3y ago
        [deleted]
- kzrdude 3y ago
  Great call-out! (Mistakes elided..)
  - LegionMammal978 3y ago
    What do you mean by "empty sequence in"? The function doesn't raise if the input iterable is empty: it only raises if the chunk size n is 0. While that does have a natural interpretation of returning an infinite sequence of empty tuples, such a behavior would be qualitatively different than for other chunk sizes. The caller would never be able to retrieve any elements from the input iterable, and the output would be infinite even if the input is finite. In that light, it makes some sense (IMO) to avoid letting applications hit such an edge case unintentionally.
    - kzrdude 3y ago
      I skimmed that too quickly and was mistaken. Thanks.
  - deleted 3y ago
    [deleted]
- zem 3y ago
  yeah! that's been in the ruby stdlib practically from day one, no idea why python was so resistant to it.
- ehsankia 3y ago
  99% of my more_itertools imports are exactly for this. there's 1-2 other stuff from more_itertools that I think should make it to itertools. I'd actually like to see statistics from huge monorepos/opensource about usage stats of various more_itertools functions.
  - miiiiiike 3y ago
    Same but for the ‘batch’, ‘ibatch’, and ‘abatch’ functions I started writing back in 2008.
- abyesilyurt 3y ago
  Checkout more-itertools for more variants: https://pypi.org/project/more-itertools/ https://pypi.org/project/more-itertools/
philshem 3y ago
Not yet available through Homebrew. But this link will go live when it is: https://formulae.brew.sh/formula/python@3.12 https://formulae.brew.sh/formula/python@3.12
- xavdid 3y ago
  pro tip: don't install user python using homebrew; use a dedicated version manager (like pyenv or adsf). More info: https://justinmayer.com/posts/homebrew-python-is-not-for-you/ https://justinmayer.com/posts/homebrew-python-is-not-for-you...
  - karlicoss 3y ago
    Another thing I noticed is that homebrew python was noticeably slower on M2 comparing to the pyenv one. I imagine homebrew compiles it with too generic flags to support wide range of macs.
    - shpx 3y ago
      From my looking around, Homebrew's Python 3.11 has specific binaries for each macOS and CPU architecture https://github.com/Homebrew/homebrew-core/blob/c1321e262920354d26cb758b2f5fd3c2ac946ca7/Formula/p/python%403.11.rb#L14-L22 https://github.com/Homebrew/homebrew-core/blob/c1321e2629203... A comment in Homebrew's formula says that Homebrew adds the --enable-optimizations flag during compilation because Homebrew has separate binaries whereas the official Python release doesn't add this flag because "they want one build that will work across many macOS releases" https://github.com/Homebrew/homebrew-core/blob/c1321e262920354d26cb758b2f5fd3c2ac946ca7/Formula/p/python%403.11.rb#L141-L145 https://github.com/Homebrew/homebrew-core/blob/c1321e2629203... and it looks to me that pyenv just installs the official binary https://github.com/pyenv/pyenv/blob/28e7000b485bff61235d8a691c3989c9a5ed0a53/plugins/python-build/share/python-build/3.11.5#L7 https://github.com/pyenv/pyenv/blob/28e7000b485bff61235d8a69... So I would expect it to be the other way around, Homebrew's Python should be faster.
justinmayer 3y ago
Python 3.12 is now available via asdf [1]: asdf install python 3.12.0 [1]: https://justinmayer.com/posts/homebrew-python-is-not-for-you/ https://justinmayer.com/posts/homebrew-python-is-not-for-you...
systemvoltage 3y ago
I don't know much about the internals of python (or compiler theory stuff), but it would be cool to productionize a script with types and make a static binary out of it. pypy/mypy and other projects are fine but they don't support latest python versions and libraries. I want the ease of python but fast once you lock down the dynamic programming features.
- TheFlyingFish 3y ago
  You may be interested in Mojo: https://modular.com/Mojo https://modular.com/Mojo Kind of a crazy project but the "just go ahead and import any Python module and we'll embed the VM in your binary" approach is interesting, at least.
godelski 3y ago
They keep getting improved error messaging and this is one of my favorite features. But I'd love if we could get some real rich text. Idk if anyone else uses rich, but it has infected all my programs now. Not just to print with colors, but because it makes debugging so much easier. Not just print(f"{var=}") but the handler[0,1]. Color is so important to these types of things and so is formatting. Plus, the progress bars are nice and have almost completely replaced tqdm for me[2]. They're just easier and prettier. [0] https://rich.readthedocs.io/en/stable/logging.html https://rich.readthedocs.io/en/stable/logging.html [1] Try this example: https://github.com/Textualize/rich/blob/master/examples/exception.py https://github.com/Textualize/rich/blob/master/examples/exce... [2] Side note: does anyone know how to get these properly working when using DDP with pytorch? I get flickering when using this and I think it is actually down to a pytorch issue and how they're handling their loggers and flushing the screen. I know pytorch doesn't want to depend on rich, but hey, pip uses rich so why shouldn't everyone?
- brenns10 3y ago
  I love and use rich too, but gosh I hope that libraries don't start depending on it just because pip does. It has a lot of dependencies of its own, and dependency creep is real. I know pytorch isn't exactly lightweight in terms of dependencies. But I prefer using libraries that make an effort do only pull in absolutely necessary dependencies.
  - godelski 3y ago
    Yeah sorry I don't think I was clear. I don't exactly want to just drop in rich into the python source (for reasons you mention). But I do think they could take some of the ideas from there and place them in. Formatting is really the most important aspect here, especially around traces because these are the real work amplifiers. So much time is spent debugging that the better tools we have to debug better the more work __everyone__ does. But debugging is strangely a underappreciated area. I think you could do colors with just simple ansii encodings (same as you'd do in posix). I'm just using rich as an example of style. r.e. pytorch: It's a love hate with me. I do think they should incorporate things that are extremely common and solve things that are daily issues. As a simple example, new users are often confused with loading and saving models when using distributed data parallel (DDP) because it creates this extra "module" name in the state_dict and so can require different usage for saving/loading models if you're distributed training or not. This can be quite annoying. Similarly there are no built in infinite samplers, which are common among generative modelers. People who don't iterate over epochs of data, but rather steps. There's of course many solutions to deal with this, but it does make sense with how prolific it is (and has been since 2015) that there just be a built in dataloader. I'd argue things like progress bars and loggers would also be highly beneficial, especially because pytorch's forte is generating research code. But we're digressing. These are just opinions.
tandav 3y ago
I wish they make it possible to use any, callable builtins for type annotations
- kuratkull 3y ago
  You can. def foobar(things: list[tuple[int, Floob | None]]) -> dict[int, int]): Possible since 3.11 I think, maybe even 3.10.
  - SushiHippie 3y ago
    You can use it since 3.7 with `from __future__ import annotations`
  - extasia 3y ago
    I think they mean the literal builtins `any` and `callable` instead of typing.Any and typing.Callable.
overflyer 3y ago
I just really, really hope the Arch people manage to roll out Python 3.12 faster then they needed for 3.11 _
bobbylarrybobby 3y ago
After seeing the new, lax f-strings, I'm realizing I'm pretty content with Rust’s identifiers-only approach to inline string formatting.
smacke 3y ago
It's not in the highlights, but one of the things that excites me most is this: https://docs.python.org/dev/whatsnew/3.12.html#pep-669-low-impact-monitoring-for-cpython https://docs.python.org/dev/whatsnew/3.12.html#pep-669-low-i... > PEP 669 defines a new API for profilers, debuggers, and other tools to monitor events in CPython. It covers a wide range of events, including calls, returns, lines, exceptions, jumps, and more. This means that you only pay for what you use, providing support for near-zero overhead debuggers and coverage tools. See sys.monitoring for details. Low-overhead instrumentation opens up a whole bunch of interesting interactive use cases (i.e. Jupyter etc.), and as the author of one library that relies heavily on instrumentation (https://github.com/ipyflow/ipyflow https://github.com/ipyflow/ipyflow), I'm very keen to explore the possibilities here.
- maegul 3y ago
  Great insight! I wouldn’t have made that connection back to reactive runtimes. Excited to see what you can do with ipyflow!
throw1234651234 3y ago
Oh hey, the useless (except for leetcode) language got an update. Yay, maybe more people try to write boring business logic without type safety. Can't wait.
- hcks 3y ago
  1/ it’s literally the most used PL atm 2/ python is type safe, actually 3/ if you need python for leetcode you can’t really look down on anyone
  - throw1234651234 3y ago
    1. WordPress was too for a time, despite being slow and choke-full of vulnerabilities. 2. Right, at run time. My great-grandma always did say "the best time to catch bugs is in production, use Python sonny". 3. It's not that I need it, it's just that I can't think of any other use for it. Hopefully big data and IoT catches on to the first two points soon enough.
- magarnicle 3y ago
  Weird, apparently all those things I've got done with Python didn't actually get done. I was just imagining it.
- syndicatedjelly 3y ago
  I recently took part A of Dan Grossman's Programming Languages' course, and he focus on Standard ML which is strongly-typed. It was enlightening to learn about, and after completing Part A, I too felt that any language that lacks type safety can't be a serious language. I recently started using Python again for a side project, and I had forgotten how good of a dev experience Python offers. I wish I could explain better than that though. Part B of that course is about weakly typed languages like Ruby. Your comment is motivating me to go and finish Part B, because I wish I could articulate better what exactly it is that makes weakly typed languages useful in their own right.
  - throw1234651234 3y ago
gorgoiler 3y ago
What is the rationale for types being part of the syntax but without shipping a built in type checker? Are we being conservative and waiting to see where the four type checkers — mypy, pyrite, pyre, pytype, and others? — lead us before rolling one into the standard distribution?
- heavyset_go 3y ago
  Read the PEP for Type Hints, the rationale is laid out there. Python will probably never have a built-in type checker beyond Python's existing strong typing at runtime.
pyuser583 3y ago
I used to wait 9 months before upgrading, so the third party packages could be update. Is that still an issue?
- themeiguoren 3y ago
  Check the list here for the biggest packages: https://pyreadiness.org/3.12/ https://pyreadiness.org/3.12/
om26er 3y ago
I most curious to know what happened to the faster-python project progress. I see there were some changes contributed by Mark Shannon and Eric Snow but the performance "upto 5x in 4 releases" doesn't look like happening. I guess the biggest question is is JIT still being pursued?
mik09 3y ago
"More flexible f-string parsing" sweet
est 3y ago
> PEP 684, a unique per-interpreter GIL We are on the track to remove GIL!
- benrutter 3y ago
  I dunno man, seems like there's more GILs than ever before now you can have one per interpreter. . .
- skrause 3y ago
  This change is completely separate from nogil, not it’s first step.
  - yxhuvud 3y ago
    Yeah, it is essentially the same idea as the guild thing in Ruby, and Ruby is explicitly not working towards nogvl.
bloopernova 3y ago
What's new: https://docs.python.org/dev/whatsnew/3.12.html https://docs.python.org/dev/whatsnew/3.12.html Summary, sorry for poor formatting, I'm not sure HN can do a list of any kind? New features More flexible f-string parsing, allowing many things previously disallowed (PEP 701). Support for the buffer protocol in Python code (PEP 688). A new debugging/profiling API (PEP 669). Support for isolated subinterpreters with separate Global Interpreter Locks (PEP 684). Even more improved error messages. More exceptions potentially caused by typos now make suggestions to the user. Support for the Linux perf profiler to report Python function names in traces. Many large and small performance improvements (like PEP 709 and support for the BOLT binary optimizer), delivering an estimated 5% overall performance improvement. Type annotations New type annotation syntax for generic classes (PEP 695). New override decorator for methods (PEP 698). Deprecations The deprecated wstr and wstr_length members of the C implementation of unicode objects were removed, per PEP 623. In the unittest module, a number of long deprecated methods and classes were removed. (They had been deprecated since Python 3.1 or 3.2). The deprecated smtpd and distutils modules have been removed (see PEP 594 and PEP 632. The setuptools package continues to provide the distutils module. A number of other old, broken and deprecated functions, classes and methods have been removed. Invalid backslash escape sequences in strings now warn with SyntaxWarning instead of DeprecationWarning, making them more visible. (They will become syntax errors in the future.) The internal representation of integers has changed in preparation for performance enhancements. (This should not affect most users as it is an internal detail, but it may cause problems for Cython-generated code.)
- liquidpele 3y ago
  Tbh the f-string stuff sounds pretty in-pythonic to me… like it would make it much harder to read and abuse.
  - notatallshaw 3y ago
    What is or isn't "pythonic" is largely determined by the community not the language. Nothing is stopping the Python community from monkey patching everything like the Ruby community does. The f-string changes arrived because there was a need to formalize the syntax, so other Python parsers, for CPython to move off having a special sub-parser just for f-strings, and to be able decide whether weird edge cases were bugs or features. Once formalized it was decided not to put arbitrary limits on it just because people can write badly formatted code, people are can already do that and it's up to the Python community to choose what is or isn't "Pythonic". FYI, one of the things I'm really looking forward to is being able to write: f"Rows: {'\n'.join(rows)}\n Done!" Which in Python 3.11 is illegal syntax.
lcenjoyer 3y ago
Thank god for type aliases
- Leynos 3y ago
  Yeah, outside of the sub-interpreters work, this is probably the change I am most happy about.
- memco 3y ago
  I am excited for the typeddict additions, but it looks like there’s still some work around unpacking needed before we can use it to get around using kwargs to forward arguments from one function to another without having to know the other’s signature.
zozbot234 3y ago
Wait, what happened to Python 3.11 for Workgroups? Something's not right here.
- titaniumtown 3y ago
  That got a chuckle out of me, good one.
- speed_spread 3y ago
  A skipped version? Nothing Novell about that.
  - theandrewbailey 3y ago
    Python 3.11 definitely exists. https://www.python.org/downloads/release/python-3115/ https://www.python.org/downloads/release/python-3115/
    - esafak 3y ago
      https://en.wikipedia.org/wiki/Windows_3.1x#Windows_for_Workgroups_3.11 https://en.wikipedia.org/wiki/Windows_3.1x#Windows_for_Workg...
    - baby_souffle 3y ago
      Novell was an alternative networking technology at the time of windows 3.11
bluish29 3y ago
I think the support for isolated sub-interpreters with separate Global Interpreter Locks is the most interesting new feature in python. It is doubtful not the best way to offer some sort of concurrency but still a step closer to maybe one day get rid of GILs.
- yxhuvud 3y ago
  It may not be a step towards that. Ruby has guilds which is a very similar idea and they are explicitly not working towards removing them altogether at this point. Matz did a full keynote on the latest Euruku defending not working towards removing the whole of it. See https://www.youtube.com/watch?v=5WmhTMcnO7U&t=1244s https://www.youtube.com/watch?v=5WmhTMcnO7U&t=1244s for the full talk if you are interested. I find the convergent evolution of features in these two languages pretty funny, as it is very clear that they don't really look at implementation details of the other language even if they quite often land on ideas that are pretty close in practice.
- zozbot234 3y ago
  Well, that's where the whole "for Workgroups" part comes in I suppose. See my sibling comment.
- qsort 3y ago
  Too bad there's no Python API available yet :( It's scheduled to be delivered next year if I'm not mistaken. The new syntax for generic types is also a very nice QOL improvement, you can now just do: class MyList[T]: ...
  - remram 3y ago
    I think you are talking about the "stable Python ABI"?
  - rmbyrro 3y ago
    Yes, they say it's scheduled for 3.13. But it seems not confirmed yet (probably still not ready for prime time).
- 3y ago
Alosra 3y ago
Everything aside, I enjoyed the "And now for something completely different" part.
- gettodachoppa 3y ago
  It's so hecking wholesome I started crying. My wife's son came to ask me, "Frank, why you crying?". I had to explain to him there's evil people in this world who oppose open borders.
  - CivBase 3y ago
    Making some pretty harsh assumptions about a massive and diverse group of people here.
  - goatlover 3y ago
    Ah yes. People who disagree with you are evil. Maybe ask them why they disagree. Maybe they see societal problems with open borders. Maybe there's a reason governments don't just have open borders, and instead try to manage the immigration process.
- lvass 3y ago
  I dislike it because it's kinda hard to understand (can't really say I did) specially without any context, and programming language release notes are something people should read even a hundred years afterwards.
  - seanhunter 3y ago
    Why should programming language release notes be something people should be able to read in a hundred years? Secondly, why does adding this section affect that even if you think that?
  - paulddraper 3y ago
    I'm curious about the circumstance leading to reading Python 3.12 release notes in 2123.
thatxliner 3y ago
Yay new generic syntax…
Jean-Papoulos 3y ago
Holy shit, actual multithreading. Never thought I'd see the day, good on the Python team for making such great improvements.
- skitter 3y ago
  Eh, you still can't pass Python objects between them. This just means that if you run multiple instances of Python, you can do so in the same process. If you use the C api.
- jjgreen 3y ago
  Steady now ... The features of Per-Interpreter GIL are - for now - only available using C-API, so there's no direct interface for Python developers. Such interface is expected to come with PEP 554, which - if accepted - is supposed to land in Python 3.13, until then we will have to hack our way to the sub-interpreter implementation. https://martinheinz.dev/blog/97 https://martinheinz.dev/blog/97
  - takeda 3y ago
    If gets approved https://peps.python.org/pep-0703/ https://peps.python.org/pep-0703/ would be much better. Yes, it might require code and package changes, but if people see that having python without GIL is an option if some less used features aren't used they likely would jump on it.
- qprofyeh 3y ago
  Ergonomic multi-core multithreading is already solved, for me at least, with ProcessPoolExecutor.map(). I wonder what advantages this direction brings to the table.
  - sitkack 3y ago
    You will now be able to, in 3.12 run multiple CPython VMs inside the same process w/o having them share state. Previously CPython VM state was spread over a bunch of global variables so you had one GIL per process embedding Python. For your usecase, eventually that ProcessPoolExecutor.map will be GillFreeThreadPoolExecutor.map and all the cross process serialization shenanigans will go away.
bloopernova 3y ago
oh cool, multi-line f-strings. With inline comments! f"This is the playlist: {", ".join([ 'Take me back to Eden', # My, my, those eyes like fire 'Alkaline', # Not acid nor alkaline 'Ascensionism' # Take to the broken skies at last ])}"
- pcerdam 3y ago
  Some great choices in that playlist ;)
- adrian17 3y ago
  This probably isn't a good motivating example - my first thought was that this is an overcomplicated way to write "This is the playlist: " + ", ".join([ 'Take me back to Eden', # My, my, those eyes like fire 'Alkaline', # Not acid nor alkaline 'Ascensionism' # Take to the broken skies at last ]) Which is easier to read and exactly as long. (In fact, without 3.12-compatible syntax highlighting, my first intuition upon reading the first line would be to suspect the code of having a typo.)
- js2 3y ago
  I don't find that example particularly compelling. To me it's more legible using regular strings: "This is the playlist: " + ", ".join([ "Take me back to Eden", # My, my, those eyes like fire "Alkaline", # Not acid nor alkaline "Ascensionism", # Take to the broken skies at last ])" edit: jinx
thatcherthorn 3y ago
As now f-strings can contain any valid Python expression inside expression components, it is now possible to nest f-strings arbitrarily: >>> f"{f"{f"{f"{f"{f"{1+1}"}"}"}"}"}" '2' Is anyone aware of the change to the interpreter that allows for this?
- striking 3y ago
  At https://peps.python.org/pep-0701/#motivation https://peps.python.org/pep-0701/#motivation > When f-strings were originally introduced in PEP 498, the specification was provided without providing a formal grammar for f-strings. Additionally, the specification contains several restrictions that are imposed so the parsing of f-strings could be implemented into CPython without modifying the existing lexer. [...] > The other issue that f-strings have is that the current implementation in CPython relies on tokenising f-strings as STRING tokens and a post processing of these tokens. This has the following problems: [...] At https://peps.python.org/pep-0701/#rationale https://peps.python.org/pep-0701/#rationale > By building on top of the new Python PEG Parser (PEP 617), this PEP proposes to redefine “f-strings”, especially emphasizing the clear separation of the string component and the expression (or replacement, {...}) component.
- DecoPerson 3y ago
  JavaScript has this with template strings. I use it often: s = `something ${cond ? `(${v})` : ''} something`
- mirekrusin 3y ago
  js can do it $ node > `${`${`${1+1}`}`}` '2'
- jhvkjhk 3y ago
  Parse it recursively.
  - mathisfun123 3y ago
    yup
- 3y ago
smallerfish 3y ago
The new generic typing seems intuitive - https://docs.python.org/3.12/whatsnew/3.12.html#pep-695-type-parameter-syntax https://docs.python.org/3.12/whatsnew/3.12.html#pep-695-type.... I've never run into its gross predecessor: https://peps.python.org/pep-0484/#generics https://peps.python.org/pep-0484/#generics. Nice change.
formerly_proven 3y ago
Breaking changes: PEP 632: Remove the distutils package. See the migration guide for advice replacing the APIs it provided. The third-party Setuptools package continues to provide distutils, if you still require it in Python 3.12 and beyond. gh-95299: Do not pre-install setuptools in virtual environments created with venv. This means that distutils, setuptools, pkg_resources, and easy_install will no longer available by default; to access these run pip install setuptools in the activated virtual environment. The asynchat, asyncore, and imp modules have been removed, along with several unittest.TestCase method aliases.
deleted 3y ago
[deleted]
roger_ 3y ago
The new `type` statement with lazy evaluation looks interesting. Wonder if it could be used to add lazy evaluation to other libraries, like numpy?
pytlicek 3y ago
:) Oh, this is neat! Python 3.12.0 has brought some cool features to the table. I'm particularly keen on the flexible f-string parsing and the performance improvements. The new type annotation syntax for generic classes is a nice touch too. It's always good to see Python getting refined with each release. Looking forward to exploring these new additions...
reaperman 3y ago
What are the best ways to utilize PEP 669? I’m always looking for two things: a good “time-travel” debugging experience (PyCrunch-Trace seems to be broken for me) and for a way to generate and view large traces (Firefox Performance Profiler can’t open traces that lasted 2-5 minutes because they're too large) PEP 669: Low impact monitoring for CPython PEP 669 defines a new API for profilers, debuggers, and other tools to monitor events in CPython. It covers a wide range of events, including calls, returns, lines, exceptions, jumps, and more. This means that you only pay for what you use, providing support for near-zero overhead debuggers and coverage tools. See sys.monitoring for details.
- parttimenerd 3y ago
  It's an interesting new API. I hope to write a blog post on implementing a debugger with it soon.
dandiep 3y ago
Ooh, seems there is a new syntax for declaring the types of kwargs [1]: from typing import TypedDict, Unpack class Movie(TypedDict): name: str year: int def foo(*kwargs: Unpack[Movie]): ... Maybe now I'll be able to actually figure out what data to send libraries without actually reading their source code. 1. https://docs.python.org/3.12/whatsnew/3.12.html#pep-692-using-typeddict-for-more-precise-kwargs-typing https://docs.python.org/3.12/whatsnew/3.12.html#pep-692-usin...
- ledauphin 3y ago
  what's especially interesting about this is that it could create a new "meta" for static typing in Python. One significant issue with static typing in Python is how much boilerplate is required to use types when also doing the sorts of things that dynamic Python is really good at - for instance proxying functions. If you want to do that now and preserve the types, you need to re-declare the types of everything in the wrapper. Now, if the underlying function already made use of Unpack, you could "reuse" that type in your own wrapper with low boilerplate and less chance of things diverging in hard-to-refactor ways.
- zackees 3y ago
  [dead]
- dagmx 3y ago
  I wonder if Unpack works on a function? I assume it’s any callable.
  - squeaky-clean 3y ago
    The type that gets Unpack[]'d needs to inherit from TypeDict, unless I'm misunderstanding what you mean.
    - dagmx 3y ago
      Basically I was hoping to use it for functions that wrap others. E.g setup something and then pass the kwargs through to the other function.
hansvm 3y ago
The f-string nesting feature is nice. After having that sort of arbitrary interpolation in C# it always annoyed me that something as dynamic as Python couldn't figure it out.
vb-8448 3y ago
Benchmarks aren't too promising[0]. I wonder if the original 500% improvement they targeted at the start of the `faster cpython` project is still a realistic target. [0] https://github.com/faster-cpython/benchmarking-public https://github.com/faster-cpython/benchmarking-public
- Varriount 3y ago
  Would those benchmarks capture the performance improvements listed in the change log? From what I read, there were two big ones: - Inlined list and set comprehensions. - Reduced data copying in asyncio.
  - vb-8448 3y ago
    They use the `pyperformance` package[0] that, according to the readme, use real world scenarios. So I expect it to capture every significant improvement. [0]https://github.com/python/pyperformance https://github.com/python/pyperformance
- notatallshaw 3y ago
  My understanding is most low hanging fruit were picked for Python 3.11. And the faster CPython team have been looking at more mid to long term goals with the addition of a JIT and other accompanying infrastructure around for 3.13 and 3.14. However, the recent no-GIL decision I think has sent a few things back to the drawing board to see what can and cannot be salvaged from their progress and plans so far.
  - vb-8448 3y ago
    Sad ... I think python need some performance improvement more than the no-gil stuff.

652 points by qsort 3y ago

What's with the politicising at the end? I agree that both the EU (from where I'm from) and the US have chosen to take a very dark and authoritarian view on immigration, but I think that that discussion doesn't belong in a Python release announcement.

nomdep 3y ago
It's a US cultural thing
brobinson 3y ago
Luxury beliefs (see: NYC right now)

nickdrozd 3y ago

Running my project's computation-intensive test suite takes 83 seconds under 3.11 and 67 seconds under 3.12. That's a pretty good improvement!

freediver 3y ago
I was not able to reproduce and speed improvements over 3.11.

georgehotelling 3y ago

I'm just happy for itertools.batched for chunking iterables: https://docs.python.org/3.12/library/itertools.html#itertools.batched https://docs.python.org/3.12/library/itertools.html#itertool...

kastden 3y ago
This is the greatest addition since f-strings!
philshem 3y ago
Yes. I’ve written explicit code that needed this 100s of times.
- intalentive 3y ago
  Yeah I've memorized it by now: for i in range(len(lst) // batch_size + 1): batch = lst[i * batch_size : (i + 1) * batch_size]
  - hddqsb 3y ago
    You have a minor bug -- when len(lst) is a multiple of batch_size, this will have an extra iteration at the end with an empty batch. The fixed version is `range((len(lst) + batch_size - 1) // batch_size)`, which emulates `ceil(len(lst) / batch_size)`. Yet more proof that this should be part of stdlib :) Personally I think I'd actually write it like this: for i in range(0, len(lst), batch_size): batch = lst[i:i+batch_size] The docs give another pretty nice implementation using iter() and islice() in a loop (but it uses the walrus operator `:=` so it requires Python 3.8+ as written).
    - rossant 3y ago
      Blatant reason why a native solution was long overdue.
    - deleted 3y ago
      [deleted]
kzrdude 3y ago
Great call-out! (Mistakes elided..)
- LegionMammal978 3y ago
  What do you mean by "empty sequence in"? The function doesn't raise if the input iterable is empty: it only raises if the chunk size n is 0. While that does have a natural interpretation of returning an infinite sequence of empty tuples, such a behavior would be qualitatively different than for other chunk sizes. The caller would never be able to retrieve any elements from the input iterable, and the output would be infinite even if the input is finite. In that light, it makes some sense (IMO) to avoid letting applications hit such an edge case unintentionally.
  - kzrdude 3y ago
    I skimmed that too quickly and was mistaken. Thanks.
- deleted 3y ago
  [deleted]
zem 3y ago
yeah! that's been in the ruby stdlib practically from day one, no idea why python was so resistant to it.
ehsankia 3y ago
99% of my more_itertools imports are exactly for this. there's 1-2 other stuff from more_itertools that I think should make it to itertools. I'd actually like to see statistics from huge monorepos/opensource about usage stats of various more_itertools functions.
- miiiiiike 3y ago
  Same but for the ‘batch’, ‘ibatch’, and ‘abatch’ functions I started writing back in 2008.
abyesilyurt 3y ago
Checkout more-itertools for more variants: https://pypi.org/project/more-itertools/ https://pypi.org/project/more-itertools/

philshem 3y ago

Not yet available through Homebrew. But this link will go live when it is: https://formulae.brew.sh/formula/python@3.12 https://formulae.brew.sh/formula/python@3.12

xavdid 3y ago
pro tip: don't install user python using homebrew; use a dedicated version manager (like pyenv or adsf). More info: https://justinmayer.com/posts/homebrew-python-is-not-for-you/ https://justinmayer.com/posts/homebrew-python-is-not-for-you...
- karlicoss 3y ago
  Another thing I noticed is that homebrew python was noticeably slower on M2 comparing to the pyenv one. I imagine homebrew compiles it with too generic flags to support wide range of macs.
  - shpx 3y ago
    From my looking around, Homebrew's Python 3.11 has specific binaries for each macOS and CPU architecture https://github.com/Homebrew/homebrew-core/blob/c1321e262920354d26cb758b2f5fd3c2ac946ca7/Formula/p/python%403.11.rb#L14-L22 https://github.com/Homebrew/homebrew-core/blob/c1321e2629203... A comment in Homebrew's formula says that Homebrew adds the --enable-optimizations flag during compilation because Homebrew has separate binaries whereas the official Python release doesn't add this flag because "they want one build that will work across many macOS releases" https://github.com/Homebrew/homebrew-core/blob/c1321e262920354d26cb758b2f5fd3c2ac946ca7/Formula/p/python%403.11.rb#L141-L145 https://github.com/Homebrew/homebrew-core/blob/c1321e2629203... and it looks to me that pyenv just installs the official binary https://github.com/pyenv/pyenv/blob/28e7000b485bff61235d8a691c3989c9a5ed0a53/plugins/python-build/share/python-build/3.11.5#L7 https://github.com/pyenv/pyenv/blob/28e7000b485bff61235d8a69... So I would expect it to be the other way around, Homebrew's Python should be faster.

justinmayer 3y ago

Python 3.12 is now available via asdf [1]: asdf install python 3.12.0 [1]: https://justinmayer.com/posts/homebrew-python-is-not-for-you/ https://justinmayer.com/posts/homebrew-python-is-not-for-you...

systemvoltage 3y ago

I don't know much about the internals of python (or compiler theory stuff), but it would be cool to productionize a script with types and make a static binary out of it. pypy/mypy and other projects are fine but they don't support latest python versions and libraries. I want the ease of python but fast once you lock down the dynamic programming features.

TheFlyingFish 3y ago
You may be interested in Mojo: https://modular.com/Mojo https://modular.com/Mojo Kind of a crazy project but the "just go ahead and import any Python module and we'll embed the VM in your binary" approach is interesting, at least.

godelski 3y ago

They keep getting improved error messaging and this is one of my favorite features. But I'd love if we could get some real rich text. Idk if anyone else uses rich, but it has infected all my programs now. Not just to print with colors, but because it makes debugging so much easier. Not just print(f"{var=}") but the handler[0,1]. Color is so important to these types of things and so is formatting. Plus, the progress bars are nice and have almost completely replaced tqdm for me[2]. They're just easier and prettier. [0] https://rich.readthedocs.io/en/stable/logging.html https://rich.readthedocs.io/en/stable/logging.html [1] Try this example: https://github.com/Textualize/rich/blob/master/examples/exception.py https://github.com/Textualize/rich/blob/master/examples/exce... [2] Side note: does anyone know how to get these properly working when using DDP with pytorch? I get flickering when using this and I think it is actually down to a pytorch issue and how they're handling their loggers and flushing the screen. I know pytorch doesn't want to depend on rich, but hey, pip uses rich so why shouldn't everyone?

brenns10 3y ago
I love and use rich too, but gosh I hope that libraries don't start depending on it just because pip does. It has a lot of dependencies of its own, and dependency creep is real. I know pytorch isn't exactly lightweight in terms of dependencies. But I prefer using libraries that make an effort do only pull in absolutely necessary dependencies.
- godelski 3y ago
  Yeah sorry I don't think I was clear. I don't exactly want to just drop in rich into the python source (for reasons you mention). But I do think they could take some of the ideas from there and place them in. Formatting is really the most important aspect here, especially around traces because these are the real work amplifiers. So much time is spent debugging that the better tools we have to debug better the more work __everyone__ does. But debugging is strangely a underappreciated area. I think you could do colors with just simple ansii encodings (same as you'd do in posix). I'm just using rich as an example of style. r.e. pytorch: It's a love hate with me. I do think they should incorporate things that are extremely common and solve things that are daily issues. As a simple example, new users are often confused with loading and saving models when using distributed data parallel (DDP) because it creates this extra "module" name in the state_dict and so can require different usage for saving/loading models if you're distributed training or not. This can be quite annoying. Similarly there are no built in infinite samplers, which are common among generative modelers. People who don't iterate over epochs of data, but rather steps. There's of course many solutions to deal with this, but it does make sense with how prolific it is (and has been since 2015) that there just be a built in dataloader. I'd argue things like progress bars and loggers would also be highly beneficial, especially because pytorch's forte is generating research code. But we're digressing. These are just opinions.

tandav 3y ago

I wish they make it possible to use any, callable builtins for type annotations

kuratkull 3y ago
You can. def foobar(things: list[tuple[int, Floob | None]]) -> dict[int, int]): Possible since 3.11 I think, maybe even 3.10.
- SushiHippie 3y ago
  You can use it since 3.7 with `from __future__ import annotations`
- extasia 3y ago
  I think they mean the literal builtins `any` and `callable` instead of typing.Any and typing.Callable.

overflyer 3y ago

I just really, really hope the Arch people manage to roll out Python 3.12 faster then they needed for 3.11 _

bobbylarrybobby 3y ago

After seeing the new, lax f-strings, I'm realizing I'm pretty content with Rust’s identifiers-only approach to inline string formatting.

smacke 3y ago

It's not in the highlights, but one of the things that excites me most is this: https://docs.python.org/dev/whatsnew/3.12.html#pep-669-low-impact-monitoring-for-cpython https://docs.python.org/dev/whatsnew/3.12.html#pep-669-low-i... > PEP 669 defines a new API for profilers, debuggers, and other tools to monitor events in CPython. It covers a wide range of events, including calls, returns, lines, exceptions, jumps, and more. This means that you only pay for what you use, providing support for near-zero overhead debuggers and coverage tools. See sys.monitoring for details. Low-overhead instrumentation opens up a whole bunch of interesting interactive use cases (i.e. Jupyter etc.), and as the author of one library that relies heavily on instrumentation (https://github.com/ipyflow/ipyflow https://github.com/ipyflow/ipyflow), I'm very keen to explore the possibilities here.

maegul 3y ago
Great insight! I wouldn’t have made that connection back to reactive runtimes. Excited to see what you can do with ipyflow!

throw1234651234 3y ago

Oh hey, the useless (except for leetcode) language got an update. Yay, maybe more people try to write boring business logic without type safety. Can't wait.

hcks 3y ago
1/ it’s literally the most used PL atm 2/ python is type safe, actually 3/ if you need python for leetcode you can’t really look down on anyone
- throw1234651234 3y ago
  1. WordPress was too for a time, despite being slow and choke-full of vulnerabilities. 2. Right, at run time. My great-grandma always did say "the best time to catch bugs is in production, use Python sonny". 3. It's not that I need it, it's just that I can't think of any other use for it. Hopefully big data and IoT catches on to the first two points soon enough.
magarnicle 3y ago
Weird, apparently all those things I've got done with Python didn't actually get done. I was just imagining it.
syndicatedjelly 3y ago
I recently took part A of Dan Grossman's Programming Languages' course, and he focus on Standard ML which is strongly-typed. It was enlightening to learn about, and after completing Part A, I too felt that any language that lacks type safety can't be a serious language. I recently started using Python again for a side project, and I had forgotten how good of a dev experience Python offers. I wish I could explain better than that though. Part B of that course is about weakly typed languages like Ruby. Your comment is motivating me to go and finish Part B, because I wish I could articulate better what exactly it is that makes weakly typed languages useful in their own right.
- throw1234651234 3y ago

gorgoiler 3y ago

What is the rationale for types being part of the syntax but without shipping a built in type checker? Are we being conservative and waiting to see where the four type checkers — mypy, pyrite, pyre, pytype, and others? — lead us before rolling one into the standard distribution?

heavyset_go 3y ago
Read the PEP for Type Hints, the rationale is laid out there. Python will probably never have a built-in type checker beyond Python's existing strong typing at runtime.

pyuser583 3y ago

I used to wait 9 months before upgrading, so the third party packages could be update. Is that still an issue?

themeiguoren 3y ago
Check the list here for the biggest packages: https://pyreadiness.org/3.12/ https://pyreadiness.org/3.12/

om26er 3y ago

I most curious to know what happened to the faster-python project progress. I see there were some changes contributed by Mark Shannon and Eric Snow but the performance "upto 5x in 4 releases" doesn't look like happening. I guess the biggest question is is JIT still being pursued?

mik09 3y ago

"More flexible f-string parsing" sweet

est 3y ago

> PEP 684, a unique per-interpreter GIL We are on the track to remove GIL!

benrutter 3y ago
I dunno man, seems like there's more GILs than ever before now you can have one per interpreter. . .
skrause 3y ago
This change is completely separate from nogil, not it’s first step.
- yxhuvud 3y ago
  Yeah, it is essentially the same idea as the guild thing in Ruby, and Ruby is explicitly not working towards nogvl.

bloopernova 3y ago

What's new: https://docs.python.org/dev/whatsnew/3.12.html https://docs.python.org/dev/whatsnew/3.12.html Summary, sorry for poor formatting, I'm not sure HN can do a list of any kind? New features More flexible f-string parsing, allowing many things previously disallowed (PEP 701). Support for the buffer protocol in Python code (PEP 688). A new debugging/profiling API (PEP 669). Support for isolated subinterpreters with separate Global Interpreter Locks (PEP 684). Even more improved error messages. More exceptions potentially caused by typos now make suggestions to the user. Support for the Linux perf profiler to report Python function names in traces. Many large and small performance improvements (like PEP 709 and support for the BOLT binary optimizer), delivering an estimated 5% overall performance improvement. Type annotations New type annotation syntax for generic classes (PEP 695). New override decorator for methods (PEP 698). Deprecations The deprecated wstr and wstr_length members of the C implementation of unicode objects were removed, per PEP 623. In the unittest module, a number of long deprecated methods and classes were removed. (They had been deprecated since Python 3.1 or 3.2). The deprecated smtpd and distutils modules have been removed (see PEP 594 and PEP 632. The setuptools package continues to provide the distutils module. A number of other old, broken and deprecated functions, classes and methods have been removed. Invalid backslash escape sequences in strings now warn with SyntaxWarning instead of DeprecationWarning, making them more visible. (They will become syntax errors in the future.) The internal representation of integers has changed in preparation for performance enhancements. (This should not affect most users as it is an internal detail, but it may cause problems for Cython-generated code.)

liquidpele 3y ago
Tbh the f-string stuff sounds pretty in-pythonic to me… like it would make it much harder to read and abuse.
- notatallshaw 3y ago
  What is or isn't "pythonic" is largely determined by the community not the language. Nothing is stopping the Python community from monkey patching everything like the Ruby community does. The f-string changes arrived because there was a need to formalize the syntax, so other Python parsers, for CPython to move off having a special sub-parser just for f-strings, and to be able decide whether weird edge cases were bugs or features. Once formalized it was decided not to put arbitrary limits on it just because people can write badly formatted code, people are can already do that and it's up to the Python community to choose what is or isn't "Pythonic". FYI, one of the things I'm really looking forward to is being able to write: f"Rows: {'\n'.join(rows)}\n Done!" Which in Python 3.11 is illegal syntax.

lcenjoyer 3y ago

Thank god for type aliases

Leynos 3y ago
Yeah, outside of the sub-interpreters work, this is probably the change I am most happy about.
memco 3y ago
I am excited for the typeddict additions, but it looks like there’s still some work around unpacking needed before we can use it to get around using kwargs to forward arguments from one function to another without having to know the other’s signature.

zozbot234 3y ago

Wait, what happened to Python 3.11 for Workgroups? Something's not right here.

titaniumtown 3y ago
That got a chuckle out of me, good one.
speed_spread 3y ago
A skipped version? Nothing Novell about that.
- theandrewbailey 3y ago
  Python 3.11 definitely exists. https://www.python.org/downloads/release/python-3115/ https://www.python.org/downloads/release/python-3115/
  - esafak 3y ago
    https://en.wikipedia.org/wiki/Windows_3.1x#Windows_for_Workgroups_3.11 https://en.wikipedia.org/wiki/Windows_3.1x#Windows_for_Workg...
  - baby_souffle 3y ago
    Novell was an alternative networking technology at the time of windows 3.11

bluish29 3y ago

I think the support for isolated sub-interpreters with separate Global Interpreter Locks is the most interesting new feature in python. It is doubtful not the best way to offer some sort of concurrency but still a step closer to maybe one day get rid of GILs.

yxhuvud 3y ago
It may not be a step towards that. Ruby has guilds which is a very similar idea and they are explicitly not working towards removing them altogether at this point. Matz did a full keynote on the latest Euruku defending not working towards removing the whole of it. See https://www.youtube.com/watch?v=5WmhTMcnO7U&t=1244s https://www.youtube.com/watch?v=5WmhTMcnO7U&t=1244s for the full talk if you are interested. I find the convergent evolution of features in these two languages pretty funny, as it is very clear that they don't really look at implementation details of the other language even if they quite often land on ideas that are pretty close in practice.
zozbot234 3y ago
Well, that's where the whole "for Workgroups" part comes in I suppose. See my sibling comment.
qsort 3y ago
Too bad there's no Python API available yet :( It's scheduled to be delivered next year if I'm not mistaken. The new syntax for generic types is also a very nice QOL improvement, you can now just do: class MyList[T]: ...
- remram 3y ago
  I think you are talking about the "stable Python ABI"?
- rmbyrro 3y ago
  Yes, they say it's scheduled for 3.13. But it seems not confirmed yet (probably still not ready for prime time).
3y ago

Alosra 3y ago

Everything aside, I enjoyed the "And now for something completely different" part.

gettodachoppa 3y ago
It's so hecking wholesome I started crying. My wife's son came to ask me, "Frank, why you crying?". I had to explain to him there's evil people in this world who oppose open borders.
- CivBase 3y ago
  Making some pretty harsh assumptions about a massive and diverse group of people here.
- goatlover 3y ago
  Ah yes. People who disagree with you are evil. Maybe ask them why they disagree. Maybe they see societal problems with open borders. Maybe there's a reason governments don't just have open borders, and instead try to manage the immigration process.
lvass 3y ago
I dislike it because it's kinda hard to understand (can't really say I did) specially without any context, and programming language release notes are something people should read even a hundred years afterwards.
- seanhunter 3y ago
  Why should programming language release notes be something people should be able to read in a hundred years? Secondly, why does adding this section affect that even if you think that?
- paulddraper 3y ago
  I'm curious about the circumstance leading to reading Python 3.12 release notes in 2123.

thatxliner 3y ago

Yay new generic syntax…

Jean-Papoulos 3y ago

Holy shit, actual multithreading. Never thought I'd see the day, good on the Python team for making such great improvements.

skitter 3y ago
Eh, you still can't pass Python objects between them. This just means that if you run multiple instances of Python, you can do so in the same process. If you use the C api.
jjgreen 3y ago
Steady now ... The features of Per-Interpreter GIL are - for now - only available using C-API, so there's no direct interface for Python developers. Such interface is expected to come with PEP 554, which - if accepted - is supposed to land in Python 3.13, until then we will have to hack our way to the sub-interpreter implementation. https://martinheinz.dev/blog/97 https://martinheinz.dev/blog/97
- takeda 3y ago
  If gets approved https://peps.python.org/pep-0703/ https://peps.python.org/pep-0703/ would be much better. Yes, it might require code and package changes, but if people see that having python without GIL is an option if some less used features aren't used they likely would jump on it.
qprofyeh 3y ago
Ergonomic multi-core multithreading is already solved, for me at least, with ProcessPoolExecutor.map(). I wonder what advantages this direction brings to the table.
- sitkack 3y ago
  You will now be able to, in 3.12 run multiple CPython VMs inside the same process w/o having them share state. Previously CPython VM state was spread over a bunch of global variables so you had one GIL per process embedding Python. For your usecase, eventually that ProcessPoolExecutor.map will be GillFreeThreadPoolExecutor.map and all the cross process serialization shenanigans will go away.

bloopernova 3y ago

oh cool, multi-line f-strings. With inline comments! f"This is the playlist: {", ".join([ 'Take me back to Eden', # My, my, those eyes like fire 'Alkaline', # Not acid nor alkaline 'Ascensionism' # Take to the broken skies at last ])}"

pcerdam 3y ago
Some great choices in that playlist ;)
adrian17 3y ago
This probably isn't a good motivating example - my first thought was that this is an overcomplicated way to write "This is the playlist: " + ", ".join([ 'Take me back to Eden', # My, my, those eyes like fire 'Alkaline', # Not acid nor alkaline 'Ascensionism' # Take to the broken skies at last ]) Which is easier to read and exactly as long. (In fact, without 3.12-compatible syntax highlighting, my first intuition upon reading the first line would be to suspect the code of having a typo.)
js2 3y ago
I don't find that example particularly compelling. To me it's more legible using regular strings: "This is the playlist: " + ", ".join([ "Take me back to Eden", # My, my, those eyes like fire "Alkaline", # Not acid nor alkaline "Ascensionism", # Take to the broken skies at last ])" edit: jinx

thatcherthorn 3y ago

As now f-strings can contain any valid Python expression inside expression components, it is now possible to nest f-strings arbitrarily: >>> f"{f"{f"{f"{f"{f"{1+1}"}"}"}"}"}" '2' Is anyone aware of the change to the interpreter that allows for this?

striking 3y ago
At https://peps.python.org/pep-0701/#motivation https://peps.python.org/pep-0701/#motivation > When f-strings were originally introduced in PEP 498, the specification was provided without providing a formal grammar for f-strings. Additionally, the specification contains several restrictions that are imposed so the parsing of f-strings could be implemented into CPython without modifying the existing lexer. [...] > The other issue that f-strings have is that the current implementation in CPython relies on tokenising f-strings as STRING tokens and a post processing of these tokens. This has the following problems: [...] At https://peps.python.org/pep-0701/#rationale https://peps.python.org/pep-0701/#rationale > By building on top of the new Python PEG Parser (PEP 617), this PEP proposes to redefine “f-strings”, especially emphasizing the clear separation of the string component and the expression (or replacement, {...}) component.
DecoPerson 3y ago
JavaScript has this with template strings. I use it often: s = `something ${cond ? `(${v})` : ''} something`
mirekrusin 3y ago
js can do it $ node > `${`${`${1+1}`}`}` '2'
jhvkjhk 3y ago
Parse it recursively.
- mathisfun123 3y ago
  yup
3y ago

smallerfish 3y ago

The new generic typing seems intuitive - https://docs.python.org/3.12/whatsnew/3.12.html#pep-695-type-parameter-syntax https://docs.python.org/3.12/whatsnew/3.12.html#pep-695-type.... I've never run into its gross predecessor: https://peps.python.org/pep-0484/#generics https://peps.python.org/pep-0484/#generics. Nice change.

formerly_proven 3y ago

Breaking changes: PEP 632: Remove the distutils package. See the migration guide for advice replacing the APIs it provided. The third-party Setuptools package continues to provide distutils, if you still require it in Python 3.12 and beyond. gh-95299: Do not pre-install setuptools in virtual environments created with venv. This means that distutils, setuptools, pkg_resources, and easy_install will no longer available by default; to access these run pip install setuptools in the activated virtual environment. The asynchat, asyncore, and imp modules have been removed, along with several unittest.TestCase method aliases.

deleted 3y ago

[deleted]

roger_ 3y ago

The new `type` statement with lazy evaluation looks interesting. Wonder if it could be used to add lazy evaluation to other libraries, like numpy?

pytlicek 3y ago

:) Oh, this is neat! Python 3.12.0 has brought some cool features to the table. I'm particularly keen on the flexible f-string parsing and the performance improvements. The new type annotation syntax for generic classes is a nice touch too. It's always good to see Python getting refined with each release. Looking forward to exploring these new additions...

reaperman 3y ago

What are the best ways to utilize PEP 669? I’m always looking for two things: a good “time-travel” debugging experience (PyCrunch-Trace seems to be broken for me) and for a way to generate and view large traces (Firefox Performance Profiler can’t open traces that lasted 2-5 minutes because they're too large) PEP 669: Low impact monitoring for CPython PEP 669 defines a new API for profilers, debuggers, and other tools to monitor events in CPython. It covers a wide range of events, including calls, returns, lines, exceptions, jumps, and more. This means that you only pay for what you use, providing support for near-zero overhead debuggers and coverage tools. See sys.monitoring for details.

parttimenerd 3y ago
It's an interesting new API. I hope to write a blog post on implementing a debugger with it soon.

dandiep 3y ago

Ooh, seems there is a new syntax for declaring the types of kwargs [1]: from typing import TypedDict, Unpack class Movie(TypedDict): name: str year: int def foo(*kwargs: Unpack[Movie]): ... Maybe now I'll be able to actually figure out what data to send libraries without actually reading their source code. 1. https://docs.python.org/3.12/whatsnew/3.12.html#pep-692-using-typeddict-for-more-precise-kwargs-typing https://docs.python.org/3.12/whatsnew/3.12.html#pep-692-usin...

ledauphin 3y ago
what's especially interesting about this is that it could create a new "meta" for static typing in Python. One significant issue with static typing in Python is how much boilerplate is required to use types when also doing the sorts of things that dynamic Python is really good at - for instance proxying functions. If you want to do that now and preserve the types, you need to re-declare the types of everything in the wrapper. Now, if the underlying function already made use of Unpack, you could "reuse" that type in your own wrapper with low boilerplate and less chance of things diverging in hard-to-refactor ways.
zackees 3y ago
[dead]
dagmx 3y ago
I wonder if Unpack works on a function? I assume it’s any callable.
- squeaky-clean 3y ago
  The type that gets Unpack[]'d needs to inherit from TypeDict, unless I'm misunderstanding what you mean.
  - dagmx 3y ago
    Basically I was hoping to use it for functions that wrap others. E.g setup something and then pass the kwargs through to the other function.

hansvm 3y ago

The f-string nesting feature is nice. After having that sort of arbitrary interpolation in C# it always annoyed me that something as dynamic as Python couldn't figure it out.

vb-8448 3y ago

Benchmarks aren't too promising[0]. I wonder if the original 500% improvement they targeted at the start of the `faster cpython` project is still a realistic target. [0] https://github.com/faster-cpython/benchmarking-public https://github.com/faster-cpython/benchmarking-public

Varriount 3y ago
Would those benchmarks capture the performance improvements listed in the change log? From what I read, there were two big ones: - Inlined list and set comprehensions. - Reduced data copying in asyncio.
- vb-8448 3y ago
  They use the `pyperformance` package[0] that, according to the readme, use real world scenarios. So I expect it to capture every significant improvement. [0]https://github.com/python/pyperformance https://github.com/python/pyperformance
notatallshaw 3y ago
My understanding is most low hanging fruit were picked for Python 3.11. And the faster CPython team have been looking at more mid to long term goals with the addition of a JIT and other accompanying infrastructure around for 3.13 and 3.14. However, the recent no-GIL decision I think has sent a few things back to the drawing board to see what can and cannot be salvaged from their progress and plans so far.
- vb-8448 3y ago
  Sad ... I think python need some performance improvement more than the no-gil stuff.