7 ms·

Advanced Python Features

393 points by BerislavLopac 1y ago

TekMol 1y ago
TFA's use-case for for/else does not convince me: for server in servers: if server.check_availability(): primary_server = server break else: primary_server = backup_server deploy_application(primary_server) As it is shorter to do this: primary_server = backup_server for server in servers: if server.check_availability(): primary_server = server break deploy_application(primary_server)
- macleginn 1y ago
  I would say that the main benefit of for/else is readability and ease of writing since nothing needs to be front-loaded. (The downside is an arguably redundant addition to the basic syntax, of course.)
- wesselbindt 1y ago
  If you replace the assignment in the else clause with something side-effectful it does make sense. But even then it harms the readability of the code to such a ridiculous extent. I've never not regretted adding an else clause to a for loop. Any cognitive benefits from Python's approach to for loops closely mirroring natural language just go out the window.
- Loranubi 1y ago
  I do like the else clause with for loops. However most people are not familiar with it, and also `else:` as a keyword is confusing. I always remember it as `no break:`.
  - stevesimmons 1y ago
    I think of it as "otherwise:"
    - int_19h 1y ago
      That's still confusing though. The problem here is that `else` is semantically attached to `break`, but syntactically attached to the body of the loop. The latter makes it look like it executes if the loop body didn't, if you interpret it in the most straightforward way. IMO a better design would be to have a block that always executes at the end of the loop - there's even a reasonable keyword for it, `finally` - but gets a boolean flag indicating whether there was a break or not: for server in servers: if server.check_availability(): primary_server = server break finally did_break: if not did_break: primary_server = backup_server Or better yet, make `break` take an optional argument (which defaults to `True` if unspecified), and that's what you get in `finally`. So this could be written: for server in servers: if server.check_availability(): break server finally server: primary_server = server if server is not None else backup_server
- hk__2 1y ago
  What about this? for server in servers: if server.check_availability(): primary_server = server break else: logger.warning("Cannot find a valid server") # <--- primary_server = backup_server deploy_application(primary_server)
  - TekMol 1y ago
    Yes, that might be a better example.
- zahlman 1y ago
  This kind of search can be done a variety of different ways, and is worth abstracting, e.g.: def first(candidates, predicate, default): try: return next(c for c in candidates if predicate(c)) except StopIteration: return default deploy_application(first(servers, Server.check_availability, backup_server))
  - sowhat25 1y ago
    Great, first() comes part of more_itertools !
    - zahlman 1y ago
      `more_itertools` is definitely on my shortlist of things I wish were in the Python standard library. (The list of things I'd like to see cut out, is probably considerably longer. But that's mainly on aesthetic principles; it's probably not creating a significant maintenance burden.) And yet somehow I keep forgetting about it in the exact moments when it would probably make my life easier.
  - nostoc 1y ago
    Instead of catching the `StopIteration` exception, you can simply provide a default case to `next` : next((c for c in candidates if predicate(c)), default)
    - zahlman 1y ago
      Indeed. I was thinking of ways to generalize for the case where a default isn't desired, then decided against introducing that complexity in the example, then forgot that I could re-simplify further.
- paolosimone 1y ago
  If the team is familiar with Functional Programming concepts, I'd actually suggest available_servers = (server for server in servers if server.check_availability()) primary_server = next(available_servers, backup_server) deploy_application(primary_server) which IMHO better conveys the intent to the reader (i.e. focus on "what to do" over "how to do it")
- HelloNurse 1y ago
  The shorter version that saves the else: line assigns primary_server twice: it is OK for an assignment but a potential problem if doing something less trivial, and very ugly style in any case.
macleginn 1y ago
The way the language is evolving, it seems likely to me that people in the applications camp (ML, simple web-dev, etc.) will soon need a "simple Python" fork or at least an agreed-upon subset of the language that doesn't have most of these complications (f-strings are a major success, though).
- globular-toast 1y ago
  The only newish thing on this list appears to be structural pattern matching. Other than that it's all typing stuff which is all optional.
- sgarland 1y ago
  Nothing stops you from simply not using features, and even having CI checks that forbid them. Not sure why you’d need a fork.
pjmlp 1y ago
Here is another one, list and expression comprehensions shared with ML languages (not that AI one), apparently many aren't aware of them. The itertools package.
- agumonkey 1y ago
  and maybe https://more-itertools.readthedocs.io/en/stable/ https://more-itertools.readthedocs.io/en/stable/ never used it but it seems worth a try
Loranubi 1y ago
I would argue that most of these features (basically everything except metaclasses) are not advanced features. These are simple, but for some reason less well known or less used features. Metaclasses are however quite complex (or at least lead to complex behavior) and I mostly avoid them for this reason. And 'Proxy Properties' are not really a feature at all. Just a specific usage of dunder methods.
- hsbauauvhabzb 1y ago
  Simple doesn’t make something basic or beginner level.
fxj 1y ago
This is all fun and games unless you have to debug someone elses code and they use a new feature that you didnt know about. Speaking for myself, I would be glad if there were a python_light version of the interpreter that has a simple syntax only like the early 3.x versions. just my 2 ct
- hk__2 1y ago
  > This is all fun and games unless you have to debug someone elses code and they use a new feature that you didnt know about. What’s wrong about learning things by looking at code from more experienced people?
- HelloNurse 1y ago
  You underestimate the social complexity of a simplified language. Everyone would disagree about the design (what is worth including because it is "simple" or necessary?), everyone would avoid using an handicapped language where features they need are gratuitously missing, and nobody would care about it enough to work on implementation and maintenance.
Starlevel004 1y ago
If context managers are considered advanced I despair at the code you're writing.
- lesser-shadow 1y ago
  As a UD, I didn't know what a context manager was before reading this article, so you are not really far from the truth, tbh.
wismwasm 1y ago
Nice! I would add assert_never to the pattern matching section for exhaustiveness checks: https://typing.python.org/en/latest/guides/unreachable.html#assert-never-and-exhaustiveness-checking https://typing.python.org/en/latest/guides/unreachable.html#...
globular-toast 1y ago
This is a nice list of "things you might not know" that is worth skimming to add to your toolkit. If you are really interested in "advanced Python", though, I would recommend the book Fluent Python by Ramalho. I have the first edition which is still highly relevant, including the async bits (you just have to translate the coroutines into async syntax). There is a second edition which is more up to date. I would also recommend checking out the functools[0] and itertools[1] modules in the standard library. Just go and read the docs on them top to bottom. It's also worth reading the first few sections of Python Data Model[2] and then bookmarking this page. [0] https://docs.python.org/3/library/functools.html https://docs.python.org/3/library/functools.html [1] https://docs.python.org/3/library/itertools.html https://docs.python.org/3/library/itertools.html [2] https://docs.python.org/3/reference/datamodel.html https://docs.python.org/3/reference/datamodel.html
- sgarland 1y ago
  +1 to itertools especially. Absurdly powerful, and the recipes at the bottom of the doc page are terrific. The only problem I’ve found is that for interviews, people often aren’t familiar with it, which can lead to you solving whatever puzzle they had in far less time than they intended, and without manually building whatever logic it was they assumed you would need.
  - globular-toast 1y ago
    > The only problem I’ve found is that for interviews, people often aren’t familiar with it, which can lead to you solving whatever puzzle they had in far less time than they intended, and without manually building whatever logic it was they assumed you would need. My favourite example of a similar thing happening to me was when I was asked to reverse the digits in a number. I somewhat jokingly asked if I was assuming base 10, which got some awkward looks so I knew something was up. They weren't impressed at all with my answer of `"".join(reversed(str(123456789)))`. I didn't get the job.
    - zahlman 1y ago
      >I didn't get the job. Their loss. Expecting an arithmetic solution to be necessary is incongruous with accepting the overhead of Python. IIRC your approach will actually be faster; the string conversion implicitly number-crunches on the bare metal, whereas any arithmetic approach in Python code won't. Unless they wanted you to convert back to int afterwards. Or wanted you to reverse the string with the slicing trick (which is, indeed, quite a bit faster yet).
mvATM99 1y ago
Good list. Some of these i knew already, but the typing overloading and keyword/positional-only arguments were new to me. One personal favorite of mine is __all__ for use in __init__.py files. It specifies which items are imported whenever uses from x import *. Especially useful when you have other people working on your codebase with the tendency to always import everything, which is rarely a good idea.
- hk__2 1y ago
  It’s never a good idea. I use `__all__` to explicitely list my exports in libraries, so that when one write `from mylib import `, the IDE auto-completes only the public classes and functions.
red_admiral 1y ago
This sounds fun if you have 10x programmers or at least IQ > 140 programmers in charge. Last place I worked, I was told never use "smart" tricks if you can do the same thing in a simpler way. For-else and f-strings and := sound like harmless enough (though the last one is controversial); "with" is useful for resources that need to be deallocated but "yield"? Really? "more readable" when you slap a decorator on the thing? Why are we packing the enter and exit operations into a single function? Playing with the typesystem and generics like this makes me worry I'm about to have a panic attack. Give me code that I can understand and debug easily, even when I didn't write it; don't do implicit magical control flow changes unless you have a very good excuse, and then document both the how and the why - and you'll get a product that launches earlier and has fewer bugs. Sometimes, a few more if statements here and there make code that is easier to understand, even if there's a clever hack that could cut a line or two here and there.
- hk__2 1y ago
  I don’t really see "smart tricks" here, just features of the language that you may not be aware of. I use most of them in my daily work without thinking about it: `@overload` and other typing features are useful to document the code, prevent mistakes, and help IDEs generate more helpful suggestions; keyword-only arguments force users to pass options as keywords, so you avoid calls like `foo(x, z, z, true, true, false, true)` and ensure the code is more explicit; for-else can sometimes clarify some complicated code; the walrus operator, short-circuiting with `or`, and chaining operators are just normal Python.
  - mpeg 1y ago
    overload is very useful, but I find some people mistakenly think it actually modifies the runtime function signature to do multiple dispatch, so I do try to avoid it in favour of simpler functions whenever possible too many overloads can be a code smell IMHO, it's ok when you are implementing a very common pattern and want proper types (decorators come to mind, so that both @decorator and @decorator() work, where the decorator might also have optional args) but I think in cases like the example in the article it should almost always be two separate functions
davnn 1y ago
Working in the ML field, I can't hate Python. But the type system (pre-3.12, of course) cost me a lot of nerves. Hoping for a better post-3.12 experience once all libraries are usable in 3.12+. After that experience, I’ve come to truly appreciate TypeScript’s type system. Never thought I’d say that.
- hk__2 1y ago
  Same experience here, Python’s typing experience is awful compared to TypeScript, even post-3.12. Mypy’s type inference is so dumb you have to write arguments like `i: int = 0`; `TypedDict`s seems promisable at first and then end up as a nightmare where you have to `cast` everything. I miss TypeScript’s `unknown` as well.
  - drumnerd 1y ago
    0 can be inferred as a float too, so doesn’t it make sense to type numbers?
    - porridgeraisin 1y ago
      I believe mypy infers i as an integer in i = 0. I remember I had to do i = 0.0 to make it accept i += someFloat later on. Or of course i:float = 0 but I preferred the former.
      - hk__2 1y ago
        Yes, but not in arguments: def f(i=0) -> None: j = i + 1 k = 1 reveal_type(i) reveal_type(j) reveal_type(k) Output: Revealed type is "Any" Revealed type is "Any" Revealed type is "builtins.int"
        jsjohnst 1y ago
        Because it shouldn’t in function arguments. The one defining the function should be responsible enough to know what input they want and actually properly type it. Assuming an int or number type here is wrong (it could be optional int for example).
drumnerd 1y ago
Those are basic, you can decompile a function and change its bytecode, you can completely revamp the parser using codecs, etc
- padjo 1y ago
  Yeah I only worked professionally with python for about 6 months and I knew about most of them.
lordnacho 1y ago
For me, the biggest benefit to python is that it feels like executable pseudocode. The language gets out of the way of your domain level instructions. This is probably why most non-programmers have the easiest time with python. The more fancy stuff you add to it, the less attractive it becomes. Sure, most of these things have some sort of use, but I reckon most people do not get deep enough into python to understand all these little things.
- 7bit 1y ago
  I disagree. Adding "fancy stuff" by definition does not remove what made it attractive to you in the first place. You even acknowledge this in your second sentence. Something you don't know of is not capable to influence your opinion in any way. And when you know of it -- and dislike it -- just keep doing it the way you did before knowing it.
  - pansa2 1y ago
    > And when you know of it -- and dislike it -- just keep doing it the way you did before knowing it. That breaks down as soon as you need to work with anyone else's code that uses the "fancy stuff".
    - deleted 1y ago
      [deleted]
    - 7bit 1y ago
      No, it does not. There's always someone else better than you. That's no fault of the language
- rainingmonkey 1y ago
  Lua is like this only moreso, and there's little fear of "progress" as large parts of the community have stuck with Lua5.1. Of course, Lua is batteries-not-included so there may be the problem of "progress" in external libraries; in practice things like Penlight barely change though.
hrtk 1y ago
Trying so hard to make it a typed language
- maleldil 1y ago
  Yes, please. Given I'm forced to use Python, I'd welcome any "compile time" tools I can get. These days, it's using pyright with strict mode, which is pretty good, but there's still a long way to go.
hiichbindermax 1y ago
Nitpick about 9.3 Short Circuit Evaluation: both things evaluate differently if you have empty strings. The if-else clause treats empty strings as valid while the or operator will treat them equivalent with None.
- TACD 1y ago
  Similarly with 9.2, assignment using a walrus operator will also fail if the value is 0 (or anything falsy: https://docs.python.org/3/library/stdtypes.html#truth-value-testing https://docs.python.org/3/library/stdtypes.html#truth-value-...)
  - maleldil 1y ago
    You can use `if (response := get_user_input()) is not None` if that's important. IME, empty strings and None would be treated the same way.
lyu07282 1y ago
Good list, its one of these things you either never heard of, or used for years and think everyone knew that. I'll add a few: * did you know __init__.py is optional nowadays? * you can do relative imports with things like "from ..other import foo" * since 3.13 there is a @deprecated decorator that does what you think it does * the new generics syntax also works on methods/functions: "def method[T](...)" very cool * you can type kwargs with typeddicts and unpack: "def fn(*kwargs: Unpack[MyKwargs])" * dataclasses (and pydantic) support immutable objects with: "class MyModel(BaseModel, frozen=True)" or "@dataclass(frozen=True)" * class attributes on dataclasses, etc. can be defined with "MY_STATIC: ClassVar[int] = 42" this also supports abstract base classes (ABC) * TypeVar supports binding to enforce subtypes: "TypeVar['T', bound=X]", and also a default since 3.13: "TypeVar['T', bound=X, default=int]" * @overload is especially useful for get() methods to express that the return can't be none if the default isn't None * instead of Union[a, b] or Optional[a] you can write "a | b" or "a | None" nowadays * with match you can use assert_never() to ensure exhaustive matching in a "case _:" block * typing has reveal_type() which lets mypy print the type it thinks something is * typing's "Self" allows you to more properly annotate class method return types * the time package has functions for monotonic clocks and others not just time() anyone know more things?
- zahlman 1y ago
  > did you know __init__.py is optional nowadays? It has an effect, and is usually worth including anyway. I used to omit it by default; now I include it by default. Also, you say "nowadays" but it's been almost 13 years now (https://peps.python.org/pep-0420/ https://peps.python.org/pep-0420/). > since 3.13 there is a @deprecated decorator that does what you think it does Nice find. Probably worth mentioning it comes from the `warnings` standard library. > the time package has functions for monotonic clocks and others not just time() There's quite a bit in there, but I question how many people need it. Anyway, it's always surprising to me how when other people make these lists, such a large fraction is taken up by tricks with type annotations. I was skeptical of the functionality when the `typing` standard library was introduced; I've only grown more and more wary of it, even as people continue to insist to me that it's somehow necessary.
  - 1y ago
krelian 1y ago
Python has evolved into quite a complex language considering the amount of features that come built-in. The documentation, while complete, does not facilitate discovery of many of those features.
- zahlman 1y ago
  The quality of documentation is a known, highlighted concern in the community. There is particular interest in adopting Diataxis principles to improve documentation, both for Python itself and for packaging (see e.g. the user guide at https://packaging.python.org/en/latest/ https://packaging.python.org/en/latest/ , or the PyPA main site https://www.pypa.io/en/latest/ https://www.pypa.io/en/latest/ ). If you want to help, there's a section on the Python forum (https://discuss.python.org/c/documentation/26 https://discuss.python.org/c/documentation/26) and a Discord server, and issues with documentation can also be reported on the main Python GitHub issue tracker (https://github.com/python/cpython/labels/docs https://github.com/python/cpython/labels/docs).
- sakesun 1y ago
  It's more complex that decade ago, but still a relatively simple language. I can understand the article without much effort, while I scratch my head really hard when read about advance feature of Typescript or Scala.
  - pansa2 1y ago
    > still a relatively simple language If only. I suspect very few Python programmers can even fully explain what `a + b` does. If `a` and `b` are instances of classes, many would say it's equivalent to `a.__add__(b)` or `type(a).__add__(a, b)`, but in fact it's much more complex.
    - cl3misch 1y ago
      Care to elaborate? Where is the complexity hidden?
      - 1y ago
shaunpud 1y ago
Another blog with no feed :(
throwwwwwawa 1y ago
[flagged]
robviren 1y ago
Every time I try to use Python I get this mixed feeling of liking how few guard rails there are between me and the logic I am making and this lingering worry that my code looks like fools attempt to use Python. So much exists as convention or loosely followed rules. Whenever I read articles like this I am wowed by the depth of things I didn't know about Python or how much has changed. It makes something like Go feel like a comfort blanket because I have reasonable certainty code I write won't feel deprecated or left behind a year or two later. Excellent article showing off even more I didn't know.
- pydry 1y ago
  There are plenty of language features which let you re-use or build your own guardrails in python. I do with the concept of truthiness and falsiness would be taken out and shot though. It's been responsible for far too many nasty bugs IME and it only cuts out a few extra characters. Not a great trade off.
  - brookst 1y ago
    +1 — and it only saves characters when someone really knows what they’re doing. Most of is end up with `if (not result) or (result == “”)` and worse.
  - Starlevel004 1y ago
    Truthiness and falsiness is one of my more used features. ``if not collection`` comes up in nearly every project multiple times.
    - pydry 1y ago
      Yeah, everybody uses it. That's partly how it ends up causing so many bugs - e.g. some user puts 0 in a text box and instead of the program reacting as if you put 0 in it reacts the same way as if no value was supplied. Then everybody loses their minds because the system suddenly started doing something it was never supposed to.
      - 1y ago
steinuil 1y ago
Turns out I managed to use almost all of these during a refactor of a project at work, even metaclasses... (Metaclass usage is justified in my case: we have a sort of language evaluator and using a metaclass lets us define function arguments with their types and validators in a very coincise and obvious way similar to Pydantic.) I think this list should also include descriptors[0]: it's another metaprogramming feature that allows running code when accessing or setting class attributes similar to @property but more powerful. (edit: nvm, I saw that they are covered in the proxy properties section!) I think the type system is quite good actually, even if you end up having to sidestep it when doing this kind of meta-programming. The errors I do get are generally the library's fault (old versions of SQLAlchemy make it impossible to assign types anywhere...) and there's a few gotchas (like mutable collections being invariant, so if you take a list as an argument you may have to type it as `Sequence[]` or you'll get type errors) but it's functional and makes the language usable for me. I stopped using Ruby because upstream would not commit on type checking (yes I know you have a few choices if you want typing, but they're a bit too much overhead for what I usually use Ruby for, which is writing scripts), and I'm glad Python is committing here. [0]: https://docs.python.org/3/howto/descriptor.html https://docs.python.org/3/howto/descriptor.html
William_BB 1y ago
I enjoyed reading the article. I'm far from a Python expert, but as an observation, most of these features are actually just typing module features. In particular, I wasn't sold on Generics or Protocols as I would have just used duck typing in both cases... Does modern, production-level python code use types everywhere? Is duck typing frowned upon?
- maleldil 1y ago
  > Does modern, production-level python code use types everywhere? I don't know about code inside companies, but most new open source projects I encounter use typing. Many old ones have been converted too. > Is duck typing frowned upon? No. You can use Protocols to define what shape you expect your duck to be. There's some discussion about whether you should use abstract classes or protocols, though.
- d0mine 1y ago
  Structural typing is a static equivalent of dynamic duck typing. It is compatible with type checkers. Protocols mentioned in the article work without inheritance. A duck doesn't need to know that there is SupportsQuack protocol. If it quacks, it passes the type checks.
djoldman 1y ago
This is wild and something I didn't know about: https://blog.edward-li.com/tech/advanced-python-features/#2-keyword-only-and-positional-only-arguments https://blog.edward-li.com/tech/advanced-python-features/#2-... def bar(a, /, b): ... # == ALLOWED == bar(1, 2) # All positional bar(1, b=2) # Half positional, half keyword # == NOT ALLOWED == bar(a=1, b=2) # Cannot use keyword for positional-only parameter
- zahlman 1y ago
  Reference: https://peps.python.org/pep-0570/ https://peps.python.org/pep-0570/
- lordgrenville 1y ago
  And you can use * for the reverse (every parameter from here on needs to be keyword-only). https://docs.python.org/3.12/reference/compound_stmts.html#function-definitions https://docs.python.org/3.12/reference/compound_stmts.html#f...
  - mckirk 1y ago
    That feature I'm using regularly and can recommend doing so, e.g. when passing boolean flags to functions, to clarify their meaning at the call-site by ensuring they have to be passed by name.
1ncunabula 1y ago
[flagged]
edwardjxli 1y ago
Hey yall! Original author of the blog here! I did not expect to wake up at 4am seeing my post on front page HN, but here we are nevertheless :D As the intro mentioned, these started off as 14 small tweets I wrote a month prior to starting my blog. When I finally got that set up, I just thought, "hey, I just spent the better part of two weeks writing these nifty Python tricks, might as well reuse them as a fun first post!" That's why the flow might seem a little weird (as some pointed out, proxy properties are not really a Python "feature" in of itself). They were just whatever I found cool that day. I tried to find something more esoteric if it was a Friday, and something useful if it was a Monday. I was also kinda improving the entire series as it was going on, so that was also a factor. Same goes with the title. These were just 14 feature I found interesting while writing Python both professionally and as a hobby. Some people mentioned these are not very "advanced" per se, and fair enough. I think I spent a total of 5 second thinking of a title. Oh well!
- _l7dh 1y ago
  Hey, I really enjoyed reading your post. It's straight to the point, the list is very rich and well illustrated with examples. I have been coding in python for the last 20 years and I nowadays rarely like reading these posts as much as I did for yours. Kudos.
- hsbauauvhabzb 1y ago
  I’ve coded python for 15 years and I’ve not seen many of these, so I’d consider them reasonably advanced personally. One I think you missed is getters and setters on attributes!
- jay-barronville 1y ago
  I appreciate you writing the post and I enjoyed reading it. Congratulations for ending up on the front page! (I hope the server hosting your blog is okay!)
- morkalork 1y ago
  The tricks are more advanced than what I've seen in most people's code at some jobs before! The post is refreshing because it sits in the neglected middle ground between the tons of Python-for-beginners content out there and the arcane "I'm a core developer writing about very specific things" content.
smjburton 1y ago
Some of these newer features don't seem like improvements to the language (e.g. the Walrus operator): ''' # ===== Don't write this ===== response = get_user_input() if response: print('You pressed:', response) else: print('You pressed nothing') # ===== Write this instead ===== if response := get_user_input(): print('You pressed:', response) else: print('You pressed nothing') ''' The first implementation is immediately clear, even if you're not familiar with Python syntax. If you don't know what the ":=" operator does, the code becomes less readable, and code clarity is traded away in favor of being slightly more concise.
- edwardjxli 1y ago
  In that example I gave, I totally agree with you. One area where I find walrus operators kinda useful is with dealing with iterators. For example, ''' iterable = iter(thing) while val := next(iterable, None): print(val) ''' is a lot cleaner in my opinion compared to ''' iterable = iter(thing) val = next(iterable, None) while val is not None: print(val) val = next(iterable, None) ''' Reason why I did not use this example outright was because I wasn't sure if people were familiar with the iter api, so I just chose a simpler example for the blog.
  - maleldil 1y ago
    Isn't this equivalent to just `for val in thing`?
    - sgarland 1y ago
      If `thing` already supports iteration, then yes, but sometimes you need to build your own [0]. [0]: https://docs.python.org/3/library/functions.html#iter https://docs.python.org/3/library/functions.html#iter
      - maleldil 1y ago
        Parent's example had a direct `iter(thing)`. Even if you had to use `iter(thing, sentinel)`, you'd still use `for val in iter(thing, sentinel)`, not while.
ratmeadow 1y ago
Good article! I found the typing bits particularly interesting. This part of the language has been evolving rapidly recently, which is cool if you like static typing. Although I think the example of type alises in section 4 is not quite right. NewType creates a new "subtype" which is not equivalent to the original type. That's different to TypeAlias, which simply assigns a name to an existing type. Hence NewType is still useful in Python 3.12+.
- mdaniel 1y ago
  > which is cool if you like static typing. I don't believe any reasonable person would call Python statically typed, it just now has a pathway though which one can send additional documentation, with all of its caveats python3.13 -c ' foo: dict[str, list] = "lol" print(foo.keys()) ' And, yes, I am aware of the chorus of "just sprinkler more linters"
tzury 1y ago
My own opinion is that Python shall remain Python, and golang, Rust and Typescript each should be whichever they are with their unique philosophy and design. I am coding in all 4, with some roughly 28 years now, and I don't like what is becoming of Python There is a reason why python become that popular and widely adapted and used, and it is not the extra layers of type checking, annotations and the likes. this looks familiar to me, but from other languages. response := get_user_input() I am aware of the factI am in minority, and not trying to change anyone's mind, simply what this voice to be heard from time to time. All in all, a very comprehensive list of some of the recent introduced features. There is an older list on SO which readers might also find useful: https://stackoverflow.com/questions/101268/hidden-features-of-python https://stackoverflow.com/questions/101268/hidden-features-o...
- edwardjxli 1y ago
  Extra stuff like type checking and annotations are definitely not the reason why python became that popular and widely adapted and used, but it certainly doesn't hurt to add and use them. To be clear, I'm not expecting people to start adding generics to their quick hacked together Python scripts (in fact please don't do that). Instead, if you're building a library or maintaining a larger Python codebase, a lot of these start becoming very useful. A lot of the typing features I mentioned are already used by Python under the hood, and that a lot of Python developers just take for granted. Case in point, the python-opencv (https://github.com/opencv/opencv-python https://github.com/opencv/opencv-python) library has basically no types and it's an absolute pain to work with. BTW thats a really good SO thread, thanks for linking it!
- abirch 1y ago
  I love Guido's balance. You don't need type checking and annotations if you don't want to use them. They are nice when you're playing in more complex systems, however, completely optional.
- mjr00 1y ago
  > There is a reason why python become that popular and widely adapted and used, and it is not the extra layers of type checking, annotations and the likes. Python got popular and widely adopted for the same reason PHP did: 1) it was readily available everywhere and 2) it's an easy language for beginners to pick up and get real applications working quickly. But it turns out that the language features desirable for a quick prototype done by your fresh-out-of-college founding engineer aren't the same as the language features desirable for your team of 100 engineers trying to work together on the same codebase. Python is at an odd intersection where it's used by everything from large teams building robust backend applications (particularly in the data/ML space), to data scientists hacking on scripts interactively in Jupyter, to ops people writing deployment scripts. You don't need type checking and annotations if you're writing a few one-off scripts, but you'd be crazy to not take advantage of them for a larger application.
curtisszmania 1y ago
[dead]
stitched2gethr 1y ago
> @overload is a decorator from Python’s typing module that lets you define multiple signatures for the same function. I'll be honest, I've never understood this language feature (it exists in several languages). Can someone honestly help me understand? When is a function with many potential signatures more clear than just having separate function names?
- klankeser 1y ago
  I would recommend reading the article example once more. Going from the example, without the overload, the function would return a union type which means any time you use the function, you have to put a type check to the result to know if the output is a list or not. With overload, as soon as an argument to the function is a certain type, the output is determined, so you won't need to type check.
  - pdabbadabba 1y ago
    I think that GP's point is that you could accomplish the same thing by simply having separate functions with different names, signatures, and return types.
- Izkata 1y ago
  In other languages it's an alternate way to do default or optional values in function arguments. Python instead usually uses kwargs for this.
- morkalork 1y ago
  That's what you get when you add a typing system and come full circle to doing OOP like Java.
- ryandv 1y ago
  It's an implementation of "ad-hoc polymorphism", where, for example, it may make sense to "add" (+) together numbers of various types: integers, floating points, rational numbers, etc. Thus the (+) operator for addition is "overridden" or "polymorphic" in the types of numbers that can be added together. The argument for having a polymorphic signature rather than just multiple separate "monomorphic" functions is similar to that for "generics," otherwise known as "parametric polymorphism": why not just have a function `forEachInt` for iterating over lists of ints, a separate function `forEachChar` for iterating over lists of characters, and so on? Higher levels of abstraction and generality, less boilerplate and coupling to any particular choice of data structure or implementation. You could of course go the route of golang which indeed just had you write "monomorphized" versions of everything. Several years later generics were added to the language. Alternatively, you throw everything out and never have to worry about typing or polymorphism, at the cost of static safety.
lucideer 1y ago
As someone coming from Javascript/Typescript & now working full time in Python, this is a lovely - mostly fantastically useful - little resource. Some choice observations: 1. Typing overloads: TS has typed overloads I think largely as an affordance to an unfortunate feature of Javascript. In my experience overloads are an anti-pattern or at best code smell. It's nice that you can type them if you're cleaning up an existing codebase that uses them but I would consider them tech debt. 2. Keyword-only and Positional-only Arguments: This is the opposite of the 1st feature (ability to make method signatures more strict) but man is the syntax cryptically terse. I'd love to use this everywhere but I'd be concerned about readability. 3. Future Annotations: Thank you for this section - forward references have been a real pain for me recently & this is the first explanation that's scratches the service of the "why" (rather than just focusing on case-by-case solutions), which is much more helpful. Bring on PEP 649. 4. Generics: Cries in legacy 3.10 codebase 5. Protocols: As a Typescript guy, this seems very cosy & familiar. And not very Pythonic. I'm not sure how to feel. 14. Metaclasses: > if you are that 1% which has a unique enough problem that only metaclasses can solve, they are a powerful tool that lets you tinker with the internals of the Python object system. OR if you're one of the many devs that believes their problem is unique & special & loves to apply magical overengineered solutions to simple common problems, then the next person who inherits your code is going to really love your metaclasses. They sure make tracing codepaths fun.
- phailhaus 1y ago
  > 5. Protocols: As a Typescript guy, this seems very cosy & familiar. And not very Pythonic. I'm not sure how to feel. Isn't it super pythonic? One of the first things you learn about Python is that "everything is duck typed", but then the type system is primarily nominally typed. It seems like Protocols should have been there from the start, like Typescript interfaces.
  - lucideer 1y ago
    Maybe you're right. I'm new to Python & what is or isn't "pythonic" doesn't seem very intuitive to me (beyond just reading PEPs all day) - I guess I'm speaking from using the current type system & the idea of nominal types coexisting with structural seems a little disjointed.
deleted 1y ago
[deleted]
addoo 1y ago
The only things I’d have probably change about this list is the inclusion of some of the collections.abc containers for type annotations, TypedDict and how it can make working with strictly structured dictionaries not terrible (but if it looks like an class and quacks like a class, make it a class if you can), and Counter (only because I forget it exists every single time I should have used it). Several comments disliking the walrus operator, like many of the features on this list I also hated it… until I found a good use for it. I almost exclusively write strictly typed Python these days (annotations… another feature I originally hated). The walrus operator makes code so much cleaner when you’re dealing with Optionals (or, a Union with None). This comes up a lot with regex patterns: if (match := pattern.search(line)) is not None: print(match.group()) Could you evaluate match on a separate line before the conditional? Sure. But I find this is a little clearer that the intended life of match is within the conditional, making it less tempting to reuse it elsewhere. Not a Python feature specifically, but I’d also love to see code that uses regex patterns to embrace named capturing groups more often. .group(“prefix”) is a lot more readable than .group(1).
nurettin 1y ago
My favorite hack is to use a single threaded ThreadPoolExecutor as my main thread. When another thread wants to communicate with it, it just queues the callback function into main using submit. This cleans up the architecture and avoids the use of Queue which is a bit slower.
hk1337 1y ago
I thought the typing overload was interesting because I thought it was going to be like overloading in C# and Java but it's really just useful for documentation and intellisense tools in IDEs.
librasteve 1y ago
not sure whether to laugh or cry at this ... looks like over the years python has agglomerated some of the features raku has had from day 1 (2015) in sadly inconsistent ways, often via module includes: - multi subs & methods = "typing overloads" - named & positional args - stubs = "future annotations" - subsets = "Generics" - protocols - wrappers = "context managers" - given / when = "structural pattern matching" - my declaration = "walrus op" - short circuit evaluation - operator chaining - fmt - concurrency If you are a Python coder and you feel the need for some of this, I humbly suggest you take a look at https://raku.org https://raku.org
OutOfHere 1y ago
It is a great article, but it completely misses what's new in Python parallelism with the GIL being optional since 3.13, also multiple interpreters via InterpreterPoolExecutor coming in 3.14.

393 points by BerislavLopac 1y ago

TFA's use-case for for/else does not convince me: for server in servers: if server.check_availability(): primary_server = server break else: primary_server = backup_server deploy_application(primary_server) As it is shorter to do this: primary_server = backup_server for server in servers: if server.check_availability(): primary_server = server break deploy_application(primary_server)

macleginn 1y ago
I would say that the main benefit of for/else is readability and ease of writing since nothing needs to be front-loaded. (The downside is an arguably redundant addition to the basic syntax, of course.)
wesselbindt 1y ago
If you replace the assignment in the else clause with something side-effectful it does make sense. But even then it harms the readability of the code to such a ridiculous extent. I've never not regretted adding an else clause to a for loop. Any cognitive benefits from Python's approach to for loops closely mirroring natural language just go out the window.
Loranubi 1y ago
I do like the else clause with for loops. However most people are not familiar with it, and also `else:` as a keyword is confusing. I always remember it as `no break:`.
- stevesimmons 1y ago
  I think of it as "otherwise:"
  - int_19h 1y ago
    That's still confusing though. The problem here is that `else` is semantically attached to `break`, but syntactically attached to the body of the loop. The latter makes it look like it executes if the loop body didn't, if you interpret it in the most straightforward way. IMO a better design would be to have a block that always executes at the end of the loop - there's even a reasonable keyword for it, `finally` - but gets a boolean flag indicating whether there was a break or not: for server in servers: if server.check_availability(): primary_server = server break finally did_break: if not did_break: primary_server = backup_server Or better yet, make `break` take an optional argument (which defaults to `True` if unspecified), and that's what you get in `finally`. So this could be written: for server in servers: if server.check_availability(): break server finally server: primary_server = server if server is not None else backup_server
hk__2 1y ago
What about this? for server in servers: if server.check_availability(): primary_server = server break else: logger.warning("Cannot find a valid server") # <--- primary_server = backup_server deploy_application(primary_server)
- TekMol 1y ago
  Yes, that might be a better example.
zahlman 1y ago
This kind of search can be done a variety of different ways, and is worth abstracting, e.g.: def first(candidates, predicate, default): try: return next(c for c in candidates if predicate(c)) except StopIteration: return default deploy_application(first(servers, Server.check_availability, backup_server))
- sowhat25 1y ago
  Great, first() comes part of more_itertools !
  - zahlman 1y ago
    `more_itertools` is definitely on my shortlist of things I wish were in the Python standard library. (The list of things I'd like to see cut out, is probably considerably longer. But that's mainly on aesthetic principles; it's probably not creating a significant maintenance burden.) And yet somehow I keep forgetting about it in the exact moments when it would probably make my life easier.
- nostoc 1y ago
  Instead of catching the `StopIteration` exception, you can simply provide a default case to `next` : next((c for c in candidates if predicate(c)), default)
  - zahlman 1y ago
    Indeed. I was thinking of ways to generalize for the case where a default isn't desired, then decided against introducing that complexity in the example, then forgot that I could re-simplify further.
paolosimone 1y ago
If the team is familiar with Functional Programming concepts, I'd actually suggest available_servers = (server for server in servers if server.check_availability()) primary_server = next(available_servers, backup_server) deploy_application(primary_server) which IMHO better conveys the intent to the reader (i.e. focus on "what to do" over "how to do it")
HelloNurse 1y ago
The shorter version that saves the else: line assigns primary_server twice: it is OK for an assignment but a potential problem if doing something less trivial, and very ugly style in any case.

macleginn 1y ago

The way the language is evolving, it seems likely to me that people in the applications camp (ML, simple web-dev, etc.) will soon need a "simple Python" fork or at least an agreed-upon subset of the language that doesn't have most of these complications (f-strings are a major success, though).

globular-toast 1y ago
The only newish thing on this list appears to be structural pattern matching. Other than that it's all typing stuff which is all optional.
sgarland 1y ago
Nothing stops you from simply not using features, and even having CI checks that forbid them. Not sure why you’d need a fork.

pjmlp 1y ago

Here is another one, list and expression comprehensions shared with ML languages (not that AI one), apparently many aren't aware of them. The itertools package.

agumonkey 1y ago
and maybe https://more-itertools.readthedocs.io/en/stable/ https://more-itertools.readthedocs.io/en/stable/ never used it but it seems worth a try

Loranubi 1y ago

I would argue that most of these features (basically everything except metaclasses) are not advanced features. These are simple, but for some reason less well known or less used features. Metaclasses are however quite complex (or at least lead to complex behavior) and I mostly avoid them for this reason. And 'Proxy Properties' are not really a feature at all. Just a specific usage of dunder methods.

hsbauauvhabzb 1y ago
Simple doesn’t make something basic or beginner level.

fxj 1y ago

This is all fun and games unless you have to debug someone elses code and they use a new feature that you didnt know about. Speaking for myself, I would be glad if there were a python_light version of the interpreter that has a simple syntax only like the early 3.x versions. just my 2 ct

hk__2 1y ago
> This is all fun and games unless you have to debug someone elses code and they use a new feature that you didnt know about. What’s wrong about learning things by looking at code from more experienced people?
HelloNurse 1y ago
You underestimate the social complexity of a simplified language. Everyone would disagree about the design (what is worth including because it is "simple" or necessary?), everyone would avoid using an handicapped language where features they need are gratuitously missing, and nobody would care about it enough to work on implementation and maintenance.

Starlevel004 1y ago

If context managers are considered advanced I despair at the code you're writing.

lesser-shadow 1y ago
As a UD, I didn't know what a context manager was before reading this article, so you are not really far from the truth, tbh.

wismwasm 1y ago

Nice! I would add assert_never to the pattern matching section for exhaustiveness checks: https://typing.python.org/en/latest/guides/unreachable.html#assert-never-and-exhaustiveness-checking https://typing.python.org/en/latest/guides/unreachable.html#...

globular-toast 1y ago

This is a nice list of "things you might not know" that is worth skimming to add to your toolkit. If you are really interested in "advanced Python", though, I would recommend the book Fluent Python by Ramalho. I have the first edition which is still highly relevant, including the async bits (you just have to translate the coroutines into async syntax). There is a second edition which is more up to date. I would also recommend checking out the functools[0] and itertools[1] modules in the standard library. Just go and read the docs on them top to bottom. It's also worth reading the first few sections of Python Data Model[2] and then bookmarking this page. [0] https://docs.python.org/3/library/functools.html https://docs.python.org/3/library/functools.html [1] https://docs.python.org/3/library/itertools.html https://docs.python.org/3/library/itertools.html [2] https://docs.python.org/3/reference/datamodel.html https://docs.python.org/3/reference/datamodel.html

sgarland 1y ago
+1 to itertools especially. Absurdly powerful, and the recipes at the bottom of the doc page are terrific. The only problem I’ve found is that for interviews, people often aren’t familiar with it, which can lead to you solving whatever puzzle they had in far less time than they intended, and without manually building whatever logic it was they assumed you would need.
- globular-toast 1y ago
  > The only problem I’ve found is that for interviews, people often aren’t familiar with it, which can lead to you solving whatever puzzle they had in far less time than they intended, and without manually building whatever logic it was they assumed you would need. My favourite example of a similar thing happening to me was when I was asked to reverse the digits in a number. I somewhat jokingly asked if I was assuming base 10, which got some awkward looks so I knew something was up. They weren't impressed at all with my answer of `"".join(reversed(str(123456789)))`. I didn't get the job.
  - zahlman 1y ago
    >I didn't get the job. Their loss. Expecting an arithmetic solution to be necessary is incongruous with accepting the overhead of Python. IIRC your approach will actually be faster; the string conversion implicitly number-crunches on the bare metal, whereas any arithmetic approach in Python code won't. Unless they wanted you to convert back to int afterwards. Or wanted you to reverse the string with the slicing trick (which is, indeed, quite a bit faster yet).

mvATM99 1y ago

Good list. Some of these i knew already, but the typing overloading and keyword/positional-only arguments were new to me. One personal favorite of mine is __all__ for use in __init__.py files. It specifies which items are imported whenever uses from x import *. Especially useful when you have other people working on your codebase with the tendency to always import everything, which is rarely a good idea.

hk__2 1y ago
It’s never a good idea. I use `__all__` to explicitely list my exports in libraries, so that when one write `from mylib import `, the IDE auto-completes only the public classes and functions.

red_admiral 1y ago

This sounds fun if you have 10x programmers or at least IQ > 140 programmers in charge. Last place I worked, I was told never use "smart" tricks if you can do the same thing in a simpler way. For-else and f-strings and := sound like harmless enough (though the last one is controversial); "with" is useful for resources that need to be deallocated but "yield"? Really? "more readable" when you slap a decorator on the thing? Why are we packing the enter and exit operations into a single function? Playing with the typesystem and generics like this makes me worry I'm about to have a panic attack. Give me code that I can understand and debug easily, even when I didn't write it; don't do implicit magical control flow changes unless you have a very good excuse, and then document both the how and the why - and you'll get a product that launches earlier and has fewer bugs. Sometimes, a few more if statements here and there make code that is easier to understand, even if there's a clever hack that could cut a line or two here and there.

hk__2 1y ago
I don’t really see "smart tricks" here, just features of the language that you may not be aware of. I use most of them in my daily work without thinking about it: `@overload` and other typing features are useful to document the code, prevent mistakes, and help IDEs generate more helpful suggestions; keyword-only arguments force users to pass options as keywords, so you avoid calls like `foo(x, z, z, true, true, false, true)` and ensure the code is more explicit; for-else can sometimes clarify some complicated code; the walrus operator, short-circuiting with `or`, and chaining operators are just normal Python.
- mpeg 1y ago
  overload is very useful, but I find some people mistakenly think it actually modifies the runtime function signature to do multiple dispatch, so I do try to avoid it in favour of simpler functions whenever possible too many overloads can be a code smell IMHO, it's ok when you are implementing a very common pattern and want proper types (decorators come to mind, so that both @decorator and @decorator() work, where the decorator might also have optional args) but I think in cases like the example in the article it should almost always be two separate functions

davnn 1y ago

Working in the ML field, I can't hate Python. But the type system (pre-3.12, of course) cost me a lot of nerves. Hoping for a better post-3.12 experience once all libraries are usable in 3.12+. After that experience, I’ve come to truly appreciate TypeScript’s type system. Never thought I’d say that.

hk__2 1y ago
Same experience here, Python’s typing experience is awful compared to TypeScript, even post-3.12. Mypy’s type inference is so dumb you have to write arguments like `i: int = 0`; `TypedDict`s seems promisable at first and then end up as a nightmare where you have to `cast` everything. I miss TypeScript’s `unknown` as well.
- drumnerd 1y ago
  0 can be inferred as a float too, so doesn’t it make sense to type numbers?
  - porridgeraisin 1y ago
    I believe mypy infers i as an integer in i = 0. I remember I had to do i = 0.0 to make it accept i += someFloat later on. Or of course i:float = 0 but I preferred the former.
    - hk__2 1y ago
      Yes, but not in arguments: def f(i=0) -> None: j = i + 1 k = 1 reveal_type(i) reveal_type(j) reveal_type(k) Output: Revealed type is "Any" Revealed type is "Any" Revealed type is "builtins.int"
      - jsjohnst 1y ago
        Because it shouldn’t in function arguments. The one defining the function should be responsible enough to know what input they want and actually properly type it. Assuming an int or number type here is wrong (it could be optional int for example).

drumnerd 1y ago

Those are basic, you can decompile a function and change its bytecode, you can completely revamp the parser using codecs, etc

padjo 1y ago
Yeah I only worked professionally with python for about 6 months and I knew about most of them.

lordnacho 1y ago

For me, the biggest benefit to python is that it feels like executable pseudocode. The language gets out of the way of your domain level instructions. This is probably why most non-programmers have the easiest time with python. The more fancy stuff you add to it, the less attractive it becomes. Sure, most of these things have some sort of use, but I reckon most people do not get deep enough into python to understand all these little things.

7bit 1y ago
I disagree. Adding "fancy stuff" by definition does not remove what made it attractive to you in the first place. You even acknowledge this in your second sentence. Something you don't know of is not capable to influence your opinion in any way. And when you know of it -- and dislike it -- just keep doing it the way you did before knowing it.
- pansa2 1y ago
  > And when you know of it -- and dislike it -- just keep doing it the way you did before knowing it. That breaks down as soon as you need to work with anyone else's code that uses the "fancy stuff".
  - deleted 1y ago
    [deleted]
  - 7bit 1y ago
    No, it does not. There's always someone else better than you. That's no fault of the language
rainingmonkey 1y ago
Lua is like this only moreso, and there's little fear of "progress" as large parts of the community have stuck with Lua5.1. Of course, Lua is batteries-not-included so there may be the problem of "progress" in external libraries; in practice things like Penlight barely change though.

hrtk 1y ago

Trying so hard to make it a typed language

maleldil 1y ago
Yes, please. Given I'm forced to use Python, I'd welcome any "compile time" tools I can get. These days, it's using pyright with strict mode, which is pretty good, but there's still a long way to go.

hiichbindermax 1y ago

Nitpick about 9.3 Short Circuit Evaluation: both things evaluate differently if you have empty strings. The if-else clause treats empty strings as valid while the or operator will treat them equivalent with None.

TACD 1y ago
Similarly with 9.2, assignment using a walrus operator will also fail if the value is 0 (or anything falsy: https://docs.python.org/3/library/stdtypes.html#truth-value-testing https://docs.python.org/3/library/stdtypes.html#truth-value-...)
- maleldil 1y ago
  You can use `if (response := get_user_input()) is not None` if that's important. IME, empty strings and None would be treated the same way.

lyu07282 1y ago

Good list, its one of these things you either never heard of, or used for years and think everyone knew that. I'll add a few: * did you know __init__.py is optional nowadays? * you can do relative imports with things like "from ..other import foo" * since 3.13 there is a @deprecated decorator that does what you think it does * the new generics syntax also works on methods/functions: "def method[T](...)" very cool * you can type kwargs with typeddicts and unpack: "def fn(*kwargs: Unpack[MyKwargs])" * dataclasses (and pydantic) support immutable objects with: "class MyModel(BaseModel, frozen=True)" or "@dataclass(frozen=True)" * class attributes on dataclasses, etc. can be defined with "MY_STATIC: ClassVar[int] = 42" this also supports abstract base classes (ABC) * TypeVar supports binding to enforce subtypes: "TypeVar['T', bound=X]", and also a default since 3.13: "TypeVar['T', bound=X, default=int]" * @overload is especially useful for get() methods to express that the return can't be none if the default isn't None * instead of Union[a, b] or Optional[a] you can write "a | b" or "a | None" nowadays * with match you can use assert_never() to ensure exhaustive matching in a "case _:" block * typing has reveal_type() which lets mypy print the type it thinks something is * typing's "Self" allows you to more properly annotate class method return types * the time package has functions for monotonic clocks and others not just time() anyone know more things?

zahlman 1y ago
> did you know __init__.py is optional nowadays? It has an effect, and is usually worth including anyway. I used to omit it by default; now I include it by default. Also, you say "nowadays" but it's been almost 13 years now (https://peps.python.org/pep-0420/ https://peps.python.org/pep-0420/). > since 3.13 there is a @deprecated decorator that does what you think it does Nice find. Probably worth mentioning it comes from the `warnings` standard library. > the time package has functions for monotonic clocks and others not just time() There's quite a bit in there, but I question how many people need it. Anyway, it's always surprising to me how when other people make these lists, such a large fraction is taken up by tricks with type annotations. I was skeptical of the functionality when the `typing` standard library was introduced; I've only grown more and more wary of it, even as people continue to insist to me that it's somehow necessary.
- 1y ago

krelian 1y ago

Python has evolved into quite a complex language considering the amount of features that come built-in. The documentation, while complete, does not facilitate discovery of many of those features.

zahlman 1y ago
The quality of documentation is a known, highlighted concern in the community. There is particular interest in adopting Diataxis principles to improve documentation, both for Python itself and for packaging (see e.g. the user guide at https://packaging.python.org/en/latest/ https://packaging.python.org/en/latest/ , or the PyPA main site https://www.pypa.io/en/latest/ https://www.pypa.io/en/latest/ ). If you want to help, there's a section on the Python forum (https://discuss.python.org/c/documentation/26 https://discuss.python.org/c/documentation/26) and a Discord server, and issues with documentation can also be reported on the main Python GitHub issue tracker (https://github.com/python/cpython/labels/docs https://github.com/python/cpython/labels/docs).
sakesun 1y ago
It's more complex that decade ago, but still a relatively simple language. I can understand the article without much effort, while I scratch my head really hard when read about advance feature of Typescript or Scala.
- pansa2 1y ago
  > still a relatively simple language If only. I suspect very few Python programmers can even fully explain what `a + b` does. If `a` and `b` are instances of classes, many would say it's equivalent to `a.__add__(b)` or `type(a).__add__(a, b)`, but in fact it's much more complex.
  - cl3misch 1y ago
    Care to elaborate? Where is the complexity hidden?
    - 1y ago

shaunpud 1y ago

Another blog with no feed :(

throwwwwwawa 1y ago

[flagged]

robviren 1y ago

Every time I try to use Python I get this mixed feeling of liking how few guard rails there are between me and the logic I am making and this lingering worry that my code looks like fools attempt to use Python. So much exists as convention or loosely followed rules. Whenever I read articles like this I am wowed by the depth of things I didn't know about Python or how much has changed. It makes something like Go feel like a comfort blanket because I have reasonable certainty code I write won't feel deprecated or left behind a year or two later. Excellent article showing off even more I didn't know.

pydry 1y ago
There are plenty of language features which let you re-use or build your own guardrails in python. I do with the concept of truthiness and falsiness would be taken out and shot though. It's been responsible for far too many nasty bugs IME and it only cuts out a few extra characters. Not a great trade off.
- brookst 1y ago
  +1 — and it only saves characters when someone really knows what they’re doing. Most of is end up with `if (not result) or (result == “”)` and worse.
- Starlevel004 1y ago
  Truthiness and falsiness is one of my more used features. ``if not collection`` comes up in nearly every project multiple times.
  - pydry 1y ago
    Yeah, everybody uses it. That's partly how it ends up causing so many bugs - e.g. some user puts 0 in a text box and instead of the program reacting as if you put 0 in it reacts the same way as if no value was supplied. Then everybody loses their minds because the system suddenly started doing something it was never supposed to.
    - 1y ago

steinuil 1y ago

Turns out I managed to use almost all of these during a refactor of a project at work, even metaclasses... (Metaclass usage is justified in my case: we have a sort of language evaluator and using a metaclass lets us define function arguments with their types and validators in a very coincise and obvious way similar to Pydantic.) I think this list should also include descriptors[0]: it's another metaprogramming feature that allows running code when accessing or setting class attributes similar to @property but more powerful. (edit: nvm, I saw that they are covered in the proxy properties section!) I think the type system is quite good actually, even if you end up having to sidestep it when doing this kind of meta-programming. The errors I do get are generally the library's fault (old versions of SQLAlchemy make it impossible to assign types anywhere...) and there's a few gotchas (like mutable collections being invariant, so if you take a list as an argument you may have to type it as `Sequence[]` or you'll get type errors) but it's functional and makes the language usable for me. I stopped using Ruby because upstream would not commit on type checking (yes I know you have a few choices if you want typing, but they're a bit too much overhead for what I usually use Ruby for, which is writing scripts), and I'm glad Python is committing here. [0]: https://docs.python.org/3/howto/descriptor.html https://docs.python.org/3/howto/descriptor.html

William_BB 1y ago

I enjoyed reading the article. I'm far from a Python expert, but as an observation, most of these features are actually just typing module features. In particular, I wasn't sold on Generics or Protocols as I would have just used duck typing in both cases... Does modern, production-level python code use types everywhere? Is duck typing frowned upon?

maleldil 1y ago
> Does modern, production-level python code use types everywhere? I don't know about code inside companies, but most new open source projects I encounter use typing. Many old ones have been converted too. > Is duck typing frowned upon? No. You can use Protocols to define what shape you expect your duck to be. There's some discussion about whether you should use abstract classes or protocols, though.
d0mine 1y ago
Structural typing is a static equivalent of dynamic duck typing. It is compatible with type checkers. Protocols mentioned in the article work without inheritance. A duck doesn't need to know that there is SupportsQuack protocol. If it quacks, it passes the type checks.

djoldman 1y ago

This is wild and something I didn't know about: https://blog.edward-li.com/tech/advanced-python-features/#2-keyword-only-and-positional-only-arguments https://blog.edward-li.com/tech/advanced-python-features/#2-... def bar(a, /, b): ... # == ALLOWED == bar(1, 2) # All positional bar(1, b=2) # Half positional, half keyword # == NOT ALLOWED == bar(a=1, b=2) # Cannot use keyword for positional-only parameter

zahlman 1y ago
Reference: https://peps.python.org/pep-0570/ https://peps.python.org/pep-0570/
lordgrenville 1y ago
And you can use * for the reverse (every parameter from here on needs to be keyword-only). https://docs.python.org/3.12/reference/compound_stmts.html#function-definitions https://docs.python.org/3.12/reference/compound_stmts.html#f...
- mckirk 1y ago
  That feature I'm using regularly and can recommend doing so, e.g. when passing boolean flags to functions, to clarify their meaning at the call-site by ensuring they have to be passed by name.

1ncunabula 1y ago

[flagged]

edwardjxli 1y ago

Hey yall! Original author of the blog here! I did not expect to wake up at 4am seeing my post on front page HN, but here we are nevertheless :D As the intro mentioned, these started off as 14 small tweets I wrote a month prior to starting my blog. When I finally got that set up, I just thought, "hey, I just spent the better part of two weeks writing these nifty Python tricks, might as well reuse them as a fun first post!" That's why the flow might seem a little weird (as some pointed out, proxy properties are not really a Python "feature" in of itself). They were just whatever I found cool that day. I tried to find something more esoteric if it was a Friday, and something useful if it was a Monday. I was also kinda improving the entire series as it was going on, so that was also a factor. Same goes with the title. These were just 14 feature I found interesting while writing Python both professionally and as a hobby. Some people mentioned these are not very "advanced" per se, and fair enough. I think I spent a total of 5 second thinking of a title. Oh well!

_l7dh 1y ago
Hey, I really enjoyed reading your post. It's straight to the point, the list is very rich and well illustrated with examples. I have been coding in python for the last 20 years and I nowadays rarely like reading these posts as much as I did for yours. Kudos.
hsbauauvhabzb 1y ago
I’ve coded python for 15 years and I’ve not seen many of these, so I’d consider them reasonably advanced personally. One I think you missed is getters and setters on attributes!
jay-barronville 1y ago
I appreciate you writing the post and I enjoyed reading it. Congratulations for ending up on the front page! (I hope the server hosting your blog is okay!)
morkalork 1y ago
The tricks are more advanced than what I've seen in most people's code at some jobs before! The post is refreshing because it sits in the neglected middle ground between the tons of Python-for-beginners content out there and the arcane "I'm a core developer writing about very specific things" content.

smjburton 1y ago

Some of these newer features don't seem like improvements to the language (e.g. the Walrus operator): ''' # ===== Don't write this ===== response = get_user_input() if response: print('You pressed:', response) else: print('You pressed nothing') # ===== Write this instead ===== if response := get_user_input(): print('You pressed:', response) else: print('You pressed nothing') ''' The first implementation is immediately clear, even if you're not familiar with Python syntax. If you don't know what the ":=" operator does, the code becomes less readable, and code clarity is traded away in favor of being slightly more concise.

edwardjxli 1y ago
In that example I gave, I totally agree with you. One area where I find walrus operators kinda useful is with dealing with iterators. For example, ''' iterable = iter(thing) while val := next(iterable, None): print(val) ''' is a lot cleaner in my opinion compared to ''' iterable = iter(thing) val = next(iterable, None) while val is not None: print(val) val = next(iterable, None) ''' Reason why I did not use this example outright was because I wasn't sure if people were familiar with the iter api, so I just chose a simpler example for the blog.
- maleldil 1y ago
  Isn't this equivalent to just `for val in thing`?
  - sgarland 1y ago
    If `thing` already supports iteration, then yes, but sometimes you need to build your own [0]. [0]: https://docs.python.org/3/library/functions.html#iter https://docs.python.org/3/library/functions.html#iter
    - maleldil 1y ago
      Parent's example had a direct `iter(thing)`. Even if you had to use `iter(thing, sentinel)`, you'd still use `for val in iter(thing, sentinel)`, not while.

ratmeadow 1y ago

Good article! I found the typing bits particularly interesting. This part of the language has been evolving rapidly recently, which is cool if you like static typing. Although I think the example of type alises in section 4 is not quite right. NewType creates a new "subtype" which is not equivalent to the original type. That's different to TypeAlias, which simply assigns a name to an existing type. Hence NewType is still useful in Python 3.12+.

mdaniel 1y ago
> which is cool if you like static typing. I don't believe any reasonable person would call Python statically typed, it just now has a pathway though which one can send additional documentation, with all of its caveats python3.13 -c ' foo: dict[str, list] = "lol" print(foo.keys()) ' And, yes, I am aware of the chorus of "just sprinkler more linters"

tzury 1y ago

My own opinion is that Python shall remain Python, and golang, Rust and Typescript each should be whichever they are with their unique philosophy and design. I am coding in all 4, with some roughly 28 years now, and I don't like what is becoming of Python There is a reason why python become that popular and widely adapted and used, and it is not the extra layers of type checking, annotations and the likes. this looks familiar to me, but from other languages. response := get_user_input() I am aware of the factI am in minority, and not trying to change anyone's mind, simply what this voice to be heard from time to time. All in all, a very comprehensive list of some of the recent introduced features. There is an older list on SO which readers might also find useful: https://stackoverflow.com/questions/101268/hidden-features-of-python https://stackoverflow.com/questions/101268/hidden-features-o...

edwardjxli 1y ago
Extra stuff like type checking and annotations are definitely not the reason why python became that popular and widely adapted and used, but it certainly doesn't hurt to add and use them. To be clear, I'm not expecting people to start adding generics to their quick hacked together Python scripts (in fact please don't do that). Instead, if you're building a library or maintaining a larger Python codebase, a lot of these start becoming very useful. A lot of the typing features I mentioned are already used by Python under the hood, and that a lot of Python developers just take for granted. Case in point, the python-opencv (https://github.com/opencv/opencv-python https://github.com/opencv/opencv-python) library has basically no types and it's an absolute pain to work with. BTW thats a really good SO thread, thanks for linking it!
abirch 1y ago
I love Guido's balance. You don't need type checking and annotations if you don't want to use them. They are nice when you're playing in more complex systems, however, completely optional.
mjr00 1y ago
> There is a reason why python become that popular and widely adapted and used, and it is not the extra layers of type checking, annotations and the likes. Python got popular and widely adopted for the same reason PHP did: 1) it was readily available everywhere and 2) it's an easy language for beginners to pick up and get real applications working quickly. But it turns out that the language features desirable for a quick prototype done by your fresh-out-of-college founding engineer aren't the same as the language features desirable for your team of 100 engineers trying to work together on the same codebase. Python is at an odd intersection where it's used by everything from large teams building robust backend applications (particularly in the data/ML space), to data scientists hacking on scripts interactively in Jupyter, to ops people writing deployment scripts. You don't need type checking and annotations if you're writing a few one-off scripts, but you'd be crazy to not take advantage of them for a larger application.

curtisszmania 1y ago

[dead]

stitched2gethr 1y ago

> @overload is a decorator from Python’s typing module that lets you define multiple signatures for the same function. I'll be honest, I've never understood this language feature (it exists in several languages). Can someone honestly help me understand? When is a function with many potential signatures more clear than just having separate function names?

klankeser 1y ago
I would recommend reading the article example once more. Going from the example, without the overload, the function would return a union type which means any time you use the function, you have to put a type check to the result to know if the output is a list or not. With overload, as soon as an argument to the function is a certain type, the output is determined, so you won't need to type check.
- pdabbadabba 1y ago
  I think that GP's point is that you could accomplish the same thing by simply having separate functions with different names, signatures, and return types.
Izkata 1y ago
In other languages it's an alternate way to do default or optional values in function arguments. Python instead usually uses kwargs for this.
morkalork 1y ago
That's what you get when you add a typing system and come full circle to doing OOP like Java.
ryandv 1y ago
It's an implementation of "ad-hoc polymorphism", where, for example, it may make sense to "add" (+) together numbers of various types: integers, floating points, rational numbers, etc. Thus the (+) operator for addition is "overridden" or "polymorphic" in the types of numbers that can be added together. The argument for having a polymorphic signature rather than just multiple separate "monomorphic" functions is similar to that for "generics," otherwise known as "parametric polymorphism": why not just have a function `forEachInt` for iterating over lists of ints, a separate function `forEachChar` for iterating over lists of characters, and so on? Higher levels of abstraction and generality, less boilerplate and coupling to any particular choice of data structure or implementation. You could of course go the route of golang which indeed just had you write "monomorphized" versions of everything. Several years later generics were added to the language. Alternatively, you throw everything out and never have to worry about typing or polymorphism, at the cost of static safety.

lucideer 1y ago

As someone coming from Javascript/Typescript & now working full time in Python, this is a lovely - mostly fantastically useful - little resource. Some choice observations: 1. Typing overloads: TS has typed overloads I think largely as an affordance to an unfortunate feature of Javascript. In my experience overloads are an anti-pattern or at best code smell. It's nice that you can type them if you're cleaning up an existing codebase that uses them but I would consider them tech debt. 2. Keyword-only and Positional-only Arguments: This is the opposite of the 1st feature (ability to make method signatures more strict) but man is the syntax cryptically terse. I'd love to use this everywhere but I'd be concerned about readability. 3. Future Annotations: Thank you for this section - forward references have been a real pain for me recently & this is the first explanation that's scratches the service of the "why" (rather than just focusing on case-by-case solutions), which is much more helpful. Bring on PEP 649. 4. Generics: Cries in legacy 3.10 codebase 5. Protocols: As a Typescript guy, this seems very cosy & familiar. And not very Pythonic. I'm not sure how to feel. 14. Metaclasses: > if you are that 1% which has a unique enough problem that only metaclasses can solve, they are a powerful tool that lets you tinker with the internals of the Python object system. OR if you're one of the many devs that believes their problem is unique & special & loves to apply magical overengineered solutions to simple common problems, then the next person who inherits your code is going to really love your metaclasses. They sure make tracing codepaths fun.

phailhaus 1y ago
> 5. Protocols: As a Typescript guy, this seems very cosy & familiar. And not very Pythonic. I'm not sure how to feel. Isn't it super pythonic? One of the first things you learn about Python is that "everything is duck typed", but then the type system is primarily nominally typed. It seems like Protocols should have been there from the start, like Typescript interfaces.
- lucideer 1y ago
  Maybe you're right. I'm new to Python & what is or isn't "pythonic" doesn't seem very intuitive to me (beyond just reading PEPs all day) - I guess I'm speaking from using the current type system & the idea of nominal types coexisting with structural seems a little disjointed.

deleted 1y ago

[deleted]

addoo 1y ago

The only things I’d have probably change about this list is the inclusion of some of the collections.abc containers for type annotations, TypedDict and how it can make working with strictly structured dictionaries not terrible (but if it looks like an class and quacks like a class, make it a class if you can), and Counter (only because I forget it exists every single time I should have used it). Several comments disliking the walrus operator, like many of the features on this list I also hated it… until I found a good use for it. I almost exclusively write strictly typed Python these days (annotations… another feature I originally hated). The walrus operator makes code so much cleaner when you’re dealing with Optionals (or, a Union with None). This comes up a lot with regex patterns: if (match := pattern.search(line)) is not None: print(match.group()) Could you evaluate match on a separate line before the conditional? Sure. But I find this is a little clearer that the intended life of match is within the conditional, making it less tempting to reuse it elsewhere. Not a Python feature specifically, but I’d also love to see code that uses regex patterns to embrace named capturing groups more often. .group(“prefix”) is a lot more readable than .group(1).

nurettin 1y ago

My favorite hack is to use a single threaded ThreadPoolExecutor as my main thread. When another thread wants to communicate with it, it just queues the callback function into main using submit. This cleans up the architecture and avoids the use of Queue which is a bit slower.

hk1337 1y ago

I thought the typing overload was interesting because I thought it was going to be like overloading in C# and Java but it's really just useful for documentation and intellisense tools in IDEs.

librasteve 1y ago

not sure whether to laugh or cry at this ... looks like over the years python has agglomerated some of the features raku has had from day 1 (2015) in sadly inconsistent ways, often via module includes: - multi subs & methods = "typing overloads" - named & positional args - stubs = "future annotations" - subsets = "Generics" - protocols - wrappers = "context managers" - given / when = "structural pattern matching" - my declaration = "walrus op" - short circuit evaluation - operator chaining - fmt - concurrency If you are a Python coder and you feel the need for some of this, I humbly suggest you take a look at https://raku.org https://raku.org

OutOfHere 1y ago

It is a great article, but it completely misses what's new in Python parallelism with the GIL being optional since 3.13, also multiple interpreters via InterpreterPoolExecutor coming in 3.14.