15 ms·
The PEPs of Python 3.9
- jefft255 6y agoCould someone explain to me what kind of new language features the new parser will allow? I'm curious and very incompetent when it comes to understanding what LL(1) grammar would imply for the end-user (the python programmer like me).
- MHordecki 6y agoThe linked LWN article[1] mentions context-sensitive keywords, ie. a way to treat certain words as language keywords only in specific contexts. For example, a new match statement that wouldn't require reserving the `match` word as a language keyword, which would require a breaking change and break all existing code that uses `match` as a variable name. Such a feature requires support from the parser. [1]: https://lwn.net/Articles/816922/ https://lwn.net/Articles/816922/
- saagarjha 6y agoI wonder if Python will go back and address other shortcoming which I assume are tied to the parser, such as the inability to use quotes inside the interpolated segments.
- kalenx 6y agoOne good example (for those who do not want to read the full article) is the async keyword. Introducing it as keyword broke a few libraries which were already using them as kwarg in some functions (e.g. pytorch).
- pixelmonkey 6y agoExtensive explanation from Python creator and PEG parser implementor Guido van Rossum himself can be found in this video: https://youtu.be/QppWTvh7_sI https://youtu.be/QppWTvh7_sI It's also just a fun video on language parsers in general.
- wenc 6y agoI used a PEG parser for a language I designed because I was attracted to the linear time parsing achievable using a packrat parser. However, I also found that parsing is rarely a performance bottleneck so that wasn't a big plus. PEGs are however are easier to reason about, no ambiguities, so that was a good enough reason.
- justinpombrio 6y ago> PEGs are however easier to reason about Yes and no. PEG parsers are definitely easier to implement than any of the LR(k)-and-ilk parsers. So if you're writing or debugging a PEG parser, that will be easier. However, while shift-reduce conflicts are confusing, they are there to give the strong guarantee that the grammar is unambiguous. And the parser generator will tell you this as soon as you've defined the grammar, before you've even used it. PEG grammars instead remove the guarantee, and let you deal with any confusion that arises much later. Here are some methods of reasoning that standard parsers will give you that PEG parers will not: 1. A ::= B | C is exactly the same as A ::= C | B. 2. If A ::= B | C and you have a program containing a fragment that parses as an "A" because it matched "B", then you can replace that fragment with something that matches "C" and it the program will still parse. Neither of these rules hold in PEGs. Here's a practical concern that (2) helps with. Say you have a grammar for html, and a grammar for js. And you want to be able to parse html with embedded JS. So you stick the js grammar into the html grammar at the right places. If you're using a standard (e.g. LR(k)) parser, and you don't get any shift-reduce (or other) conflicts, then the combined grammar works. In contrast, if you're using a PEG grammar, it's possible that you've ordered things wrong and there are valid JS programs that will never parse because they're clobbered by html parsing rules outside of them. Or vice-versa. Also, realistically if you're using a PEG parser you'll want one that handles left recursion, because working without left recursion turns your grammar into a mess. And left recursion in PEGs can have some weird behavior.
- haberman 6y ago> PEGs are however are easier to reason about, no ambiguities, so that was good enough reason. I frequently hear this mentioned ("PEGs don't have ambiguity"). It is literally true, but I don't think it's true in the sense that actually matters. I've blogged about this in the past (https://blog.reverberate.org/2013/09/ll-and-lr-in-context-why-parsing-tools.html#dealing-with-ambiguity-in-cfgs https://blog.reverberate.org/2013/09/ll-and-lr-in-context-wh...), but I'm not the only person saying this: > PEG is not unambiguous in any helpful sense of that word. BNF allows you to specify ambiguous grammars, and that feature is tied to its power and flexibility and often useful in itself. PEG will only deliver one of those parses. But without an easy way of knowing which parse, the underlying ambiguity is not addressed -- it is just ignored. https://jeffreykegler.github.io/Ocean-of-Awareness-blog/individual/2015/03/peg.html https://jeffreykegler.github.io/Ocean-of-Awareness-blog/indi...
- 1wd 6y agohttps://pyfound.blogspot.com/2020/04/replacing-cpythons-parser-python.html https://pyfound.blogspot.com/2020/04/replacing-cpythons-pars... gives "Parenthesized with-statements" as an example.
- slightwinder 6y ago> Eventually, removeprefix() and removesuffix() seemed to gain the upper hand, which is what Sweeney eventually switched to. Great naming...missed their chance to make the functionality of strip/lstrip/rstrip more clear by name the new methods stripword/lstripword/rstripword. Which would also had the benefit of consistence.
- masklinn 6y agostripwords could (/would) imply that it acts on words. As in, whitespace separated things.
- traes 6y agoIs the next release still planned to be called Python 4? I seem to recall GvR saying that at one point, though I could be mistaken.
- BiteCode_dev 6y ago3.10. https://www.python.org/dev/peps/pep-0619/ https://www.python.org/dev/peps/pep-0619/
- twic 6y agoAfter that will be Python 3.11, and after that, Python for Workgroups 3.11.
- doersino 6y agoPython 95 is gonna blow everyone's minds.
- BiteCode_dev 6y agoWith semver, it's going to be too long to reach. I suggest we switch to the Firefox versioning scheme, it seems to already be near this goal.
- _jal 6y agoYeah, but PythoNT will be around for a long time.
- mixmastamyk 6y agoStart me up!
- pansa2 6y agoYou make a grown man cry...
- agumonkey 6y ago
- tartrate 6y agore.sub?
- OJFord 6y ago> Eric Fahlgren amusingly summed up the name fight this way: > > I think name choice is easier if you write the documentation first: > > cutprefix - Removes the specified prefix. > > trimprefix - Removes the specified prefix. > > stripprefix - Removes the specified prefix. > > removeprefix - Removes the specified prefix. Duh. :) I actually don't agree that it's so obvious, since it returns the prefix-removed string rather than modifying in-place. I think Fahlgren's argument would work better for `withoutprefix`.
- BiteCode_dev 6y agoStrings are immutable in Python, and all strings operations return new strings, including all string methods. So there is no possible confusion.
- ianhorn 6y agoIf I have a string x = "this is a very long string..." and do y = x[:10], then it's a whole new string? If x is near my memory limits, and I do y = x[:-1] will it basically double my memory usage? Is that what you meant by every string is a new string?
- deleted 6y ago[deleted]
- sadfklsjlkjwt 6y agoYes.
- ORioN63 6y agoYes. Although you can use `islice` from itertools to get around this problem, when a problem.
- BiteCode_dev 6y agoIf x is near your memory limits, and you do y = x[:-1], you will get a MemoryError :) For those situations, bytes() + memoryview() or bytearray() can be used, but then you are on your own.
- theandrewbailey 6y agoNo PEP 554 (subinterpreters). That's been moved to 3.10: https://www.python.org/dev/peps/pep-0554/ https://www.python.org/dev/peps/pep-0554/
- BiteCode_dev 6y agoGiven how heated was the debate about those, it's good we didn't try to go too fast with it. I'm full of hopes for this feature, but it's going to be a slow hard work, and we'll only rip the benefit on the long run. So no use to rush it. I feel like we rushed asyncio and type hints, and it took years to make them usable after they were introduced.
- nemetroid 6y ago> Another kind of clean up comes in PEP 585 ("Type Hinting Generics In Standard Collections"). It will allow the removal of a parallel set of type aliases maintained in the typing module in order to support generic types. For example, the typing.List type will no longer be needed to support annotations like "dict[str, list[int]]" (i.e.. a dictionary with string keys and values that are lists of integers). I think this will go a long way toward making type annotations feel less like a tacked-on feature.
- mixmastamyk 6y agoThere have been so many improvements in the typing area, I wish they'd waited a while to get it right.
- black_puppydog 6y agoI think getting them out there has already helped the ecosystem, both in terms of using the types to make working in python better, and in terms of figuring out what the typing system should really look like. This is the next iteration, and I think it's going in the right direction, so I don't really want to criticize the devs for it.
- diarrhea 6y agoLooking "back" now, it never occurred to me that importing List when there is list is particularly strange. Now it sticks out sorely. Very glad this change is happening.
- throwlaplace 6y agoThat's because we're conditioned to think of constructors as functions rather than as types. I think that's not that odd honestly but I do see how counterintuitive it is for people that don't work much in typed languages. I'm not a Haskellite but there you can clearly see the distinction when defining/instantiating sum types (where the type and data constructor live in different namespaces).
- acbart 6y ago
- tln 6y agoSometimes I wish that python strings weren't directly iterable... this article sums it up better than I ever could https://www.xanthir.com/b4wJ1 https://www.xanthir.com/b4wJ1 ...then str.strip and variants could be cleanly and logically extended to allow this functionality, because passing a string and a sequence of strings would be distinguishable. Alas, clean and logical function design can be hard to do late in a languages life. PEP 593 and PEP 585 are clean and logical... glad to see that :)
- joshuamorton 6y agoOn non-iterable strings: The recursive type problem can be solved, with something like what is proposed in [0]. (I have an implementation of a fix on github linked from that thread, there's edge case fixes and PEP's scare me, but technically it's feasible). [0]: https://mail.python.org/archives/list/typing-sig@python.org/thread/OLCQHSNCLM2TOHRQWB3WPSI66AA5A3G7/ https://mail.python.org/archives/list/typing-sig@python.org/...
- spott 6y agoWhile that may solve the recursive type problem, it doesn't really solve the "iterating over strings is rarely what you actually want to do" problem.
- BiteCode_dev 6y agoYou can easily distinguish them: if isinstance(msg, str) So I don't think that's a good argument for not accepting iterables of strings in str methods. Things like replace() would benefit a lot and it's not that hard to do, you can even accept regexes optionally: https://wonderful-wrappers.readthedocs.io/en/latest/string_wrapper.html https://wonderful-wrappers.readthedocs.io/en/latest/string_w... I agree that iterating on string is not proper design however. It's not very useful in practice, and the O(1) access has other performance consequences for more important things. Swift did it right IMO, but it's a much younger language. I also wish we stole the file api concepts from swift, and that open() would return a file like object that always gives you bytes. No "b" mode. If you want text, you to open().as_text(), and get a decoding wrapper. The idea that there are text files and binary files has been toxic for a whole generation of coders.
- gorgoiler 6y agoI was quite surprised — only a few days ago in fact — to discover the standard Python library has no support for Olson (as in tzdata) timezones. Time arithmetic is impossible without them. The ipaddress library also has no support for calculating subnets. It is quite hard to go from 2a00:aaaa:bbbb::/48 to 2a00:aaaa:bbbb:cccc::/64. It would be less weird if the essence of the documentation didn’t make it sound like the library was otherwise very thorough in the coverage of its implementation. Can anyone write a PEP? Maybe I should get off my behind and actually submit a patch for proper IP calculations? Or maybe I missed it in the documentation (which, aside, I wish wasn’t written with such GNU-info style formality.)
- chc 6y agoUnless I misunderstand what you're looking for, I think that functionality is in there. original_net_48 = ip_network("2a00:aaaa:bbbb::/48") desired_subnet = ip_network('2a00:aaaa:bbbb:cccc::/64') subnets_64 = original_net_48.subnets(16) print(f"{desired_subnet} is one of the computed subnets: {desired_subnet in subnets_64}") #=> 2a00:aaaa:bbbb:cccc::/64 is one of the computed subnets: True
- gorgoiler 6y agoThanks, but your second line kind of has the answer in it already. It’s more like... site = ip_network(‘2a00:aaaa:bbbb::/48’) subnet = f(site, 64, 0xcccc) ...and I don’t think f() is in the standard library. But maybe I just index the calculated subnets, from your example? I’ll give it a go! Edit: yes! But it’s a bit slow... https://repl.it/repls/PoshPapayawhipNaturaldocs#main.py https://repl.it/repls/PoshPapayawhipNaturaldocs#main.py
- xxpor 6y agoWell, you can speed it up slightly by using the iterator directly instead of forcing the whole generator into a list. But on the other hand, you can make it WAY faster by doing it with bit manipulation instead like you're writing C: https://repl.it/repls/ClearcutElaborateEmbeds https://repl.it/repls/ClearcutElaborateEmbeds
- heavyset_go 6y agoAm I the only one who wants multi-lined anonymous functions in Python? I find myself really wanting to reach for arrow functions sometimes while writing Python, and end up disappointed that they aren't available.
- fabatka 6y agoI think the basic idea is "if you need multiple lines,you should declare a proper fuction", so I wouldn't stand on one foot until multiline anonymous functions in python.
- trashburger 6y agoI believe Guido was against it as he is mostly opposed to the functional style, as a matter of fact he was opposed to lambas but begrudgingly added then after many requests.
- pansa2 6y agoGuido’s original plan for Python 3 included removing lambda (along with reduce, filter and map) from the language. https://www.artima.com/weblogs/viewpost.jsp?thread=98196 https://www.artima.com/weblogs/viewpost.jsp?thread=98196
- Too 6y ago"Guido was against multi line lambdas" is always brought up in these discussions and then someone digs up an old email from 10 years ago where it happens to be mentioned in passing by. Given the recent success of anonymous functions and it's widespread use and impact in other languages, like Javascript, C#, Java, C++11, maybe it's time to re-evaluate that opinion. I mean, what would javascript be today without promises and multi-line anonymous functions?
- pansa2 6y agoNow that Guido is no longer BDFL nor on the Steering Committee, it doesn’t really matter what he thinks. But, it’s still unlikely that we’ll see full anonymous functions in Python. They just don’t work with its significant-indentation syntax.
- trashburger 6y agoI welcome the terser type hints for generics. I was wishing for something terser, like: {str: [int]} being equivalent to what is currently dict[str, list[int]] in the PEP, but I guess it will have to do.
- st1ck 6y ago`{key: val}` does look nice indeed, but then it takes more effort to replace e.g. `dict` with `Mapping`. Everything would look much nicer with Haskell-like syntax: `dict key val` or `list val`. Or maybe even prefer `{} key val` and `[] val` (no that doesn't look good, I agree).
- emidln 6y agoThis was glorious with Clojure's Schema. I miss it every day I work with inferior languages and data specification systems.