7 ms·

No-GIL mode is optional, and libraries will be marked "no-GIL compatible" and the ecosystem will gradually support more and more of these. No one's flipping a

No-GIL mode is optional, and libraries will be marked "no-GIL compatible" and the ecosystem will gradually support more and more of these. No one's flipping a switch and breaking mountains of sketchy C.

Steltek 3y ago
Well someone has to say it so I might as well: Is this going to be another Python 2 -> 3 cat herding exercise again? I suppose the need to run your app in "no-GIL" mode is less than needing to jump from 2.7 to 3.
- n2d4 3y ago
  I didn't dive deep on this, but I assume that GIL mode can still run anything, including no-GIL code (it is one of their promises, at least). So, unlike 2->3, there is forwards-compatibility. It also seems like the latter isn't meant as a replacement (for the moment), but rather as an option.
  - qbasic_forever 3y ago
    Presumably C extensions will have a different API name/ABI to prevent accidentally calling into GIL code when in non-GIL mode and vice/versa so that's going to complicate the compatibility story.
    - ggm 3y ago
      stat = rt_oldFunctionName(&newStateArg, oldarg[,...]) mostly (again from memory. I'd not live or die by this)
- paulddraper 3y ago
  As sibling comments says, GIL mode can run no-GIL code. Entirely different from 2->3. That analogy is so poorly applicable it's laughable.
  - nequo 3y ago
    Without the last sentence, your comment would be better.
    - paulddraper 3y ago
      It's not nice but it is true. "Dur dur this <insert anything here> is just like 2 -> 3." HN had always been susceptible to drive-by ignorant comments, but it's reaching new levels. There's literally nothing of substance to the suggestion. (And if you think there is, present an actual informed argument.)
      - insanitybit 3y ago
        It wasn't a suggestion, they were asking a question.
        Dylan16807 3y ago
        Oh please. Even if we ignore that any question can have implications, they directly said "someone has to say it". They were explicitly suggesting something by asking that question.
        majewsky 3y ago
        They were suggesting that this is a question that is going to be on people's minds, which is very different from suggesting that the answer is yes.
        deleted 3y ago
        [deleted]
        Dylan16807 3y ago
        All I can say is that the phrasing looks quite biased to me and it would have been easy to not phrase it that way. Subtext is hard to prove, but that doesn't make it not real.
- pyuser583 3y ago
  From the article: > We want to be very careful with backward compatibility. We do not want another Python 3 situation …
miraculixx 3y ago
I hope you are right. I fear it will turn out rather painful (I have lived through 10 years of Java multithreading, I don't miss it) * update: I have also lived through 10 years of Python 2-3. It was not pretty.
qbasic_forever 3y ago
That's the exact attitude that lead to decades of pain with the 2 to 3 transition and libraries though. There was zero plan for how to help libraries migrate from python 2 to 3, or more importantly how does one library support _both_ python 2 and 3 from one codebase as their users take time to switch their python interpretor. The attitude of "just turn off GIL if you don't need it, just use libraries that support turning it off" means library authors will be asked to provide versions of their library that do and don't support non-GIL mode. That's a big burden to dump on library maintainers.
- miraculixx 3y ago
  Indeed. The burden is 2x at least. They already estimate it's going to take 5+ years, so that's 10x the cost of a current library. At least. If the burden becomes too big, at some point people may find it is the easier option to switch to a different ecosystem. I hope not.
  - JonChesterfield 3y ago
    I hope it'll be lua. I fear it'll be javascript.
    - anyoneamous 3y ago
      I hope it'll be Nim, I fear it'll be Julia.
    - concordDance 3y ago
      Lua? Really? That's by far the most unpleasant language I've coded in.
      - JonChesterfield 3y ago
        Lexical scope, first class functions, native coroutines, compiler available at runtime. Trivially extensible. It's semantically really close to lisp. Yep, lua's one of my favourites. The front end syntax isn't what I'd like but whatever, I can still see the AST through it with a little effort.
raverbashing 3y ago
> marked "no-GIL compatible" Ugh, Nomenclature is hard Because non-GIL compatible doesn't mean GIL incompatible
- _ZeD_ 3y ago
  "requires GIL" for all current projects?
mikepurvis 3y ago
Coming from a place of total ignorance, it would be nice if you could do this more incrementally, like have it be in no-gil mode by default, but then have a context manager you can use for gil sections, and have the interpreter bomb out if you try to enter gil-required code while still in no-gil mode.
- HoppyHaus 3y ago
  The issue is when someone's unattended-upgrades bumps up the version and causes something to come crashing down. The people who need to use nogil should know that they need to, and will now have the ability to enable it
  - samus 3y ago
    Almost more importantly, it is also possible to force the GIL to remain on with `PYTHONGIL=1`
  - pritambaral 3y ago
    > ... unattended-upgrades bumps up the version Surely, unattended-upgrades should NOT be bumping major versions?
collinmanderson 3y ago
> No-GIL mode is optional Long term the plan is to 100% remove the GIL from python. From the article: > Long-term, we want no-GIL to be the default, and to remove any vestiges of the GIL (without unnecessarily breaking backward compatibility). We don’t want to wait too long with this, because having two common build modes may be a heavy burden on the community (as, for example, it can double test resources and debugging scenarios), but we can’t rush it either. We think it may take as much as five years to get to this stage. Under base assumptions it also says: > We want to be very careful with backward compatibility. We do not want another Python 3 situation, so any changes in third-party code needed to accommodate no-GIL builds should just work in with-GIL builds (although backward compatibility with older Python versions will still need to be addressed).

Steltek 3y ago

Well someone has to say it so I might as well: Is this going to be another Python 2 -> 3 cat herding exercise again? I suppose the need to run your app in "no-GIL" mode is less than needing to jump from 2.7 to 3.

n2d4 3y ago
I didn't dive deep on this, but I assume that GIL mode can still run anything, including no-GIL code (it is one of their promises, at least). So, unlike 2->3, there is forwards-compatibility. It also seems like the latter isn't meant as a replacement (for the moment), but rather as an option.
- qbasic_forever 3y ago
  Presumably C extensions will have a different API name/ABI to prevent accidentally calling into GIL code when in non-GIL mode and vice/versa so that's going to complicate the compatibility story.
  - ggm 3y ago
    stat = rt_oldFunctionName(&newStateArg, oldarg[,...]) mostly (again from memory. I'd not live or die by this)
paulddraper 3y ago
As sibling comments says, GIL mode can run no-GIL code. Entirely different from 2->3. That analogy is so poorly applicable it's laughable.
- nequo 3y ago
  Without the last sentence, your comment would be better.
  - paulddraper 3y ago
    It's not nice but it is true. "Dur dur this <insert anything here> is just like 2 -> 3." HN had always been susceptible to drive-by ignorant comments, but it's reaching new levels. There's literally nothing of substance to the suggestion. (And if you think there is, present an actual informed argument.)
    - insanitybit 3y ago
      It wasn't a suggestion, they were asking a question.
      - Dylan16807 3y ago
        Oh please. Even if we ignore that any question can have implications, they directly said "someone has to say it". They were explicitly suggesting something by asking that question.
        majewsky 3y ago
        They were suggesting that this is a question that is going to be on people's minds, which is very different from suggesting that the answer is yes.
        deleted 3y ago
        [deleted]
        Dylan16807 3y ago
        All I can say is that the phrasing looks quite biased to me and it would have been easy to not phrase it that way. Subtext is hard to prove, but that doesn't make it not real.
pyuser583 3y ago
From the article: > We want to be very careful with backward compatibility. We do not want another Python 3 situation …

miraculixx 3y ago

I hope you are right. I fear it will turn out rather painful (I have lived through 10 years of Java multithreading, I don't miss it) * update: I have also lived through 10 years of Python 2-3. It was not pretty.

qbasic_forever 3y ago

That's the exact attitude that lead to decades of pain with the 2 to 3 transition and libraries though. There was zero plan for how to help libraries migrate from python 2 to 3, or more importantly how does one library support _both_ python 2 and 3 from one codebase as their users take time to switch their python interpretor. The attitude of "just turn off GIL if you don't need it, just use libraries that support turning it off" means library authors will be asked to provide versions of their library that do and don't support non-GIL mode. That's a big burden to dump on library maintainers.

miraculixx 3y ago
Indeed. The burden is 2x at least. They already estimate it's going to take 5+ years, so that's 10x the cost of a current library. At least. If the burden becomes too big, at some point people may find it is the easier option to switch to a different ecosystem. I hope not.
- JonChesterfield 3y ago
  I hope it'll be lua. I fear it'll be javascript.
  - anyoneamous 3y ago
    I hope it'll be Nim, I fear it'll be Julia.
  - concordDance 3y ago
    Lua? Really? That's by far the most unpleasant language I've coded in.
    - JonChesterfield 3y ago
      Lexical scope, first class functions, native coroutines, compiler available at runtime. Trivially extensible. It's semantically really close to lisp. Yep, lua's one of my favourites. The front end syntax isn't what I'd like but whatever, I can still see the AST through it with a little effort.

raverbashing 3y ago

> marked "no-GIL compatible" Ugh, Nomenclature is hard Because non-GIL compatible doesn't mean GIL incompatible

_ZeD_ 3y ago
"requires GIL" for all current projects?

mikepurvis 3y ago

Coming from a place of total ignorance, it would be nice if you could do this more incrementally, like have it be in no-gil mode by default, but then have a context manager you can use for gil sections, and have the interpreter bomb out if you try to enter gil-required code while still in no-gil mode.

HoppyHaus 3y ago
The issue is when someone's unattended-upgrades bumps up the version and causes something to come crashing down. The people who need to use nogil should know that they need to, and will now have the ability to enable it
- samus 3y ago
  Almost more importantly, it is also possible to force the GIL to remain on with `PYTHONGIL=1`
- pritambaral 3y ago
  > ... unattended-upgrades bumps up the version Surely, unattended-upgrades should NOT be bumping major versions?

collinmanderson 3y ago

> No-GIL mode is optional Long term the plan is to 100% remove the GIL from python. From the article: > Long-term, we want no-GIL to be the default, and to remove any vestiges of the GIL (without unnecessarily breaking backward compatibility). We don’t want to wait too long with this, because having two common build modes may be a heavy burden on the community (as, for example, it can double test resources and debugging scenarios), but we can’t rush it either. We think it may take as much as five years to get to this stage. Under base assumptions it also says: > We want to be very careful with backward compatibility. We do not want another Python 3 situation, so any changes in third-party code needed to accommodate no-GIL builds should just work in with-GIL builds (although backward compatibility with older Python versions will still need to be addressed).