7 ms·

You're saying, "just rewrite and re-release all your dependencies, it's easy!" which was exactly what happened disastrously with python 2 to 3.

wokwokwok 3y ago
You can down vote this til the cows come home but it’s historically what happened. This “that’s not how it went” stuff going down is quite just blatant historical revisionism. Maybe it’s a good idea? Maybe it’s not? …but anyone down voting “this reminds me of the Python 2/3” fiasco has no idea what they’re talking about.
- qbasic_forever 3y ago
  It's good if there's a benefit for everyone, like python 3 fixing the terrible Unicode story. It's not clear non-GIL will even be a net performance improvement for most people--you are effectively moving syncronization from the core runtime to each and every library and program at the edge. Writing safe code at that level doesn't come for free, your basic program will be slower (and likely buggier) if every call into a library is now doing its own little bespoke GIL instead of relying on python's global one like now.
  - wokwokwok 3y ago
    I’m not gonna argue that point; but it seems massively disingenuous to down vote someone who complains “but now I have to rewrite my library because some people might use it in non-GIL mode”. That’s not whining; it’s just an observation that the committee making these decisions gives zero ducks about the impact this will have for anyone other than the handful of vested parties involved in making the decisions. Pypi has what, 500k projects on it? Many abandoned. Whom exactly is going to update those? Or do packages get an automatic “doesn’t work with no-GIL” unless the author explicitly opts to enable it? Or do we live in a future where any package, with any dependency may or may not have undefined behaviour in no-GIL mode? Like, sure… it’s a good change for many people… once all the hard work is done by the community. Does that remind you of anything? Mmm.
    - qbasic_forever 3y ago
      Yep I fully agree. It's going to ultimately mean 99% of people end up running in old GIL mode with deterministic behavior. Companies will get burned and have to have policies that absolutely under no circumstances will the GIL be disabled in their codebase. A very small handful of highly skilled and funded teams, probably at big companies only, will have the time and tenacity to make their code AND all their dependencies work in a multithreaded environment without the GIL.
      - 8n4vidtmkvmk 3y ago
        I work at a big company. We had probably millions of lines of Python. We migrated to c++ instead of to Python 3.
        abdulhaq 3y ago
        I'm really struggling to think of a scenario where that makes sense
        8n4vidtmkvmk 3y ago
        I'm not convinced it's was the right choice either, but I'm not keen on Python, and not sold on Rust. Maybe Zig? I don't know. I personally like Typescript but even I will admit I'm not sure it's the best choice for a large, server-centric company.
    - miraculixx 3y ago
      At the end of the day,this is the very point the SC and core devs are ignoring in their decision. They do recognize the impact on the ecosystem will be huge, they recognize there will be at least 5 years of parallel gil/nogil versions(*), and they say they don't want a 2-3 story all over again. Yet they have decided against their own best advise (to avoid such a situation). I find it utterly confusing. (*) not just one parallel version, there will be 2 for every release following the introduction. In reality organisations will have to maintain 4-6 different baselines of Python releases along with a matching (and likely differing) set of libraries. I don't mean to fearmonger, I just happen to maintain such environements and I know the effort that goes into this first hand.
  - imtringued 3y ago
    > every call into a library is now doing its own little bespoke GIL instead of relying on python's global one like now. Is this some kind of joke? Do we live in clown world now? You do realize that a lock is a trivial primitive in multi threading? The concept of a "little bespoke GIL" is ridiculous. A lock is a lock. Sure, an extension could just put a global lock on every extension call and then you do end up with an extension wide lock but every other extension remains unaffected. There is no intelligence or genius behind putting a global lock in the interpreter that somehow gets ruined by putting the lock in the extension. In fact, GIL is the dumbest decision you could make and everything from that point on can only get better, not worse.
- ctoth 3y ago
  Except they're not at all analogous situations. In python 2 -> 3, if a new version of a library came out Python 3 only, you couldn't use both it, and your old Python 2 code at the same time. Here, you can use the GIL until every one of your dependencies have migrated, even as new versions of your dependencies come out with nogil support until one magical day they all have nogil and you can migrate. But since you have been able to keep up with the library and weren't just arbitrarily cut off at their last Python2/GIL version, it's not a huge breaking change!
  - miraculixx 3y ago
    * removed comment as it was factually false *
    - usrbinbash 3y ago
      > If it was like you say there would be no need for the --nogil cli option. Testing the difference between running it with and without `nogil` without having to install 2 different interpreters. Testing libraries during transitions. Simply giving users a choice. Convenience. Almost no one uses pre-Go.1.11 (GOPATH instead of Modules) any more for project organisation, and the transition is trivially easy. And yet, the toolchain still reacts to `GO111MODULE=off`.
- usrbinbash 3y ago
  It doesn't matter if this "happened historically". Historically, Napoleon lost. That has zero impact on whether I get a promotion. The 2 changes simply have nothing to do with each other. No-GIL python can still run GIL code, it just won't be able to run it in parallel.
- Dylan16807 3y ago
  "it supports both. Done." is not how Python 2/3 went. You have no idea what you're talking about.
paulddraper 3y ago
No, I'm saying that the problem you worry about doesn't exist. The problem -- as you point out -- with 2 -> 3 was that supporting both versions was very difficult. Because Python 2 couldn't run Python 3 code (and vice versa). And thus libraries existed in awkward states for years. But GIL can run no-GIL code. Supporting both is no harder than supporting one of those options (the no-GIL one).

wokwokwok 3y ago

You can down vote this til the cows come home but it’s historically what happened. This “that’s not how it went” stuff going down is quite just blatant historical revisionism. Maybe it’s a good idea? Maybe it’s not? …but anyone down voting “this reminds me of the Python 2/3” fiasco has no idea what they’re talking about.

qbasic_forever 3y ago
It's good if there's a benefit for everyone, like python 3 fixing the terrible Unicode story. It's not clear non-GIL will even be a net performance improvement for most people--you are effectively moving syncronization from the core runtime to each and every library and program at the edge. Writing safe code at that level doesn't come for free, your basic program will be slower (and likely buggier) if every call into a library is now doing its own little bespoke GIL instead of relying on python's global one like now.
- wokwokwok 3y ago
  I’m not gonna argue that point; but it seems massively disingenuous to down vote someone who complains “but now I have to rewrite my library because some people might use it in non-GIL mode”. That’s not whining; it’s just an observation that the committee making these decisions gives zero ducks about the impact this will have for anyone other than the handful of vested parties involved in making the decisions. Pypi has what, 500k projects on it? Many abandoned. Whom exactly is going to update those? Or do packages get an automatic “doesn’t work with no-GIL” unless the author explicitly opts to enable it? Or do we live in a future where any package, with any dependency may or may not have undefined behaviour in no-GIL mode? Like, sure… it’s a good change for many people… once all the hard work is done by the community. Does that remind you of anything? Mmm.
  - qbasic_forever 3y ago
    Yep I fully agree. It's going to ultimately mean 99% of people end up running in old GIL mode with deterministic behavior. Companies will get burned and have to have policies that absolutely under no circumstances will the GIL be disabled in their codebase. A very small handful of highly skilled and funded teams, probably at big companies only, will have the time and tenacity to make their code AND all their dependencies work in a multithreaded environment without the GIL.
    - 8n4vidtmkvmk 3y ago
      I work at a big company. We had probably millions of lines of Python. We migrated to c++ instead of to Python 3.
      - abdulhaq 3y ago
        I'm really struggling to think of a scenario where that makes sense
        8n4vidtmkvmk 3y ago
        I'm not convinced it's was the right choice either, but I'm not keen on Python, and not sold on Rust. Maybe Zig? I don't know. I personally like Typescript but even I will admit I'm not sure it's the best choice for a large, server-centric company.
  - miraculixx 3y ago
    At the end of the day,this is the very point the SC and core devs are ignoring in their decision. They do recognize the impact on the ecosystem will be huge, they recognize there will be at least 5 years of parallel gil/nogil versions(*), and they say they don't want a 2-3 story all over again. Yet they have decided against their own best advise (to avoid such a situation). I find it utterly confusing. (*) not just one parallel version, there will be 2 for every release following the introduction. In reality organisations will have to maintain 4-6 different baselines of Python releases along with a matching (and likely differing) set of libraries. I don't mean to fearmonger, I just happen to maintain such environements and I know the effort that goes into this first hand.
- imtringued 3y ago
  > every call into a library is now doing its own little bespoke GIL instead of relying on python's global one like now. Is this some kind of joke? Do we live in clown world now? You do realize that a lock is a trivial primitive in multi threading? The concept of a "little bespoke GIL" is ridiculous. A lock is a lock. Sure, an extension could just put a global lock on every extension call and then you do end up with an extension wide lock but every other extension remains unaffected. There is no intelligence or genius behind putting a global lock in the interpreter that somehow gets ruined by putting the lock in the extension. In fact, GIL is the dumbest decision you could make and everything from that point on can only get better, not worse.
ctoth 3y ago
Except they're not at all analogous situations. In python 2 -> 3, if a new version of a library came out Python 3 only, you couldn't use both it, and your old Python 2 code at the same time. Here, you can use the GIL until every one of your dependencies have migrated, even as new versions of your dependencies come out with nogil support until one magical day they all have nogil and you can migrate. But since you have been able to keep up with the library and weren't just arbitrarily cut off at their last Python2/GIL version, it's not a huge breaking change!
- miraculixx 3y ago
  * removed comment as it was factually false *
  - usrbinbash 3y ago
    > If it was like you say there would be no need for the --nogil cli option. Testing the difference between running it with and without `nogil` without having to install 2 different interpreters. Testing libraries during transitions. Simply giving users a choice. Convenience. Almost no one uses pre-Go.1.11 (GOPATH instead of Modules) any more for project organisation, and the transition is trivially easy. And yet, the toolchain still reacts to `GO111MODULE=off`.
usrbinbash 3y ago
It doesn't matter if this "happened historically". Historically, Napoleon lost. That has zero impact on whether I get a promotion. The 2 changes simply have nothing to do with each other. No-GIL python can still run GIL code, it just won't be able to run it in parallel.
Dylan16807 3y ago
"it supports both. Done." is not how Python 2/3 went. You have no idea what you're talking about.

paulddraper 3y ago

No, I'm saying that the problem you worry about doesn't exist. The problem -- as you point out -- with 2 -> 3 was that supporting both versions was very difficult. Because Python 2 couldn't run Python 3 code (and vice versa). And thus libraries existed in awkward states for years. But GIL can run no-GIL code. Supporting both is no harder than supporting one of those options (the no-GIL one).