6 ms·

> I never said that multiprocessing is "better" than multithreading. Maybe not, but you seem to be implying that multiprocessing is a sufficient for the most c

> I never said that multiprocessing is "better" than multithreading. Maybe not, but you seem to be implying that multiprocessing is a sufficient for the most case (or the average case ?) however ill define that average case is. > This conversation would've been much easier to have given specific constraints and examples (which nobody seems to give). Not quite, there is no need grand example here, processes vs threads is just a function of how chatty the interaction between the independent agent are. We should have both mechanism available and let the user choose. > My main argument, if you distill it down to the abstract, is "use the right tool for the job" - and stop pretending that a hammer and a knife are the same thing, when they're not. You don't get to decide in the abstract what the right tool is for other people and context we don't know anything about. The people pushing for proper threading are telling you that for them python + better threading is the right tool. > If you need hardcore, ultra efficient and parallelized workflows - Python is just the wrong tool for the job period. This isn't up for debate - it's a given fact. Maybe; But that's not the point here. Proper threading in python doesn't make python more efficient, but it makes the * scaling * of python performance more efficient. Those are two differents things. > Python fills a niche of its own incredibly well, something which others tools suck at, and Python is loved by millions for it. > This whole discussion now (GIL vs. no GIL) is about bending and twisting Python to fit the use cases of few companies like DeepMind or Facebook > who I'm certain represent a miniscule usage compared to the millions of students, schools and universities, research institutes, web shops, hobbyists, tinkerers, etc. Those people want to get shit done quickly - and mutexes, sempahores, events, threads, synchronization primitives, atomics, etc - will do nothing but make their lives a misery, and drive them away. This seems to me like the core of the problem, somehow you say that there is no valid need for threading, but at the same time the people needing threading are not important enough for python to care about... You have the magical ability to be plug into the python hivemind and decide which use case are valid, what are the true value of the python community and what python should and should not consider important... And after all that mental gymnastic you expect us to jump through hoops to convince you otherwise... That's a lot of non technical assumption for a technical problem. > Those people want to get shit done quickly - and mutexes, sempahores, events, threads, synchronization primitives, atomics, etc - will do nothing but make their lives a misery, and drive them away. Not really. If anything, they will benefits from better libraries and more efficient C-extensions. > absolutely needs multithreading Nothing is absolutely needed. Even something are core as a type system is not absolutely needed. The fact that you ask for this level of qualification for a feature that pretty much every other language has is the problem. The problem is not that deep, implementing multi-threading is not that complicated. I do agree that we need to be careful to provide a better API than raw-thread to end users (which no-gil is COMPLETLY agnostic about).

csmpltn 3y ago
> "there is no need [for a] grand example here" > "you don't get to decide in the abstract what the right tool is for other people and context we don't know anything about" So which one is it? Do I get to ask for concrete examples for why this work is worth all the trouble, or should we just pretend like it doesn't matter because you say so? "I want threads, threads fast, others have threads" is not good enough. > "somehow you say that there is no valid need for threading" I've never said that. All I've been asking for are concrete examples that would justify removing the GIL and the impact it would have on the ecosystem. You won't provide any, despite me asking over and over again. > "You have the magical ability to be plug into the python hivemind and decide which use case are valid" CPython has had a GIL for the past 30 years - and things worked out just fine. There are other Python VMs that don't have a GIL - yet CPython remains the most widely used and popular VM. So how about you stop putting words in my mouth, and start actually justifying the asks beyond hand-waving my arguments away? > "The fact that you ask for this level of qualification for a feature that pretty much every other language has is the problem." Again, you're twisting and putting words in my mouth. There's a clear difference between building an ecosystem with multithreading in mind from day one - and suddenly introducing it out of nowhere, 30 years into it being used by millions of workloads globally. This is why I'm asking for "qualifications" (which you won't provide), and this is why I also deem your assertion that "the problem is not that deep" to be a proof beyond any that we should end this conversation now before things get too embarrassing for you :)
- soulbadguy 3y ago
  > So which one is it? Do I get to ask for concrete examples for why this work is worth all the trouble, or should we just pretend like it doesn't matter because you say so? Both ? 1 - There is no need for a grand example to justify the need for threading in a modern language, as we have 20/30 years of background on that topic. To repeat myself, it's all about the amount of communication between compute agent... The more communication you have , the more the process isolation/serialization cost become a problem. 2 - You yourself understand that there is are valid use case for threading, somehow those valid cases are not valid python uses. That's where the disconnect is. You don't get to say by decree what is and is not a "valid" use of python. > "I want threads, threads fast, others have threads" is not good enough. Neither is i don't use thread , you shouldn't use thread. > CPython has had a GIL for the past 30 years - and things worked out just fine. That's what the no-gil people are trying to tell you, no it's not fine and it was never fine. Effort and conversation about the replacement of the GIL are at least 10/15 years old. > yet CPython remains the most widely used and popular VM Faulty logic , the correlation doesn't imply any causal relationship. > and start actually justifying the asks beyond hand-waving my arguments away? Because you aregument are not really argument, they more like strong opinion on things that are closer to esthetics and right/wrong usage of things. Happy to disagree on those one. > There's a clear difference between building an ecosystem with multithreading in mind from day one - and suddenly introducing it out of nowhere, 30 years into it being used by millions of workloads globally. 1 - no-gil isn't out of nowhere. Conversation about this are more that 10 years old. Combined with even longer conversation in other VM/programming language. 2 - no-gil doesn't introduce threading in random workload. It allow people who want threading to use threading. > This is why I'm asking for "qualifications" It's your prerogative to ask for qualifications. But it's also our to decide to judge your ask and decide if they are worth our time. Much in the same way that if feel like we don't need (in 2023) a grand example to justify why we need to add a type system, we don't need a deep conversation about threading. We all have the same information, understand the trade-off. We just have different value system and want different things out of python. And that's okay... > the problem is not that deep" to be a proof beyond any that we should end this conversation now before things get too embarrassing for you :) I think i have some comment somewhere explaining why no-gil was never a technically challenging problem. But the prof is simple... Sam Gross is definitely an exceptional dev, but the fact that a lone programmer come out with an acceptable solution is proof that the problem wasnt that deep.

csmpltn 3y ago

> "there is no need [for a] grand example here" > "you don't get to decide in the abstract what the right tool is for other people and context we don't know anything about" So which one is it? Do I get to ask for concrete examples for why this work is worth all the trouble, or should we just pretend like it doesn't matter because you say so? "I want threads, threads fast, others have threads" is not good enough. > "somehow you say that there is no valid need for threading" I've never said that. All I've been asking for are concrete examples that would justify removing the GIL and the impact it would have on the ecosystem. You won't provide any, despite me asking over and over again. > "You have the magical ability to be plug into the python hivemind and decide which use case are valid" CPython has had a GIL for the past 30 years - and things worked out just fine. There are other Python VMs that don't have a GIL - yet CPython remains the most widely used and popular VM. So how about you stop putting words in my mouth, and start actually justifying the asks beyond hand-waving my arguments away? > "The fact that you ask for this level of qualification for a feature that pretty much every other language has is the problem." Again, you're twisting and putting words in my mouth. There's a clear difference between building an ecosystem with multithreading in mind from day one - and suddenly introducing it out of nowhere, 30 years into it being used by millions of workloads globally. This is why I'm asking for "qualifications" (which you won't provide), and this is why I also deem your assertion that "the problem is not that deep" to be a proof beyond any that we should end this conversation now before things get too embarrassing for you :)

soulbadguy 3y ago
> So which one is it? Do I get to ask for concrete examples for why this work is worth all the trouble, or should we just pretend like it doesn't matter because you say so? Both ? 1 - There is no need for a grand example to justify the need for threading in a modern language, as we have 20/30 years of background on that topic. To repeat myself, it's all about the amount of communication between compute agent... The more communication you have , the more the process isolation/serialization cost become a problem. 2 - You yourself understand that there is are valid use case for threading, somehow those valid cases are not valid python uses. That's where the disconnect is. You don't get to say by decree what is and is not a "valid" use of python. > "I want threads, threads fast, others have threads" is not good enough. Neither is i don't use thread , you shouldn't use thread. > CPython has had a GIL for the past 30 years - and things worked out just fine. That's what the no-gil people are trying to tell you, no it's not fine and it was never fine. Effort and conversation about the replacement of the GIL are at least 10/15 years old. > yet CPython remains the most widely used and popular VM Faulty logic , the correlation doesn't imply any causal relationship. > and start actually justifying the asks beyond hand-waving my arguments away? Because you aregument are not really argument, they more like strong opinion on things that are closer to esthetics and right/wrong usage of things. Happy to disagree on those one. > There's a clear difference between building an ecosystem with multithreading in mind from day one - and suddenly introducing it out of nowhere, 30 years into it being used by millions of workloads globally. 1 - no-gil isn't out of nowhere. Conversation about this are more that 10 years old. Combined with even longer conversation in other VM/programming language. 2 - no-gil doesn't introduce threading in random workload. It allow people who want threading to use threading. > This is why I'm asking for "qualifications" It's your prerogative to ask for qualifications. But it's also our to decide to judge your ask and decide if they are worth our time. Much in the same way that if feel like we don't need (in 2023) a grand example to justify why we need to add a type system, we don't need a deep conversation about threading. We all have the same information, understand the trade-off. We just have different value system and want different things out of python. And that's okay... > the problem is not that deep" to be a proof beyond any that we should end this conversation now before things get too embarrassing for you :) I think i have some comment somewhere explaining why no-gil was never a technically challenging problem. But the prof is simple... Sam Gross is definitely an exceptional dev, but the fact that a lone programmer come out with an acceptable solution is proof that the problem wasnt that deep.