5 ms·

And why GIL is the problem, in your opinion? Honest question, explain your point of view.

by mdomans 10y ago

jerf 10y ago
In a nutshell, if your thesis was true, it would be true. If Celery was the solution to the GIL, nobody would still be talking about the GIL. Celery may be slick and well-developed, but sharing stuff out to processes has been well-established tech for a long time. If that were all people wanted, the problem would be solved. Since it's not, people must want different things. It's great that you solved your problems with it. Many other people have too. Many others have problems that are less amenable to this approach.
- mdomans 10y ago
  Assuming that people will stop talking about something because it's not a problem or it isn't real is a bit ... uninformed :) The main point I was trying to make is that if you can force the programmer to guarantee safety, you don't need to rewrite GIL. And approach Celery enforces you to use, can be, with some extra syntax sugar, back ported to Python to have nice threading API
  - jerf 10y ago
    Very well, then I'll also point out that among the people still talking about the GIL are those who know enough about Python to successfully take steps to remove it, which is pretty much the highest possible level of knowledge of Python you can ask for. This is probably a topic "in the air" precisely because of the story of someone's most recent attempt/implementation of this a couple of days ago. It is not the case that only "people who don't really know Python" are talking about the GIL and considering the problem unsolved. If you were right, these people would not be talking about it. I would strongly, but politely, suggest that you should consider the possibility that there either things you don't know about the issue or problems that others have that you don't. It's not like Celery is the only obscure library nobody's ever heard of that has this basic approach; "multiprocessing" is built into the standard library now, for instance.
  - mdomans 10y ago
    I disagree. Most people working on GIL in Python this days are definitely smart, but they're brute forcing the problem. It's not about writing more code, for me, but about writing smarter code. GCD was, for me, an example of such feat. It introduced an approach that obsoleted thread as a concept you work with
    - jerf 10y ago
      "Most people working on GIL in Python this days are definitely smart, but they're brute forcing the problem." I have no idea how to translate that into the real actions I'm seeing. What is "brute forcing the GIL"? If you mean "solving the problem in the runtime instead of writing 'better' code", well, per my previous post there are times the "better code" can't be written in Python anyhow, but putting that aside, there's all sorts of reasons to fix things in runtimes rather than require people to "write better code". First of all, you get massive leverage fixing things in runtimes because the improvements directly apply to the staggeringly, incomprehensibly enormous pile of already existing code in the field. Secondly, the "just write better code, everybody" has a multi-decade track record of multidimensional failure in every metric, from code quality, security, performance, reliability, everything. What can work is making correct code easier to write, which doing things like fixing the GIL can for the certain cases where that's a problem; telling everybody to do the harder-but-correct thing does not work at any scale much beyond 1-developer code.

And why GIL is the problem, in your opinion? Honest question, explain your point of view.

by mdomans 10y ago

And why GIL is the problem, in your opinion? Honest question, explain your point of view.

jerf 10y ago

In a nutshell, if your thesis was true, it would be true. If Celery was the solution to the GIL, nobody would still be talking about the GIL. Celery may be slick and well-developed, but sharing stuff out to processes has been well-established tech for a long time. If that were all people wanted, the problem would be solved. Since it's not, people must want different things. It's great that you solved your problems with it. Many other people have too. Many others have problems that are less amenable to this approach.

mdomans 10y ago
Assuming that people will stop talking about something because it's not a problem or it isn't real is a bit ... uninformed :) The main point I was trying to make is that if you can force the programmer to guarantee safety, you don't need to rewrite GIL. And approach Celery enforces you to use, can be, with some extra syntax sugar, back ported to Python to have nice threading API
- jerf 10y ago
  Very well, then I'll also point out that among the people still talking about the GIL are those who know enough about Python to successfully take steps to remove it, which is pretty much the highest possible level of knowledge of Python you can ask for. This is probably a topic "in the air" precisely because of the story of someone's most recent attempt/implementation of this a couple of days ago. It is not the case that only "people who don't really know Python" are talking about the GIL and considering the problem unsolved. If you were right, these people would not be talking about it. I would strongly, but politely, suggest that you should consider the possibility that there either things you don't know about the issue or problems that others have that you don't. It's not like Celery is the only obscure library nobody's ever heard of that has this basic approach; "multiprocessing" is built into the standard library now, for instance.
- mdomans 10y ago
  I disagree. Most people working on GIL in Python this days are definitely smart, but they're brute forcing the problem. It's not about writing more code, for me, but about writing smarter code. GCD was, for me, an example of such feat. It introduced an approach that obsoleted thread as a concept you work with
  - jerf 10y ago
    "Most people working on GIL in Python this days are definitely smart, but they're brute forcing the problem." I have no idea how to translate that into the real actions I'm seeing. What is "brute forcing the GIL"? If you mean "solving the problem in the runtime instead of writing 'better' code", well, per my previous post there are times the "better code" can't be written in Python anyhow, but putting that aside, there's all sorts of reasons to fix things in runtimes rather than require people to "write better code". First of all, you get massive leverage fixing things in runtimes because the improvements directly apply to the staggeringly, incomprehensibly enormous pile of already existing code in the field. Secondly, the "just write better code, everybody" has a multi-decade track record of multidimensional failure in every metric, from code quality, security, performance, reliability, everything. What can work is making correct code easier to write, which doing things like fixing the GIL can for the certain cases where that's a problem; telling everybody to do the harder-but-correct thing does not work at any scale much beyond 1-developer code.