5 ms·

It's really a shame you are getting down voted. You are basically just saying, "Sometimes shared memory model makes a more efficient and simpler solution over e

It's really a shame you are getting down voted. You are basically just saying, "Sometimes shared memory model makes a more efficient and simpler solution over enforced actor model. Like in this case." This shouldn't be controversial, engineering choices come with tradoffs and the GIL certainly pushes you for making a specific choice. Yes, if you are going to be doing caching in python, you are almost certainly going to be caching it in a separate service and serializing across process boundaries. If you are going to be holding onto a connection pool as well (PG bouncer being the most typical). More generally, any sort of shared resource that would require or benefit strongly from shared memory model needs to be pushed into a sidecar service written in a language that actually supports efficient shared memory model. These are precisely the sorts of problems that the GIL causes. There are standard ways to work around it and most python developers can reach for these solutions, but this is not a contradiction to the fact that the GIL has limited your choice in the matter.

dragonwriter 5y ago
> It’s really a shame you are getting down voted. You are basically just saying, “Sometimes shared memory model makes a more efficient and simpler solution over enforced actor model. Like in this case.” If he was just saying that, he’d probably be downvoted for being true but irrelevant and not meaningfully contributing. But he’s actually saying that plus “therefore, Python is bad (implying that Python can’t use shared memory between processes).” Which would be a productive, relevant contribution except for the fact that it’s not true. > More generally, any sort of shared resource that would require or benefit strongly from shared memory model needs to be pushed into a sidecar service written in a language that actually supports efficient shared memory model. That’s I guess technically true, if you included the stdlib modules which directly support this and are themselves written in (I assume) C. But, “you might have to use the standard library” is…not much of an impediment.
- BlackFly 5y ago
  No, he is making the useful observation that the GIL has an impact on system architecture. That is why typically python webservers run in multiprocess mode but Java (for example) webservers typically runs in threaded mode.
- matsemann 5y ago
  I didn't say python was bad. And I didn't say that it's impossible to share data between processes (I even mentioned some options). My point was just that the GIL, while not a dealbreaker or anything, actually _does_ affect how a program is written. So it's implicitly noticeable in my day-to-day.
  - zo1 5y ago
    I've been developing python for all manner of use-cases the last 15 years and have not once had the GIL impact me in any meaningful way. I have however been impacted way more by the amount of misconceptions about python that I've had to explain over the years.
    - matsemann 5y ago
      That's the point, though: One doesn't see the impact directly. But after 15 years you just do it the python way by default, not considering other ways that stuff could be solved if it weren't for the limitations imposed by the platform. The python ways work fine, so it's OK even with GIL, but there are other ways as well.
lmm 5y ago
> You are basically just saying, "Sometimes shared memory model makes a more efficient and simpler solution over enforced actor model. Like in this case." Is this actually showing that it's less efficient though? People rarely complain about Erlang doing the same thing. "You have to spin up a proper task queue rather than firing off an unmanaged background thread" does not sound like an unambiguous negative to me.
- Joker_vD 5y ago
  Well, there are shared ETS in Erlang which we actually use for caching lots of things (e.g., the formatted timestamp with 1 second lifetime -- it's faster to look it up than recalculate in every process, -- or, yes, auth tokens).

dragonwriter 5y ago

> It’s really a shame you are getting down voted. You are basically just saying, “Sometimes shared memory model makes a more efficient and simpler solution over enforced actor model. Like in this case.” If he was just saying that, he’d probably be downvoted for being true but irrelevant and not meaningfully contributing. But he’s actually saying that plus “therefore, Python is bad (implying that Python can’t use shared memory between processes).” Which would be a productive, relevant contribution except for the fact that it’s not true. > More generally, any sort of shared resource that would require or benefit strongly from shared memory model needs to be pushed into a sidecar service written in a language that actually supports efficient shared memory model. That’s I guess technically true, if you included the stdlib modules which directly support this and are themselves written in (I assume) C. But, “you might have to use the standard library” is…not much of an impediment.

BlackFly 5y ago
No, he is making the useful observation that the GIL has an impact on system architecture. That is why typically python webservers run in multiprocess mode but Java (for example) webservers typically runs in threaded mode.
matsemann 5y ago
I didn't say python was bad. And I didn't say that it's impossible to share data between processes (I even mentioned some options). My point was just that the GIL, while not a dealbreaker or anything, actually _does_ affect how a program is written. So it's implicitly noticeable in my day-to-day.
- zo1 5y ago
  I've been developing python for all manner of use-cases the last 15 years and have not once had the GIL impact me in any meaningful way. I have however been impacted way more by the amount of misconceptions about python that I've had to explain over the years.
  - matsemann 5y ago
    That's the point, though: One doesn't see the impact directly. But after 15 years you just do it the python way by default, not considering other ways that stuff could be solved if it weren't for the limitations imposed by the platform. The python ways work fine, so it's OK even with GIL, but there are other ways as well.

lmm 5y ago

> You are basically just saying, "Sometimes shared memory model makes a more efficient and simpler solution over enforced actor model. Like in this case." Is this actually showing that it's less efficient though? People rarely complain about Erlang doing the same thing. "You have to spin up a proper task queue rather than firing off an unmanaged background thread" does not sound like an unambiguous negative to me.

Joker_vD 5y ago
Well, there are shared ETS in Erlang which we actually use for caching lots of things (e.g., the formatted timestamp with 1 second lifetime -- it's faster to look it up than recalculate in every process, -- or, yes, auth tokens).