17 ms·

Could someone who really wants to get rid of the GIL explain the appeal? As far as I understand, the only time it would be useful is when you have an applicat

• make3 9y ago
  There are many cases where the objects are too big to be passed around. Python is used a huge amount in Machine learning and datascience, where being able to do parallel work on stuff already in memory would be great.
  • leereeves 9y ago
    Are those applications often bottlenecked by the CPU, as opposed to GPU or data transfer?
    • rspeer 9y ago
      The world of algorithms that run well on a CPU is still much, much bigger than the world of algorithms that run well on a GPU, even in machine learning. And even if you're fortunate enough that Nvidia designs their GPUs to solve your problem, why should the CPU cores sit idle?
  • dec0dedab0de 9y ago
    So work on data that can not be broken down into smaller chunks? That makes sense, and is something I never come across.
    • chrisseaton 9y ago
      I'm sure they can be broken down into smaller chunks, but is it more efficient if they aren't broken down and instead shared memory is used? If you want parallelism you're obviously already worried about performance.
  • smaddox 9y ago
    Can't this already be handled by calling out to a C/C++ or FORTRAN procedure that processes the data in multiple threads? For number crunching, Python is almost exclusively used as glue.
    • ams6110 9y ago
      Today's machine learning and data science students don't know how to code in those languages. They know python, and maybe java.
      • orangejewce 9y ago
        Don't forget R... shudders
        • sin7 9y ago
          What's wrong with R? I know it's not a programmers language but it's great for getting things done.
    • foobarchu 9y ago
      You CAN handle it, but why should you have to? If it's possible to remove that barrier, then it absolutely should be removed. If the only answer to a problem is "use another language", then the language in question has a limitation that needs to be addressed.
      • Drdrdrq 9y ago
        It is not a limitation at all in this case. Python is just a front to Tensorflow and similar libraries/frameworks so GIL doesn't matter there.
    • deleted 9y ago
      [deleted]
  • fulafel 9y ago
    Something like the web worker primitives might work there (transferables & sharing read only data).
• ant6n 9y ago
  Using multiprocessing is a pain to use, and it's slow.
  • dec0dedab0de 9y ago
    in what was is it a pain that threading is not?
    • cyphar 9y ago
      You can't just use functions defined in your tool, you need to create a faux-cli interface in order to run each parallel worker. Also, copying large datasets between processes is not efficient. And also, there are cases where the fan-out approach is not the best way of parallelizing a task, and passing information back up to a parent task is more complicated than necessary.
      • dec0dedab0de 9y ago
        "You can't just use functions defined in your tool, you need to create a faux-cli interface in order to run each parallel worker." the multiprocessing library allows you to launch multiple processes using your function definitions. It's almost the same as the multithreading library but does not share data. It seems the real problem, as you pointed out, is the additional memory. I didn't consider situations where each process would need an identical large data set, instead of just a small chunk to work on.
        • ant6n 9y ago
          It gets more interesting when you have a large data set that's required for the computation, but as you compute, you may discover partial solutions that can be cached and used by other workers. So not only a large read-only data set, but also a read-write cache used by all workers. This sort of thing is relatively easy with threads, but basically impossible with multiprocessing.
          • 9y ago
• btown 9y ago
  Say you're running CPU-bound workers that need to load significant data into RAM - say, a machine learning or NLP model. The most cost-effective theoretical approach would be to have that in shared memory, so you're not paying for that RAM multiple times in order to fully utilize all cores. Even if you need multiple boxes, the cost savings per core would be substantial. My understanding is that multiprocessing makes you jump through hoops to set up that shared memory; this would make it largely transparent to the user while remaining performant. I haven't used multiprocessing in production, though, so I could be wildly off base there.
  • eddigo 9y ago
    I've run into this exact situation. I have a C extension that does some specialized vector operations. These functions use some language models that can be fairly large. The language models are fixed throughout the calculations, and the other data required can be segmented so the operations can happen in parallel. Also, the language model and data can be handled as read-only. Trying to do this with the multiprocessing module is not terribly straightforward.
  • dom0 9y ago
    Unless your model actually consists of a large number of Python objects (and not a handful of PyObjects referencing something like a np array), there isn't really anything blocking you from doing so. You can have a master process map the blob of static data into a block of shared memory that's mapped by the secondary processes; ctypeslib lets you access it as a numpy array again.
• m_mueller 9y ago
  In my five years of python I've run up against this boundary at least once. In your list I would * take out #2. if something can make use of multiple nodes it can usually make even better use of multi-core parallelization (which affects both computational and memory bandwidth performance). multi-node comes with a much higher communications overhead, so there's a relatively wide range of applications that scale well on multi-core but not multi-node. * add that #3 comes up as soon as you have complex data structures to share. Serializing and Deserializing (by default with pickle) is a huge overhead for anything a bit more involved. If you design for this from the start you can be fine, but often these things grow and eat up bigger and bigger usecases until you run against the GIL. This basically happens with anything that has enough data and users and need - hey I heard your scheduler tool works well for the cafeteria, I'm sure it can handle our global operations right? * about #4 - see the previous point.
  • njharman 9y ago
    Once (or a few times) in 5 years puts this problem into the "not worth(ROI) solving" bucket for me. Those few times, put down the hammer and use some other tool for those not naillike jobs.
    • neolefty 9y ago
      Certainly not worth it for one person to tackle the GIL, but a million people running into it a few times in 5 years, and I think it's economical.
    • m_mueller 9y ago
      Here's the thing: Python, especially 3.6, is such a well rounded language that all other major limitations have IMO been solved already. In my view the GIL is the main one left, and reason to pause and think whether python is a good idea at the start of a project. Removing it is therefore worth it, and would also give a nice additional incentive for everyone to switch to python 3.x, so we don't have to keep on maintaining 2.7 with the same code (i.e. the worst of both worlds).
• dkersten 9y ago
  To add to what everyone else said, if you need transactional semantics, its much simpler in multiple threads. With multiple processes (local or remote), you can't simply share an atomic data structure or a lock, you have to use a distributed lock or consensus algorithm, which are more complex and usually quite "chatty". If memory or network bandwidth are constrained, it may be especially desirable to eliminate this, but even if not, fast locking/transactions may be desirable regardless. If you're using multiple processes for CPU-bound performance, why not squeeze as much as you can out of each CPU?
  • dom0 9y ago
    Just like you can share memory between processes, you can also share OS-level locks and semaphores between them. A distributed lock manager is not required for the single-node case.
• colesbury 9y ago
  Your criteria 2, 3, and 4 doesn't make much sense to me. We often have workloads that require multiple boxes, but we still want to make effective use of each box. Common server hardware has dozens of cores, which requires a lot of parallelism to fully utilize. The GIL hinders that, even when most of the work doesn't hold the GIL (see Amdahl's law) Python multiprocessing doesn't work well with a lot of external libraries. For example, CUDA doesn't work across forks and many system resources can be shared across threads but not processes. Python objects must be pickled to be sent to another process, but not all objects can be pickled (including some built-in objects like tracebacks). A lot of different parallel programming models can be built on top of threads (shared memory, fork-join, message passing), and to a certain extent they can be mixed. That's not true of Python multiprocessing, which only allows a narrow form of message passing. (It's also buggy, has internal race conditions, and easily leaks resources.) The problem for CPython is that it may not be possible to remove the GIL without breaking the C API, and a lot of the benefit of Python is the huge number of high-quality packages, many of which use the C API.
  • PrimHelios 9y ago
    CPython doesn't have any reservations about breaking the Python API between minor versions, so why care about the C API? I get where you're coming from, but they've already shown they don't care much for compatibility, so I don't see why that's a big obstacle.
    • dom0 9y ago
      Removing the GIL (in a non-braindead way) likely entails breaking all existing code using the C API. PyPy could do so without breaking cpyext, by maintaining the illusion of a GIL whenever control passes to cpyext.
      • std_throwaway 9y ago
        Does it lock the GIL so numpy can release it again immediately afterwards?
• sandGorgon 9y ago
  +1 on this - what is more important for me is some kind of Numba LLVM jit to automatically optimize hotspots : kind of like the JVM hotspot compiler. Numba already does some of this. Additionally, I cannot help but wonder if the answer to these problems has been the JVM all along. Especially with JVM 9 and the Truffle framework - https://github.com/securesystemslab/zippy https://github.com/securesystemslab/zippy
  • rbjorklin 9y ago
    I was just about to mention Graal & Truffle when I saw your post! I wasn't aware of ZipPy but it looks promising! Java 9 will provide a proper interface for Graal through JVMCI and is only 37 days away from GA [1]. With Graal supposedly only months away from GA [2], ZipPy may very well prove to be the future of high performance Python. [1] http://www.java9countdown.xyz/ http://www.java9countdown.xyz/ [2] https://www.infoq.com/presentations/polyglot-jvm-graal https://www.infoq.com/presentations/polyglot-jvm-graal (see roughly 42:00 - 47:00) EDIT: Wording.
• kllrnohj 9y ago
  #1 & #2: Consumer CPUs are now pushing 16 cores & 32 threads. Python is limited to ~1/20th of what a single box is capable of. That's a pretty big bottleneck. #4: Even if you're just talking message passing sending a message between threads is in the 10s of nanoseconds while between processes is 10s of microseconds. That's a ~1000x slowdown on core communication. Given that CPU cores are not getting any faster, that's a pretty big hit to efficiency to take. Similarly simply moving data between processes is expensive, while moving data between threads is free.
  • sqeaky 9y ago
    You are right about the majority of what you said, but I am pedantically picking on one point. CPU cores are getting faster, but they aren't doing it with clock speed, they are dispatching more instructions per cycle or otherwise making the work faster.
    • kllrnohj 9y ago
      IPC gains per generation are vanishingly tiny, if they exist at all. Skylake -> Kaby Lake, for example, had no IPC improvements at all. A very small clock bump to the various tiers was it. Even if you look over a large generation gap there's only a ~20% IPC improvement going from an i7-2600K to an i7-7700K ( https://www.hardocp.com/article/2017/01/13/kaby_lake_7700k_vs_sandy_bridge_2600k_ipc_review/ https://www.hardocp.com/article/2017/01/13/kaby_lake_7700k_v... ) 6 years & a shrink from 32nm to 14nm and all it can muster is +20%. Cores are just not getting faster by any meaningful amount.
      • happycube 9y ago
        Yeah - I remember when five year old CPUs were basically useless! (Kaby Lake is basically a new stepping of Skylake - if intel wasn't having problems with new process nodes it likely wouldn't have been released at all, and if it was it would've been used for a one-off chip in the same generation ala the 4770K)
      • sqeaky 9y ago
        My compiles have gotten more than 20% faster so something is making my newere machines faster than my older machines. That it is not 10x as fast I blame on AMD for not being as competitive as they could have been.
• gaius 9y ago
  I haven't written shared memory code in literally years, I just use Redis now.
• sametmax 9y ago
  It's just a low hanging fruit for perfs from the dev point of view. It's nice and useful, just nowhere as needed as most people asking for it pretend it is.
• ddustin 9y ago
  My god. This website is full of web script-kiddies. I've run into this need at least a dozen times.
• omarforgotpwd 9y ago
  Motivation for removing the GIL is basically that when people hear about it they go "hmmm that doesn't sound good". Obviously many applications have been written in GIL languages and there aren't really many practical problems that can't be overcome easily.
  • neolefty 9y ago
    I think it may be some Stockholm Syndrome -- people have worked very hard to get around the GIL, and they've come to expect its limitations and respect those solutions. But I've never heard of someone asking for a GIL to be added to the JVM.
    • astrodust 9y ago
      Exactly! If a feature is good it's worth adding, and the GIL in any language is not good. People are just as quick to bemoan a language for not having something (generics, templates, pre-processors) because they see some perceived need, but a GIL is never one of those things.
    • titanix88 9y ago
      This! Try to implement a controlled task scheduler using multiprocessing and sooner or later you are going to hit some unexpected behavior, like - multiprocessing.Queue belly-upping for no reason, UNIX signals propagating throughout the process chain and killing them left and right, hitting some data/object which are not serializable etc. Getting multiprocessing to work right takes a LOT of careful efforts, which breaks the whole promise python. I've since moved to clojure, which is a language designed with concurrency from ground up. Look at clojure's `atom` - it's basically what every beginning programmer expects from globally shared variables, minus the gymnastics of handling race conditions on your own. Also, `core.async` is such a beautiful thing to work with for writing schedulers. Compared to this, python's asyncio is an unfunny joke. I don't think python's GIL can be removed with ad-hoc locking. Nothing sort of complete re-implementation will do.
• ddustin 9y ago
  This comes up constantly when python is used for compiler scripts. It super needs to be fixed.