5 ms·

I agree regarding the symptoms, but am less convinced that run-time efficiency is the sole (or perhaps even major) cause. If it were, I'd argue that we'd see l

I agree regarding the symptoms, but am less convinced that run-time efficiency is the sole (or perhaps even major) cause. If it were, I'd argue that we'd see less usage of, e.g., Python. I don't know the true cause -- I wish I did. But I do do see a trend towards an ever more "hands-off" style of software development: large volumes of automated (especially unit) tests in preference to interactive approaches (or hybrids, like running subsets of tests via a REPL), and running test instances on remote servers (often via extra layers of indirection, such as CI servers) rather than running on your own machine. I'd love to see a resurgence of interactivity, but if anything it seems to be going against recent trends. Edit: The subjective impression I've got is that doing things the indirect, hands-off way is being presented as somehow more "professional" while hands-on interactivity is the realm of the self-taught and hackers. Do others see this? What can be done to change these perceptions?

lisper 8y ago
This decision was made and become deeply entrenched long before Python came along. Python is pretty good, but even it is constrained by the decisions that went into the design of C, since Python is implemented in C. Its underlying data structures are C data structures. Its memory management is C memory management. The GIL is there because C.
- jstimpfle 8y ago
  > The GIL is there because C. Oh please.
- zbentley 8y ago
  > The GIL is there because C. That's not at all true. There are very, very few platforms/libraries/languages that both a) allow you to manage concurrent access to complex, arbitrary data structures in a sane, predictable, and not-dangerous way and b) provide the capabilities (including performance) necessary to implement a general-purpose programming language on top of them. Even fewer of those tools existed when Python came about. Was it technically possible to make a GIL-free Python or a Python not based in C? Sure. But it wasn't in any way likely, or a reasonable-at-the-time decision. If you look into the history of the GIL things will make more sense; it has next to nothing to do with the implementation language.
  - lisper 8y ago
    I'm not saying that trading the GIL for the benefits of C was not a reasonable tradeoff. Nonetheless, it is simply a fact that the GIL is needed because of the constraints imposed by choosing to work in C.
    - zbentley 8y ago
      Citation needed. “Because there existed languages with different concurrency models” doesn’t really explain why the choice of C for Python’s implementation necessitated a GIL (there are plenty of languages and tools built on C with non-GIL-ish concurrency systems, and many languages/tools built on not-C that don’t expose the concurrency features of their underlying language at all). Look into how/why then GIL was added; it has much more to do with not wanting to reimplement the language/break compatibility than it has to do with what the language was implemented in.
      - lisper 8y ago
        > it has much more to do with not wanting to reimplement the language/break compatibility I don't know what "language/break compatibility" means. The GIL is needed because Python's reference-counted memory management system is not thread-safe, and this can't be fixed without compromising either portability or performance. It's really that simple.
        zbentley 8y ago
        > The GIL is needed because Python's reference-counted memory management system is not thread-safe. That is correct. What does that have to do with C? Thread-unsafe code exists in all languages. The GIL's lock is itself a pthread mutex and condvar underneath, and equivalent constructs exist in all (to my knowledge) modern threaded programming environments. "not wanting to reimplement the language/break compatibility" is a reference to the successful efforts that have been made to remove the GIL in CPython. Those efforts have not (yet) moved towards merging into mainline CPython because they require a) lots of reimplementation work and added complexity in the language core, and b) would very likely break the majority of compiled extension modules. I think that's additional evidence that the GIL isn't a C problem; they removed it, in C, without fighting or otherwise working around the language.
- CamouflagedKiwi 8y ago
  The GIL is not there because C. It's there because of refcounting and Python guaranteeing that many nontrivial operations are thread safe.
  - lisper 8y ago
    Yes, but refcounting is there because it is impossible to build a proper GC in portable C.
    - hexomancer 8y ago
      Can you elaborate on this?
      - lisper 8y ago
        You can't trace the stack in portable C.
jeremyjh 8y ago
Sometimes I do work this way in the repl - usually when I have little confidence I even understand how to fit together a new library or API call. But if you spend more time in the repl building a data structure and interactively iteratively testing your function, when it is all finished all you have checked into source control is a function, not the tests. You have to then write the tests. In a lot of cases its faster to just write the tests and interact with the code through those. In some language environments this is not exclusive of using a repl - in Haskell for instance it is pretty easy to interactively run a module and tests saved to disk in the repl.
- dasmoth 8y ago
  Perhaps if great interactive development and debugging tools were more pervasive, there would be less demand for lots of tests written at the same time as the actual function. (The fine-grained "unit" stuff, anyway. System/integration tests come with different trade-offs).
deleted 8y ago
[deleted]

lisper 8y ago

This decision was made and become deeply entrenched long before Python came along. Python is pretty good, but even it is constrained by the decisions that went into the design of C, since Python is implemented in C. Its underlying data structures are C data structures. Its memory management is C memory management. The GIL is there because C.

jstimpfle 8y ago
> The GIL is there because C. Oh please.
zbentley 8y ago
> The GIL is there because C. That's not at all true. There are very, very few platforms/libraries/languages that both a) allow you to manage concurrent access to complex, arbitrary data structures in a sane, predictable, and not-dangerous way and b) provide the capabilities (including performance) necessary to implement a general-purpose programming language on top of them. Even fewer of those tools existed when Python came about. Was it technically possible to make a GIL-free Python or a Python not based in C? Sure. But it wasn't in any way likely, or a reasonable-at-the-time decision. If you look into the history of the GIL things will make more sense; it has next to nothing to do with the implementation language.
- lisper 8y ago
  I'm not saying that trading the GIL for the benefits of C was not a reasonable tradeoff. Nonetheless, it is simply a fact that the GIL is needed because of the constraints imposed by choosing to work in C.
  - zbentley 8y ago
    Citation needed. “Because there existed languages with different concurrency models” doesn’t really explain why the choice of C for Python’s implementation necessitated a GIL (there are plenty of languages and tools built on C with non-GIL-ish concurrency systems, and many languages/tools built on not-C that don’t expose the concurrency features of their underlying language at all). Look into how/why then GIL was added; it has much more to do with not wanting to reimplement the language/break compatibility than it has to do with what the language was implemented in.
    - lisper 8y ago
      > it has much more to do with not wanting to reimplement the language/break compatibility I don't know what "language/break compatibility" means. The GIL is needed because Python's reference-counted memory management system is not thread-safe, and this can't be fixed without compromising either portability or performance. It's really that simple.
      - zbentley 8y ago
        > The GIL is needed because Python's reference-counted memory management system is not thread-safe. That is correct. What does that have to do with C? Thread-unsafe code exists in all languages. The GIL's lock is itself a pthread mutex and condvar underneath, and equivalent constructs exist in all (to my knowledge) modern threaded programming environments. "not wanting to reimplement the language/break compatibility" is a reference to the successful efforts that have been made to remove the GIL in CPython. Those efforts have not (yet) moved towards merging into mainline CPython because they require a) lots of reimplementation work and added complexity in the language core, and b) would very likely break the majority of compiled extension modules. I think that's additional evidence that the GIL isn't a C problem; they removed it, in C, without fighting or otherwise working around the language.
CamouflagedKiwi 8y ago
The GIL is not there because C. It's there because of refcounting and Python guaranteeing that many nontrivial operations are thread safe.
- lisper 8y ago
  Yes, but refcounting is there because it is impossible to build a proper GC in portable C.
  - hexomancer 8y ago
    Can you elaborate on this?
    - lisper 8y ago
      You can't trace the stack in portable C.

jeremyjh 8y ago

Sometimes I do work this way in the repl - usually when I have little confidence I even understand how to fit together a new library or API call. But if you spend more time in the repl building a data structure and interactively iteratively testing your function, when it is all finished all you have checked into source control is a function, not the tests. You have to then write the tests. In a lot of cases its faster to just write the tests and interact with the code through those. In some language environments this is not exclusive of using a repl - in Haskell for instance it is pretty easy to interactively run a module and tests saved to disk in the repl.

dasmoth 8y ago
Perhaps if great interactive development and debugging tools were more pervasive, there would be less demand for lots of tests written at the same time as the actual function. (The fine-grained "unit" stuff, anyway. System/integration tests come with different trade-offs).

deleted 8y ago

[deleted]