9 ms·
Lots of C library code for decades carried man page warnings it was unstable for use in async, re-entrant and recursive contexts. We learned how to cope and inc
by ggm 3y ago
Lots of C library code for decades carried man page warnings it was unstable for use in async, re-entrant and recursive contexts. We learned how to cope and incrementally re-entrant safe versions deployed without too much API instability. Maybe time has healed wounds and caused me memory loss of the pain of discovery you'd tripped over them.
String parsing which tokenised in-place. DNS calls which used static buffers. Things which exploited Vax specific stack behaviour.
I think the GIL has been a blessing and a curse.
- neilv 3y agoI remember scouring those C runtime docs, for every non-reentrant function. It might be what got me in the habit of checking docs when using some API that I know moderately well, just in case there's some important detail I missed before, or something had changed. Around that time, doing cross-platform C++, I got an early look at Java, with concurrency built in from the start, along with GC and various other nice features that were easier to use than C++, and I "knew" it was going to be huge. (But who knew that the MIS people would take over Java, when it seemed clearly targeted at non-MIS programmers, and now MIS people are stuck with the C++ syntax and verbosity, after coming from 4GLs, etc.) Then mainstream programmers picked up Python, which, IIRC, originally was an embeddable extension language, which was why it was simple. And for which the GIL made more sense.
- eru 3y ago> I got an early look at Java, with concurrency built in from the start, [...] and I "knew" it was going to be huge. And Java doesn't even have good (conceptual) support for concurrency. Compared with eg Erlang, Rust or even Haskell. But Java was still better at it than C or C++ at the time.
- dboreham 3y agoIn your opinion. Some of us believe the exact opposite.
- soulbadguy 3y agoThere are a lot of good things about java (and the jvm by some extend), but good concurency is not one of them.
- bitcharmer 3y agoThis is not true. JSR-133 was one of the first and most prominent implementation of memory model among general purpose languages. You don't seem to know enough about the platform and it's history to make definite statements like that, imo.
- grumpyprole 3y agoJSR-133 was quite a few years after Java's release. But even after these changes, many problems remained. For example, standard library classes were often not thread safe out-of-the-box, even something seemingly innocuous like a date formatter class would malfunction in a threaded context. The rules around thread safety were also very complex, especially with regard to constructing objects. IMHO, it did not compare well with other languages around the same time built with concurrency in mind, for example Erlang. I was once very fortunate to meet Joe Armstrong and we briefly discussed Java, suffice to say he was not impressed by it.
- soulbadguy 3y ago+1
- eru 3y agoBy exact opposite you mean that when Java just came out ('I got an early look at Java, [...]') had good concurrency support? One can make a solid argument that modern Java is adequate. But I don't buy you believe that of early Java?
- ungamedplayer 3y agoOld mate isn't alone.
- pseudocomposer 3y agoI agree that Rust handles concurrency about as well as Java (in addition to its memory safety). But could you provide docs on Erlang or Haskell concurrent data structures that were better than Java’s, say, 10-20 years ago? Java’s support for concurrent data structures was pretty unprecedented in its heyday.
- quickthrower2 3y agoSTM, 2006? https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.67.3686&rep=rep1&type=pdf https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.67... Still way after Java was released I guess
- grumpyprole 3y agoJava didn't initially ship with lock-free data structures, those came later thanks to Doug Lea.
- seanhunter 3y agoThe best example of this is probably the “Beautiful Concurrency” paper from Simon Peyton-Jones in 2007 https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/beautiful.pdf?from=https://research.microsoft.com/en-us/um/people/simonpj/Papers/stm/beautiful.pdf&type=exact https://www.microsoft.com/en-us/research/wp-content/uploads/...
- SkiFire13 3y agoCompared to those other languages, Java has no mechanism to enfore the use of concurrent data structures, it's entirely up to the programmer to use them.
- grumpyprole 3y agoJava still doesn't provide persistent data structures out-of-the-box. IMHO these are more important than lock-free mutable data structures. When one really does need shared mutable state, Haskell supports transactional updates to mutable state (STM), with optimistic locking and rollbacks. It's awesome, but unfortunately just not practically possible in any language with rampant untracked side-effects.
- almostnormal 3y ago> But Java was still better at it than C or C++ at the time. That's not even a contest. Java was released in 1995. At that time C did not have any kind of support for concurrency, all solutions were platform dependend and not part of the language.
- PaulDavisThe1st 3y ago> I remember scouring those C runtime docs, for every non-reentrant function. hint: % cd /usr/include % ag --no-color '[^A-Z]_REENTRANT'
- cjfd 3y agoWhat are 'MIS people'?
- vore 3y agoManagement Information Systems, I think.
- andybak 3y agoI do so love playing guessing games for every comment I read.
- bigbillheck 3y agoI don't think it's reasonable to expect people here to cater to a least common denominator of comprehension.
- paganel 3y agoNothing “least common denominator of comprehension” about not knowing what that acronym means. I also didn’t know.
- nirvdrum 3y agoThat's fine. But, communication is a skill and like all skills, can be improved. For my part, this is the first time I can remember seeing MIS used in the wild (i.e., not in university). Given the tech industry's propensity for overloading initialisms and redefining terms, I have to go by context to work out the meaning. In this case, the context was insufficient. I suppose I could just dismiss the comment, but curiosity got the better part of me. I'm glad someone asked. I also had to search what "4GL" means. I'm assuming this top Wikipedia hit [1] is what the OP meant. I don't know if it's reasonable to expect people to know these terms. But, the concise text is not saving anyone beyond the OP any appreciable amount of time, so what's its value? [1] https://en.m.wikipedia.org/wiki/Fourth-generation_programming_language https://en.m.wikipedia.org/wiki/Fourth-generation_programmin...
- otabdeveloper4 3y ago> originally was an embeddable extension language Python was originally a teaching language, to replace BASIC and Pascal for kids and beginners. (Yes, there once was a time when people took teaching programming seriously.)
- tromp 3y agoQuoting from [1], section "The ABC Project" > The letter “B” was chosen because it is the first letter of the word “beginner” and because the project was meant to become a language for teaching programming to absolute beginners. [1] https://inference-review.com/article/the-origins-of-python https://inference-review.com/article/the-origins-of-python
- eesmith 3y agoPython was not originally a teaching language. Yes, van Rossum was an implementer of ABC, which was a teaching language. Python drew on that experience, but also from Modula-2 and Modula-3, and the needs for writing an extension language for the Amoeba operating system. From the initial release notes at https://www.tuhs.org/Usenet/alt.sources/1991-February/001749.html https://www.tuhs.org/Usenet/alt.sources/1991-February/001749... : [Python is] an extensible interpreted programming language that combines remarkable power with very clear syntax. This is version 0.9 (the first beta release), patchlevel 1. Python can be used instead of shell, Awk or Perl scripts, to write prototypes of real applications, or as an extension language of large systems, you name it. No mention of it being a teaching language.
- weinzierl 3y agoThat Java had concurrency built in from the start is a blessing mostly, but also a bit of a curse. Most of the Java ecosystem is still in the mindset that threads are cheap and firing up a couple more cannot hurt. So we end up with apps that run thousands of threads and this disease is hard to contain.
- samus 3y agoFortunately Virtual Threads have landed, so when the ecosystem finally comes around to adopt it, many things might become faster.
- pjmlp 3y agoLike not everyone toy app isn't going to be the next FAANG, there are plenty of workloads where it hardly matters, while 30 years later it is still a mess in C and C++. And between C++ and Rust coroutines, still not sure which one I like less.
- doliveira 3y agoI thought so too until I got to interact with databases and Big Data tools written in Java. God, what a mess that requires so much upkeeping, more dependency problems than I remember from C++ and probably some orders of magnitude more resources than they should.
- pjmlp 3y agoDoing code review for C++ code delivered by most well know offshoring companies, versus what they deliver in Java, will help get another point of view.
- intelVISA 3y agoThankfully LLMs are becoming a viable, cost-effective, option for stuffing your favorite offshored codebase with even more unmaintainable spaghetti. What a time to be alive.
- 3y ago
- formerly_proven 3y agoPython has many similar restrictions even in the language itself, they just tend to be exceptionally poorly documented. For example, you can't safely use print in a Python signal handler, because the entire I/O system is not reentrant [1], yet the documentation shows doing this in a "how to do it properly" example [2] while explaining that it's horribly unsafe to throw from a signal handler, which is the default for SIGINT. (And also the reason why the threading module doesn't expose PyThreadState_SetAsyncExc). So despite appearances to the contrary (since the Python signal handler is invoked by the VM, it doesn't actually run in a signal handler context; the signal module registers a C handler which simply sets a flag that is checked by the VM every instruction), you should probably only do in a Python signal handler what you would do in C, which is very little. Set a flag or write to a pipe. Don't go around and call library functions. [1] E.g. https://bugs.python.org/issue24283 https://bugs.python.org/issue24283 [2] https://docs.python.org/3/library/signal.html#note-on-signal-handlers-and-exceptions https://docs.python.org/3/library/signal.html#note-on-signal...
- paulddraper 3y agoNo-GIL mode is optional, and libraries will be marked "no-GIL compatible" and the ecosystem will gradually support more and more of these. No one's flipping a switch and breaking mountains of sketchy C.
- Steltek 3y agoWell someone has to say it so I might as well: Is this going to be another Python 2 -> 3 cat herding exercise again? I suppose the need to run your app in "no-GIL" mode is less than needing to jump from 2.7 to 3.
- n2d4 3y agoI didn't dive deep on this, but I assume that GIL mode can still run anything, including no-GIL code (it is one of their promises, at least). So, unlike 2->3, there is forwards-compatibility. It also seems like the latter isn't meant as a replacement (for the moment), but rather as an option.
- qbasic_forever 3y agoPresumably C extensions will have a different API name/ABI to prevent accidentally calling into GIL code when in non-GIL mode and vice/versa so that's going to complicate the compatibility story.
- ggm 3y agostat = rt_oldFunctionName(&newStateArg, oldarg[,...]) mostly (again from memory. I'd not live or die by this)
- paulddraper 3y agoAs sibling comments says, GIL mode can run no-GIL code. Entirely different from 2->3. That analogy is so poorly applicable it's laughable.
- 3y ago
- deleted 3y ago[deleted]
- winter_blue 3y agoI wonder why so many library developers even chose to build native libraries on the shaky and poorly-architected foundation that Python is. Even writing a JVM native JNI library would have allowed to avoid a lot of that pain (and the library would have been useable from Clojure, Kotlin , Scala, JRuby[1], Jython[2], Java, etc) without any painful threading issues. [1] which I’m aware of having been used in production by companies in the past [2] which I’m aware has been quite a bit under maintained for the last several years
- lanstin 3y agoBecause with the GIL it was dead simple to glue a C library into Python, and also was the canonical way to address hot inner loops in Python: rewrite into C. Nothing fancy but a little trial and error with module loading and you get 30% speed ups without being a great C programmer. I don’t think it’s false to say that the ease of moving hot spots into C is part of the reason Python has been so successful for thirty years. I hope this works, but I am very sceptical about being able to port code that worked with a locking solution provided for the enthusiast trying out C working without running into concurrency bugs.
- Cacti 3y agoMy understanding is the thing about the binding system that drove adoption more than anything was due to how easy it was to generate bindings for an existing C library. This drove stdlib expansion and allowed rapid repurpose of huge amounts of long-standing and popular C libraries, and it also added instant credibility for those that already trusted those libraries. This helped avoid the chicken and egg problem of not enough devs for a serious stdlib, but not serious enough stdlib to draw dev (which is where most languages, even good ones, die). I distinctly remember that during the time period where Python grew from “minor” to “dominate” (roughly 1995-2005), doing Python dev was often a process of answering “are there bindings for that?” And usually _there were_, because of that tooling.
- d0mine 3y agobtw, many C extensions release GIL already e.g., one Python thread can do regex work, another some number crunching, yet another waiting IO--all in parallel.
- weinzierl 3y agoI remember these warnings as well from when I started programming C seriously in the 90s. When I first encountered them I was convinced these issues will be resolved in a matter of weeks, maybe a couple of month at worst. Ohh, so little did I know.
- zrball10 3y agoI have a hard time seeing the equivalence here. In C the interaction with non thread safe functions is much more direct. Most people are also more cautious when writing in C. In Python you have whole C modules with global state. Load 10 of them, add the interpreter complexity and soon enough no one knows what is going on any more. As it is, most developers (including core devs!) don't even bother to check for memory leaks. I don't think they'll run tsan, and if they do, it will be on a small test suite that only covers 10% of the code. Given the software development practices in Python and especially in the AI space, I'm very pessimistic about this feature.
- dale_glass 3y agoThat can't stay forever. We're now up to 128 core CPUs, and even cheap CPUs have 6 cores on them. Restricting things to a single core's performance gets more and more limiting as time passes.
- westurner 3y agoWhile I'm happy to see optional GIL approved and happening, I also suspect that the GIL has saved us from debugging reentrant and/or dangerously concurrent code for years, and I salute the GIL for forcing us to build Arrow for IPC in Python, in particular. Someday, URI attributes in CPython docstrings might specify which functions are constant time, non re-entrant, functions. Reentrancy (computing): https://en.wikipedia.org/wiki/Reentrancy_(computing) https://en.wikipedia.org/wiki/Reentrancy_(computing) Global Interpreter Lock: https://en.wikipedia.org/wiki/Global_interpreter_lock https://en.wikipedia.org/wiki/Global_interpreter_lock
- pwdisswordfishc 3y agoWhat is Vax specific stack behaviour?