3 ms·

"Yes, the threads currently are not usable for number crunching in the background. And yes, there are bugs, and trying to do many things from the background thr

by jerf 3y ago

"Yes, the threads currently are not usable for number crunching in the background. And yes, there are bugs, and trying to do many things from the background thread doesn't work, sometimes in unexpected ways. You can still block the main thread from the background thread since some things block the event loop, no matter where they were started." Many dynamic languages have bodged on threading over the past decade. (I don't think dynamic languages are intrinsically unthreadable or anything, but their interpreters were pretty deeply based on not having threads.) What that has shown is that 90%-effective threading is useless, and 99%-effective threading is superficially appealing but always, always blows up at any sort of scale. You really need threads that don't come with all those caveats. I expect it will get there, but another thing we've learned from previous efforts is that telling the community it's ready before it's ready causes "$LANGUAGE threading" searches to be filled with posts telling people how bad it is, even years and years after it has actually been fixed. It's probably a blessing in disguise it's not something the community is pervasively trying to use.

klibertp 3y ago
Well, threading - shared-state parallelism, more precisely - is hard to do well, and retrofitting it into a program that wasn't designed with that kind of threading in mind is even more challenging. I don't think any popular languages solved this, save for Java. Especially in the recent versions, with the new virtual threads - as much as I dislike Java, I have to say they did an excellent job on this. Other languages and platforms (that I know of; what's the .NET story here?) are all shitshows to varying degrees, trying to catch up and failing over and over again. I think the only sensible way to offer parallelism in Emacs is to exclude the "shared-state" part, the way Racket (places) and OCaml do it. I think Python also tries to do it with subinterpreters? Let another instance of the interpreter run in the same process and communicate via message passing. That's probably still a huge undertaking, but at least it seems more viable than going through the whole codebase and adding locks everywhere... Still, the "threads" in Emacs, as incomplete and half-baked as they are, can be useful. And if nobody uses them, there's no incentive for the developers to improve them. So I think we need at least some early adopters if we want the threading support in Emacs to get better.
- neonsunset 3y ago
  On concurrency, Java is the last to the party. Many languages had solved it in various ways before: - async/await camp with C# and F#, Swift, Rust, Python, TS/JS (yes, I know, event loop) - Erlang/Elixir and BEAM family - virtual threads and/or coroutines - Go, Java, Kotlin Generally, I don't feel like in 2023 concurrency and parallelism are problematic areas anymore aside from existing aged stacks. From a perspective of mainly .NET ecosystem resident, it has been a shitshow outside of it for a long time indeed with many architectural choices throughout the industry paying for the Java sins (e.g. Kafka) and imposing limitations that seemed nonsensical and embarrassing even 7 years ago.
  - klibertp 3y ago
    We're talking about parallelism here, not concurrency. async/await solve concurrency, not parallelism (on their own). Kotlin coroutines solve parallelism only because they piggyback on Java threads. I'm not sure about Go, but it's probably M:N concurrency (so with parallelism) like what you get on the BEAM. Then again, on the BEAM you don't get to "share" anything (other than binaries, IIRC). I'd say concurrency is largely a solved problem, yes; limited parallelism (e.g., with message passing) also mostly works. We don't need to worry about the "C10K problem" anymore. But shared (mutable) state parallelism is, I think, still far from solved - if it can ever be "solved", which is a pretty big assumption :)

by jerf 3y ago

klibertp 3y ago

Well, threading - shared-state parallelism, more precisely - is hard to do well, and retrofitting it into a program that wasn't designed with that kind of threading in mind is even more challenging. I don't think any popular languages solved this, save for Java. Especially in the recent versions, with the new virtual threads - as much as I dislike Java, I have to say they did an excellent job on this. Other languages and platforms (that I know of; what's the .NET story here?) are all shitshows to varying degrees, trying to catch up and failing over and over again. I think the only sensible way to offer parallelism in Emacs is to exclude the "shared-state" part, the way Racket (places) and OCaml do it. I think Python also tries to do it with subinterpreters? Let another instance of the interpreter run in the same process and communicate via message passing. That's probably still a huge undertaking, but at least it seems more viable than going through the whole codebase and adding locks everywhere... Still, the "threads" in Emacs, as incomplete and half-baked as they are, can be useful. And if nobody uses them, there's no incentive for the developers to improve them. So I think we need at least some early adopters if we want the threading support in Emacs to get better.

neonsunset 3y ago
On concurrency, Java is the last to the party. Many languages had solved it in various ways before: - async/await camp with C# and F#, Swift, Rust, Python, TS/JS (yes, I know, event loop) - Erlang/Elixir and BEAM family - virtual threads and/or coroutines - Go, Java, Kotlin Generally, I don't feel like in 2023 concurrency and parallelism are problematic areas anymore aside from existing aged stacks. From a perspective of mainly .NET ecosystem resident, it has been a shitshow outside of it for a long time indeed with many architectural choices throughout the industry paying for the Java sins (e.g. Kafka) and imposing limitations that seemed nonsensical and embarrassing even 7 years ago.
- klibertp 3y ago
  We're talking about parallelism here, not concurrency. async/await solve concurrency, not parallelism (on their own). Kotlin coroutines solve parallelism only because they piggyback on Java threads. I'm not sure about Go, but it's probably M:N concurrency (so with parallelism) like what you get on the BEAM. Then again, on the BEAM you don't get to "share" anything (other than binaries, IIRC). I'd say concurrency is largely a solved problem, yes; limited parallelism (e.g., with message passing) also mostly works. We don't need to worry about the "C10K problem" anymore. But shared (mutable) state parallelism is, I think, still far from solved - if it can ever be "solved", which is a pretty big assumption :)