8 ms·
Notes on Virtual Threads and Clojure
- dgb23 4y agoI'm hopeful for first class support in core.async. JVM support for virtual threads is the last missing piece IMO.
- Skinney 4y agoWhen JVM lands support for virtual threads, what reasons remain for using core.async? core.async will still be useful in ClojureScript land, though.
- dgb23 4y agoChannels & alts etc. are useful constructs regardless right?
- Skinney 4y agoSure, but do these have API's that work with regular threads?
- cr__ 4y agoYep. They’re decoupled completely from the “go” bits in core.async that are meant to make up for the lack of green threads. All of the channel operations have blocking variants that work fine with regular threads now, and they seem likely to work fine with fibers.
- Skinney 4y agoDidn't know that, that's great!
- pgt 4y agoDoes that mean we can use core.async channels without the `go` macro? How would this look?
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- bcrosby95 4y agoI don't know if you mean something deeper, but core.async has <!, >!, and alts! variants that don't require a go block: <!!, >!!, and alts!!. You can just use them like a function. But they block the thread. So if you e.g. naively call it in the REPL you will block it indefinitely. E.g.: (>!! (chan) "this will block the REPL") The idea would be to pass a channel to multiple virtual threads which use the blocking versions.
- didibus 4y agoYes you can. Core.async already supports using normal threads with the `thread` macro instead of using the `go` macro. All code looks exactly the same otherwise.
- bcrosby95 4y agoAlthough it doesn't seem to have a nice API for doing so, it seems like you can change the core.async executor by using alter-var-root. https://stackoverflow.com/questions/18779296/clojure-core-async-any-way-to-control-number-of-threads-in-that-go-thread https://stackoverflow.com/questions/18779296/clojure-core-as...
- Copenjin 4y agoFor those wondering about the "tail call" part that would be more interesting for Clojure than fibers: As adding the ability to manipulate call stacks to the JVM will undoubtedly be required, it is also the goal of this project to add an even lighter-weight construct that will allow unwinding the stack to some point and then invoke a method with given arguments (basically, a generalization of efficient tail-calls). We will call that feature unwind-and-invoke, or UAI. It is not the goal of this project to add an automatic tail-call optimization to the JVM. [1] https://cr.openjdk.java.net/~rpressler/loom/Loom-Proposal.html https://cr.openjdk.java.net/~rpressler/loom/Loom-Proposal.ht...
- gorjusborg 4y ago> It is not the goal of this project to add an automatic tail-call optimization to the JVM. Wait what? Are we getting TCO or not with Loom?
- joe_fishfish 4y agoTail-call optimisation on the JVM is already somewhat supported with an annotation / keyword (depending on language). My understanding is that the project will improve the current implementation, but you'll still need to define when a function is tail-recursive, the compiler won't do it automatically.
- funcDropShadow 4y agoE.g. Scala provides such an annotation, but that is implemented by rewriting the recursive method to a non-recursive method with a loop.
- geokon 4y agoSo maybe a more basic and fundamental question - but why can't OS threads be as good as these virtual threads? Why do we have to have two layers of thread abstraction? My first reaction is "eww" quickly followed by "I'm prolly not understanding something here". Why can't we make the OS handle thousands of threads?
- dboreham 4y agoOS can handle thousands of threads easily. Kernel threading runs into scaling problems up at the 100's of thousands to millions level.
- sidpatil 4y agoMy understanding is that OS thread operations require the processor/OS to switch between user mode and kernel mode, which is slow. Virtual threads don't require this, since they are managed entirely by the user process.
- belter 4y agoCorrect. This is the main reason.
- renox 4y agoThey could: Solaris had M:N threads at some point in time but 1) they're complicated 2) you pay a performance cost even if you don't use many threads..
- pjmlp 4y agoCalled LWT, but they were eventually dropped exactly for those reasons.
- Blackthorn 4y agoWe can, this was inspired by a talk given by Google about their own infrastructure. But it was apparently never open sourced.
- 4y ago
- dboreham 4y agoQuick note that this: > Each thread easily uses an additional Megabytes of memory isn't quite the problem it seems because unless the thread actually _uses_ large amounts of stack, the memory consumed is mostly virtual. With today's 64-bit architectures, there is plenty of virtual address space to be burned.
- spacechild1 4y agoYes! I can't count how often I've read this when people talk about OS threads...
- funcDropShadow 4y agoBut it requires changing the virtual memory mapping tables, i.e. it requires a context switch to the kernel and back. Therefore it decreases effectiveness of TLAB caches. I.e. there is a performance cost. One that is hard to quantify for very short lived threads and only slightly better quantifiable for longer lived threads. And every argument that OS threads are good enough is in stark contrast to the length to which people go to avoid using them. I am talking about all the async libraries/patterns/language features people use to avoid using OS-level threads.
- native_samples 4y agoWell not quite. The problem is that it only takes one call tree with a deep stack to cause all that memory to be allocated (slowly, using lots of page faults) and after that, it's used. The only way to get rid of it again is to swap it out, which means if you then go ahead and do another deep call it's even slower as the kernel doesn't know what it can discard and what it can't. Deep stacks are pretty common in modern software especially for Java software.
- didibus 4y agoI'm a little confused about the "structured concurency" example? I don't understand what the advantages of it are? Are Virtual Threads resources that need to be explicitly cleaned up? Can't the pool grow and shrink based on demand? In Go for example, I don't remember having to clean up fibers manually. For example, how is the structured concurency example different from: (defn run-concurrently [] (let [f1 (future (identity 2000)) f2 (future (prn "Starting a long running operation")) f3 (future (Thread/sleep 1000)) f4 (future (prn "Done."))] (run! deref [f1 f2 f3 f4]) 4)) (run-concurrently) Also as an FYI, the example can be simplified by using `with-open`.
- Matthias247 4y agoThe advantage is that it provides guarantees, that wouldn't be there otherwise. Let's for example say one of those 4 tasks would write to a file. With structured concurrency you have a guarantee that once the supertask finishes, the file is no longer in use. Without structured concurrency you won't have that, and a subtask might error on trying to use the file or find garbage data in it. Running the supertask in a loop with structured concurrency would have no risk of running into resource contention, while without it there is that risk. Plus structured concurrency also helps adding a deterministic bound on the amount of concurrency in the system, while just spawning more background tasks can lead to concurrency grow in an unbounded fashion. > In Go for example, I don't remember having to clean up fibers manually. The equivalent in Go would be to wait for subtasks to complete, using the WaitGroup pattern. And to notify the subtasks that they should complete using "Context".
- didibus 4y agoMy code waits for all futures to complete, that's what: (run! deref [f1 f2 f3 f4]) does.
- comma_at 4y agoAt a bare minimum your code might have different behavior. Your `run! deref` will wait for each future one by one. What if the last one threw an exception while the others are running? You'd probably want the other tasks to get canceled immediately. I'm not sure what Executor.close() does exactly, but in theory it could do this for you automatically. Not something I want to write by hand for sure[1]. > In Go for example, I don't remember having to clean up fibers manually. And that is awful if you think about it a bit. There's no composition happening, your go block just goes off somewhere without you having any control of it. It's like goto, only it forks first and never gives back the process handle[2]. [1] https://clojureverse.org/t/missionary-new-release-with-streaming-support-design-notes/4510/4 https://clojureverse.org/t/missionary-new-release-with-strea... [2] https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ https://vorpus.org/blog/notes-on-structured-concurrency-or-g...
- jp57 4y agoI gotta say, as a programmer "of a certain age", a lot of recent advances have an everything-old-is-new-again feel to me. When I first heard of "threads" in the 90s, the term specifically referred to a mechanism for intra-process concurrency that was "lightweight", i.e. it didn't have the heavy overhead of forking a new OS process. IIRC they weren't even preemptively multitasked. Then preemptive "OS threads" became the new hotness and everything using cooperative threads was old and busted. Now a generation has passed and we have "virtual threads" or things like asyncio to solve the problems created by the thing that solved the problems of the thing before. A similar thing has happened at the OS level with support for multiple programs. Computers started out running one program at a time. Then "multitasking" was invented, and along with it all kinds of things to make it safer and easier like users and permissions and protected memory, and it was great, many different users could run different programs on the same machine. Then we moved from running statically linked binary executables to dynamic linking and interpreted languages that load libraries at run time. Managing environments for different programs on one machine got hard. VMs and hypervisors let us segment a machine into virtual machines where the environments were easy to control. But VMs were a heavyweight solution for managing environments, so along came containers, which are kind of virtual virtual machines. So now we're in the position where it's perfectly normal to run a program with several virtual threads running in a "real" thread in a Java virtual machine running inside a Docker container running on a VM instance in a hypervisor on some giant box somewhere. And if we're all living in a simulation, then its' probably turtles all the way down. I don't mean this entirely as a "get off my lawn" rant. I recently re-implemented an io-heavy threaded program using asyncio and I found it genuinely better. It was easier to reason about what was happening and to get our task dependencies to execute in the right order. The code is simpler and I'm pretty happy about it. But I fully expect asyncio and virtual threads to go out of fashion in ten years, and some new preemptive, OS-managed thing will come along and become the new hotness. ps - I found this quote in the description of Virtual Threads[1] > Virtual threads are preemptive, not cooperative — they do not have an explicit await operation at scheduling (task-switching) points. Rather, they are preempted when they block on I/O or synchronization. Sorry, that's not preemption, that's yielding at specific safe points. I.e. cooperative multitasking. For example, Classic MacOS's cooperative multitask didn't have an explicit yield, it yielded automatically when you polled for a new event. [1] - http://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.html#right-sizing-threads http://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.h...