20 ms·
Project Loom and Structured Concurrency
- nfoz 6y agoThe article seems to assume you know what Project Loom is. (Not to be confused with Google's Project Loon, the balloon thing.) From https://wiki.openjdk.java.net/display/loom/Main https://wiki.openjdk.java.net/display/loom/Main, it's an OpenJDK project: > Project Loom is to intended to explore, incubate and deliver Java VM features and APIs built on top of them for the purpose of supporting easy-to-use, high-throughput lightweight concurrency and new programming models on the Java platform. A bit more history/explanation here: http://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.html http://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.h...
- technologia 6y agoThank you, I totally clicked on this thinking how could Google Loon have anything to do with this
- moks 6y agoalso discussed here https://news.ycombinator.com/item?id=23201559 https://news.ycombinator.com/item?id=23201559
- pbourke 6y agoTo summarize: up till now, Java Threads have been 1:1 with OS threads. They’re limited to a few thousand per JVM. This project moves to an M:N threading model but retains the Thread API. It allows for millions of threads per JVM and async/await style performance of the existing synchronous Java libraries without language changes and with minimal changes to the standard library.
- dboreham 6y agoThere was a before time when they weren't 1:1
- gpderetta 6y agoOriginally (about 20 years ago) java threads were M:N I think. How is this different? If I had to guess, they are not opaque to the VM which has more freedom to optimize them.
- simiones 6y agoThis is mentioned in the article: the old green threads model was not blocking aware - if a green thread scheduled on an OS thread issued a blocking call (say a listen on a socket) than the whole OS thread was blocked. With Project Loom, if a virtual thread executes a blocking call, the virtual thread is suspended and the OS thread is free to execute another virtual thread.
- anonymousDan 6y agoSo how does it actually execute the system call under the hood? Using some kind of background OS thread pool?
- Cojen 6y agoFrom what I can tell, async system calls are used whenever possible. A blocking call on a socket doesn't make a blocking system call, thus permitting the carrier OS thread to go do something else. As for file I/O, things are a bit messy. Older linux kernels don't support "true" async file I/O, and the Java NIO async file channels do use thread pools to emulate async behavior in that case.
- kasperni 6y agoGreen threads was only available on Solaris. They did not support more than 1 cpu. They were not particular particular performant either. And on top of that had issues with calling native code.
- kasperni 6y agoFor those that care about numbers: Loom targets ~ 200b memory overhead per virtual thread. And ~ 100ns per context switch between virtual threads.
- deleted 6y ago[deleted]
- deleted 6y ago[deleted]
- pron 6y ago> It does nothing for you if you have computationally intensive tasks and want to keep all processor cores busy. I would argue this isn't concurrency at all (the job of juggling mostly independent tasks, and scheduling them to a relatively small number of processing units), but parallelism (the job of performing a single computational task faster by employing multiple processing units), and exactly the job of parallel streams. > It doesn’t help you with user interfaces that use a single event thread. It might. Loom allows you to plug in your own scheduler, and it is a one-liner to schedule virtual threads on the UI thread: var uiVirtualThreadFactory = Thread.builder().virtual(java.awt.EventQueue::invokeLater).factory(); All threads created by this factory will be virtual threads that are always "carried" by the UI OS thread. This won't currently work because each of those threads will have its own identity, and various tests in the UI code check that the current thread is actually the UI thread and not any thread that is mapped to the same OS thread. Changing these tests is something the UI team is looking into.
- jayd16 6y ago>All threads created by this factory will be virtual threads that are always "carried" by the UI OS thread Does this actually solve the problem? I don't see it. We want to interweave foreground and background work. Sometimes that means blocking work will yield, sometimes that means it should not yield because conceptually several tasks should retain exclusive control of that thread. You might want some IO task on the background but you need a block of OpenGL tasks to retain control. I just don't see how you can do this implicitly in a way that's cleaner than async/await. It seems like posting tasks to this thread factory or that will get the job done but is that an improvement? It sounds like for now this stuff will still be using the current model of posting unyielding runnables to a thread. That's fine I guess. Loom still seems very cool, it just doesn't cover the cases I deal with a lot more often.
- pron 6y agoIt does (or will do, once the checks in the UI code are fixed) exactly what you want it to do. All computation will be done on the UI OS thread. All blocking operations will release it to do other things so it remains responsive. If you want to compute something outside it -- just spawn another thread that's not mapped to it.
- deleted 6y ago[deleted]
- lincpa 6y agoasync/await, Project Loom fiber, Warehouse/Workshop Model https://github.com/linpengcheng/PurefunctionPipelineDataflow#async-wait-fiber https://github.com/linpengcheng/PurefunctionPipelineDataflow...
- anonymousDan 6y agoI don't quite get the point of the executor service with virtual threads. If they are really cheap to create then why not just create them as required? It's been a while since I programmed in Java though, am I missing something? Edit: Ah - I read the rest of the article. Using it as a synchronisation primitive makes sense I guess, if a bit clunky.
- kasperni 6y agoIn the examples with structured concurrency. The point of using an executor service is not to reuse the threads. But instead to control their termination. If you read Nathaniel J. Smith's primer [1] on structured concurrency. The ExecutorService in the examples act as the nursery. Loom is just being "lazy" and reusing ExecutorService for something that it wasn't originally intended to do. Earlier versions had a specific class called FiberScope [2]. Whether or not we will see more specialised classes for this in the future I don't know. [1] https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ https://vorpus.org/blog/notes-on-structured-concurrency-or-g... [2] https://www.javaadvent.com/2019/12/project-loom.html https://www.javaadvent.com/2019/12/project-loom.html
- jnwatson 6y agoIt is an unfortunate abuse of an existing facility for something quite different.
- pron 6y agoThe API is not yet final, but I wonder why you think it's "an abuse of an existing facility for something quite different," and whether this could just be a matter of a habit rather than purpose. Take a look at the description of ExecutorService (https://docs.oracle.com/en/java/javase/15/docs/api/java.base/java/util/concurrent/ExecutorService.html https://docs.oracle.com/en/java/javase/15/docs/api/java.base...) and Executors (https://docs.oracle.com/en/java/javase/15/docs/api/java.base/java/util/concurrent/Executors.html https://docs.oracle.com/en/java/javase/15/docs/api/java.base...). I think we're using them exactly as intended: those are APIs meant to manage and control task's execution and lifetime. I'm interested to hear why this feels wrong to you.
- jillesvangurp 6y agoSince the article goes out of its way to not mention Kotlin, I'll do it for them since this is both lame and more than a bit disingenuous. Arguably, Kotlin co-routines (and the Flow API) provides a very nice implementation of the exact same concepts on the JVM. As far as I know, the loom integration is already planned and probably implemented to a large degree. Mostly doing that should be straightforward as this pretty much just maps 1 to 1 to things like suspend functions, co-routine scopes, etc. That is a different way of saying that Oracle is doing the right things with Loom. Although bolting this onto the Thread API without cleaning that up is probably an open invitation for hordes of people to do the wrong things. That API already provides plenty of well documented ways to take shots at your feet. IMHO it's a mistake to pretend it's all the same. The main difference with Kotlin co-routines is that the Kotlin implementation is multiplatform and also has implementations that work on IOS (native), in a browser etc. Additionally, you get to depend on nice language features like internal DSL support, the suspend keyword, etc, that make writing code a lot less tedious and error prone. But it's the same kind of code with the same kind of concepts. Finally, it also has lots of higher level primitives. Flow is a recent addition that allows for building properly reactive applications that sits on top of this. So, to answer the obvious question will this replace/deprecate co-routines: no, this will have little to no impact as it will be trivial to support the low level primitives Loom provides just like they already work seamlessly with other implementations like rxjava, spring reactor, javascript promises, etc. They'll support it because it probably provides some performance benefits to use Loom if it's available on the platform but it should not impact how you use co-routines. The same co-routine code you write today will just work on top of Loom once that is available and implemented.
- vbezhenar 6y agoAs I see it, the main difference is that existing code will magically work with Loom and will require rewrite with coroutines. I can get Oracle 9i JDBC driver from 1999 and use it under Loom and it probably will work just fine. Probably Oracle will not rewrite its JDBC drivers with Kotlin coroutines any time soon.
- jillesvangurp 6y ago
- anonymousDan 6y agoSounds like a useful building block for a better java-based actor library.
- Traubenfuchs 6y agoI am still not comfortable enough with this concept to answer this question myself, but will this, by default, lead to speed ups and/or reduced resource consumption in a) application server like tomcat and b) web frameworks like Spring? Assuming it's implemented...
- amitport 6y agoProbably no, but that's not the point. These server frameworks have complex code that helps balance load across threads. Loom may make such programs simpler to write, but will not automatically give a boost to already optimized code.
- aardvark179 6y agoIt depends a lot on how the service handles requests. If it takes the one thread per request model, and those requests are mostly bound by blocking calls like IO, then replacing those OS level threads with virtual threads will almost certainly see a reduction in resources (as virtual threads are smaller) and potentially more consistent response times (because scheduling the correct thread is easier at the JVM level). However if your service has been written in an async style, or you are mostly CPU bound, then you aren't likely to see a change. Our hope is that by making simple blocking code perform better you won't have to spend your time converting code to an async style to scale your services.
- nsaje 6y agoIf I understand correctly this is to Java what Gevent/Eventlet are to Python. Asynchronous execution without having to write asynchronous code. I've been doing this in Python for a long time and I love it, even though it can lead to some hard-to-debug issues (due to monkeypatching).
- jnwatson 6y agoI have a lot of experience using concurrency in Go, and for the last couple years have been at the bleeding edge of Python async. The tradeoffs between the two approaches are immense. With the virtual thread model you have: * No function coloring problem. This also means existing code is easier to port. * possibility of transparent M:N scheduling. * Impedence mismatch with OS primitives. * Much more sophisticated runtime. * Problematic task cancellation. * Lots of care still needed for non-trivial inter-task synchronization. With the async API model you have: * Viral asyncification (the method color problem). * Simpler runtime. * Obvious and safe task cancellation. * Completely orthogonal to parallelism (actually doing more than one thing simultaneously) for good and for bad. * Inter-task coordination is straightforward and low-overhead even for sophisticated use cases. * Higher initial learning curve. I'm leaning toward liking the async approach more, but that might be just because I'm deep in the middle of it. I think the biggest argument in favor of virtual threads is the automatic parallelism; that's also the biggest argument against: free running threads require more expensive synchronization and introduce nondeterminism.
- pron 6y ago* Java offers both user-mode and kernel threads. You pick at creation, and can even plug your own scheduler. * Loom's virtual threads are completely scheduled in library code, written in Java. * FFI that bypasses the JDK and interacts with native code that does either IO or OS-thread synchronization is extremely rare in Java. * Cancellation is the same for both. Also, IMO, coordination is simpler for threads than for async. Where they differ is in their choice of defaults: thread allow scheduling points anywhere except where explicitly excluded; async/await allows scheduling points nowhere except where explicitly allowed. Putting aside that some languages have both, resulting in few if any guarantees, threads' defaults are better for correct concurrency. The reason is that correctness relies on atomicity, or lack of scheduling points in critical sections. When you explicitly exclude them, none of your callees can break your correctness. When you explicitly allow them, any callee can become async and break its caller's logic. True, the type system will show you where the relevant callsites are, but it will not show you whether there is a reliance on atomicity or not. Async/await does, however, make sense for JavaScript, where all existing code already has an implicit assumption of atomicity, so breaking it would have broken the world. For languages that have both, async/await mostly adds a lot of complexity, although sometimes it is needed for implementation reasons.
- mping 6y agoFor me the real advantage is not on performance but on the programming model. I have been tinkering with Loom (and clojure) and the idea of "just" calling some library without worrying about blocking is refreshing. That means that for the most of it, you can write your code without worrying too much about some kind of callbacks or async support from your library and it just works. Of course, for those with extreme performance requirements, they will probably have their own custom scheduler and concurrency/parallelism mechanisms but for the vast majority of jvm users out there I think Loom will be a great thing. If Loom integrates with GraalVM/native-image it would be even nicer.
- lackbeard 6y agoI think the vast majority of JVM users won't even need Loom. OS threads perform well enough for most use cases. You can go a very long way with just a ThreadPoolExecutor.
- closeparen 6y agoNo one “needs” Loom; you can always write in callback oriented style. The point is it will free you from that.
- dangerboysteve 6y agotimley Episode 8 “Project Loom” with Ron Pressler https://inside.java/2020/11/24/podcast-008/ https://inside.java/2020/11/24/podcast-008/
- ljackman 6y ago`Async`/`await` or something like Kotlin's `suspend` are great language features for certain domains in which a developer needs to manage blocking system calls: in lower-level languages such as Rust or C, you probably don't want to pay for a lightweight "task runtime" Like Go's or Erlang's. They bring not only a scheduling overhead but also FFI complications. However, for application languages that can afford a few extra nicities like garbage collection, I fail to understand why the stackless coroutine model (`suspend` in Kotlin) or `async`/`await` continue to be the developer's choice. Why do languages like Kotlin adopt these features, specifically? Manually deciding where to yield in order to avoid blocking a kernel thread seems outside of the domain of problems that those using a _higher level_ language want to solve, surely? The caller should decide whether to do something "in the background". And this applies to non-IO capabilities too, as sometimes pure computations are also expensive enough to warrant not blocking the current task. Go and Erlang seem to have nailed this, so I'm glad Java is following in their footsteps rather than the more questionnable strategy of C# and Kotlin. (Lua's coroutines and Scheme's `call-with-current-continuation` deserve an honourable mention too.)
- geodel 6y agoKotlin runs on JVM so if JVM does not support something natively Kotlin can't have that feature like task runtime.
- jpgvm 6y agoAdditionally Kotlin also targets native. `suspend` in Kotlin was very much designed with this in mind as it's easier to implement than something that requires an extensive runtime like Loom. Kotlin will still support Loom on JVM and there will likely be integration with suspend/flows etc also.
- ljackman 6y agoThat's a good point. Generally, I opt for languages that either compile away their nicities to avoid runtime hits, such as Rust being compiled to WASM, or languages that bring nicities in runtimes that they _completely own_, such as Java on the JVM. The problem with Kotlin and the like is that they can't easily compile away their features due to inherent runtime dependencies, e.g. garbage collection, making them poorly suited to environments with a very minimal runtime like WASM, while also being at the mercy of the host language creating runtime abstractions that have mismatches with their own language's features. Although it'd be unfair for me to say the JVM is designed only for Java; invokedynamic and non-reified generics both assist JVM targetting for non-Java languages such as Clojure.
- latchkey 6y agoProgramming challenge in golang: Create a persistent tcp client that can connect to a server, read responses… and be disconnected via context.WithCancel().
- Cojen 6y agoI'm not familiar enough with Go to understand why this is a challenge. What point are you trying to illustrate with the challenge? Is this easy or hard, and how does this compare to Loom and Structured Concurrency?
- latchkey 6y agoMany people in this thread are talking about structured concurrency in other languages. I brought up a fun one for golang that, as far as I can tell, is a 3 year old open issue [0] that I've recently bumped into. Maybe this is a fun challenge for you to become more familiar with golang? [0] https://github.com/golang/go/issues/20280 https://github.com/golang/go/issues/20280