8 ms·
Loom can't land fast enough! The current issue the JVM has is that all threads have a corresponding operating system thread. That, unfortunately, is really he
by cogman10 5y ago
Loom can't land fast enough!
The current issue the JVM has is that all threads have a corresponding operating system thread. That, unfortunately, is really heavy memory wise and on the OS context switcher.
Loom allows java to have threads as light weight as a goroutine. It's going to change the way everything works. You might still have a dedicated CPU bound thread pool (the common fork join pool exists and probably should be used for that). But otherwise, you'll just spin up virtual threads and do away with all the consternation over how to manage thread pools and what a thread pool should be used for.
- bestinterest 5y agoWhats the difference between goroutines and project loom? Is their any?
- cogman10 5y agoTerminology mostly :D I've not looked into the goroutine implementation, so I couldn't tell you how it compares to what I've read loom is doing. Loom is looking to have some extremely compact stacks which means each new "virtual thread" as they are calling them will end up having bytes worth of memory allocated. Another thing coming with loom that go lacks is "structured concurrency". It's the notion that you might have a group of tasks that need to finish before moving on from a method (rather than needing to worry about firing and forgetting causing odd things to happen at odd times).
- jayd16 5y ago>structured concurrency That's good to hear. You see a lot of these Loom discussions talk about implicit and magical asynchronous execution. I was afraid fine grained thread control would be left out. Its super useful if you want to interface with how most GUI frameworks function (ie a Main thread), or important OS threads like a thread with a bound GL context or what have you.
- cogman10 5y agoYeah, while virtual threads are the bread and butter of Loom, they are also adding a lot of QoL things. In particular, the notion of "ScopedVariables" will be a godsend to a lot of concurrent work I do. It's the notion of "I want this bit of context to be carried through from one thread of execution to the next". Beyond that, one thing the loom authors have suggested is that when you want to limit concurrency the better way to do that is using concurrency constructs like semaphores rather than relying on a fixed pool size.
- ccday 5y agoNot sure if it counts as structured concurrency but Go has the feature you describe: https://gobyexample.com/waitgroups https://gobyexample.com/waitgroups
- jayd16 5y agoThe biggest difference is probably that the JVM will support both OS and lightweight threads. That's really useful for certain things talking to the GPU in a single thread context.
- _old_dude_ 5y agoUnlike go routine, Loom virtual threads are not preempted by the scheduler. I believe you may be able to explicitly preempt a virtual thread but the last time i checked it was not part of the public API
- vips7L 5y agoUnless I'm misunderstanding, virtual threads are preemptive: https://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.html https://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1....
- _old_dude_ 5y agoBy the OS, not by the scheduler see https://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part2.html#forced-preemption https://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part2....
- vips7L 5y agoWhat about pron's comments here then? https://news.ycombinator.com/item?id=27885569 https://news.ycombinator.com/item?id=27885569 > Second, Loom's virtual threads can also be forcibly preempted by the scheduler at any safepoint to implement time sharing
- _old_dude_ 5y agoFor me, preemption by the Java scheduler is not currently supported but may be added in the future, after all the goroutine were not preempted at the beginning in Go. The whole quote > Second, Loom's virtual threads can also be forcibly preempted by the scheduler at any safepoint to implement time sharing. Currently, this capability isn't exposed because we're yet to find a use-case for it I believe it's a reference to [1] but i may be wrong. [1] https://download.java.net/java/early_access/loom/docs/api/java.base/java/lang/Continuation.html#tryPreempt(java.lang.Thread) https://download.java.net/java/early_access/loom/docs/api/ja...
- ovis 5y agoWhat benefits does loom provide vs using something like cats-effect fibres?
- _old_dude_ 5y agoYou can actually debug the code you write because you get a real stacktrace, not few frames that shows the underlying implementation.
- clhodapp 5y agoAdmittedly, loom will do much better but cats-effect does try its best within the limitations of the current JVM: https://typelevel.org/cats-effect/docs/2.x/guides/tracing https://typelevel.org/cats-effect/docs/2.x/guides/tracing
- Nullabillity 5y agoOn the other hand, you'll spend a lot more time debugging Loom code, because it reuses the same broken-by-design thread API.
- elygre 5y agoWhat is broken-by-design about the api?
- Nullabillity 5y agoFundamentally, an async API is either data-oriented (Futures/Promises: tell me what data this task produced) or job-oriented (Threads: tell me when this task is done). You can think of it like functions vs subroutines. Since you typically care about the data produced by the task, threads require you to sort out your own backchannel for communicating this data back (such as: a channel, a mutexed variable, or something else). Unscientifically speaking, getting this backchannel wrong is the source of ~99% of multithreading bugs, and they are a huge pain to fix. You can implement futures on top of threads by using a thread + oneshot channel, but that requires that you know about it, and keep them coupled. The point of futures is that this becomes the default correct-by-default API, unless someone goes out of their way to do it some other way. On the other hand, implementing threads on top of futures is trivial: just return an empty token value. There are also some performance implications: depending on your runtime it might be able to detect that future A is only used by future B, and fuse them into one scheduling unit. This becomes harder when the channels are decoupled from the scheduling.
- christkv 5y agoAre we coming full circle going back a variant of the original Java green threads?
- hashmash 5y agoNot quite. The original green threads were seen as more of a hack until Solaris supported true threads. Green threads could only support one CPU core, and so without a major redesign, it was a dead end.
- AtlasBarfed 5y agoBasically yes. Longer answer: devs back in the day couldn't really grok the difference between green and real threads. Java made its bones as an enterprise language, which can have smart programmers, but they will conversely not be closer-to-metal knowledgewise. Too many devs back in the day expected a java thread to be a real thread, so java re-engineered to accomodate this. I think the JDK/JVM teams also viewed it as a maturation of the JVM to be directly using OS resources so closely across platforms, rather than "hacking" it with green threads. These days, our high performance fanciness means the devs are demanding green thread analogues, and go/elixir/others are seemingly superior because of those. So to remain competitive in the marketplace, Java now needs threads that aren't threads even though Java used to have threads that weren't threads.
- cogman10 5y agoYes and no. The new Loom threads will be much lighter weight than the original Java green threads. Further, the entire IO infrastructure of the JVM is being reworked for Loom to make sure the OS doesn't block the VM's thread. What's more, Loom does M:N threading. Same concept, very different implementation.
- iamcreasy 5y agoSo, with Loom now we can tell exactly in which order theses threads were executed as it's not up to OS to decide thread execution order anymore?
- Spivak 5y agoYou are ignoring the downside to green threads which is that it’s cooperative. If the thread doesn’t yield control back to the event loop then the real OS thread backing the loop is now stuck. Which leads to dirty things like inserting sleep 0 at the top of loops and dealing with really unbalanced scheduling of threads don’t hit yields often enough. Plus with loom it might not be obvious that some function is a yield since it’s meant to be transparent so if you grab a lock and yield you make everyone wait until your scheduled again. Green threads are great! I love them and they’re the only real solutions to really concurrent IO heavy workloads but it’s not a panacea and trades one kind of discipline for another.
- sudhirj 5y agoSleep 0 sounds like quite a hack, Go has the neater https://pkg.go.dev/runtime#Gosched https://pkg.go.dev/runtime#Gosched instead, and I assume there will be a Java equivalent as well. And if most stdlib methods and all blocking methods call it, it's going to be pretty difficult to hang a green thread.
- WatchDog 5y agoFWIW, Java has had `Thread#yield()`[0] since inception. [0]: https://docs.oracle.com/javase/7/docs/api/java/lang/Thread.html#yield https://docs.oracle.com/javase/7/docs/api/java/lang/Thread.h...()
- kaba0 5y agoSince there is a runtime that knows everything about the state of the thread, my understanding is that there is no need for explicit yields. Everything will turn automagically into non-blocking (except for FFI)
- brokencode 5y agoI was under the impression that Loom was implementing preemptable lightweight threads. Is that not the case?
- clhodapp 5y ago
- jeffbee 5y agoAre you quite certain that a (linux, nptl) thread costs more memory than a goroutine? You've implied that but it's not obviously true.
- dragontamer 5y agoWouldn't any linux/nptl thread require at at least the register-state of the entire x86 (or ARM) CPU? I don't think goroutines would need such information. A goroutine knows that "int foobar;" is currently being stored in "rbx", and that "int foobar" is currently saved on the stack. Therefore, rbx doesn't need to be saved. ------ Linux/NPTL threads don't know when they are interrupted. So all register state (including AVX512 state if those are being used) needs to be saved. AVX512 x 32 is 2kB alone. Even if AVX512 isn't being used by a thread (Linux detects all AVX512 registers to be all-zero), RAX through R15 is 128-bytes, plus SSE-registers (another 128-bytes) or ~256 bytes of space that the goroutines don't need. Plus whatever other process-specific information needs to be saved off (CPU time and other such process / thread details that Linux needs to decide which threads to process next)
- jeffbee 5y agoI don't think the question is dominated by machine state, I think it would be more of a question of stack size. They are demand-paged and 4k by default for native threads, 2k by default for goroutines but stored on a GC'd heap that defaults to 100% overhead, so it sounds like a wash to me.
- dragontamer 5y agoHmmm. It seems like you're taking this from a perspective of "Pthreads in C++ vs Coroutines in Go", which is correct in some respects, but different from how I was taking the discussion. I guess I was taking it from a perspective of "pthreads in C++ vs Go-like coroutines reimplemented in C++", which would be pthreads vs C++20 coroutines. (Or really: it seems like this "Loom" discussion is more of a Java thing but probably a close analog to the PThreads in C++ vs C++20 Coroutines) I agree with you that that the garbage collector overhead is a big deal in practice. But its an aspect of the discussion I was purposefully avoiding. But I'm also not the person you responded to.
- cbsmith 5y ago> That, unfortunately, is really heavy memory wise and on the OS context switcher. So, there was a time where a broad statement like that was pretty solid. These days, I don't think so. The default stack size (on 64-bit Linux) is 1MB, and you can manipulate that to be smaller if you want. That's also the virtual memory. The actually memory usage depends on your application. There was a time where 1MB was a lot of memory, but these days, for a lot of contexts, it's kind of peanuts unless you have literally millions of threads (and even then...). Yes, you can be more memory efficient, but it wouldn't necessarily help that much. Similarly, at least in the case of blocking IO (which is normally why you'd have so many threads), the overhead on the OS context switcher isn't necessarily that significant, as most threads will be blocked at any given time, and you're already going to have a context switch from the kernel to userspace. Depending on circumstance, using polling IO models can lead to more context switching, not less. There's certainly circumstances where threads significantly impede your application's efficiency, but if you are really in that situation you likely already know it. In the broad set of use cases though, switching from a thread-based concurrency model to something else isn't going to be the big win people think it will be.
- user5994461 5y ago>>> The default stack size (on 64-bit Linux) is 1MB The default thread stack size is 8 or 10 MB on most Linux. The exception is Alpine that's below 1 MB.
- ori_b 5y agoThe default reserved size is 8mb. The allocated size starts at a page (usually 4k), and grows in page sized increments as you use it.
- cbsmith 5y agoTo clarify, the 1MB is the default stack size for threads with the JVM on 64-bit Linux. Search for "-Xss": https://docs.oracle.com/en/java/javase/16/docs/specs/man/java.html https://docs.oracle.com/en/java/javase/16/docs/specs/man/jav...
- kllrnohj 5y ago
- lmilcin 5y agoI have discovered ReactiveX for Java and Reactor in particular. I am working with Kafka and MongoDB and it is normal for my app to have a million in flight transactions at various stages of completion. In the past it required a lot of planning (and a lot of code) but Reactor let's me build these processes as pipelines with whatever concurrency or scheduler I desire, at any stage of the processing. We are even doing tricks like merging unrelated queries to MongoDB so that sometimes thousands of same queries are executed together (one query with huge in() or one bulk write rather than separate ones). This is improving our throughputs by orders of magnitude while the pipeline pulls millions of documents per second from the database. I just don't see how Loom helps. Loom could help if you had blocking APIs to start, but you get much better results if you just resolve to use async, non-blocking wrapped in ReactiveX.
- geodel 5y agoLoom will help folks who prefer writing straightforward Java code instead of some random reactive library with obscure exception handling and poor to impossible debuggability. Now I get it is hard for many folks to understand that part. Just like at my workplace people think it is impossible to write micro service without SpringBoot. > Loom could help if you had blocking APIs to start, but you get much better results if you just resolve to use async, non-blocking wrapped in ReactiveX. There might be billions of lines of legacy code which would adapt to Loom with minimal changes but impossible to turn in ReactiveX etc without enormous investment and risk. Your ideas are rather simplistic for real world.
- deleted 5y ago[deleted]
- dikei 5y agoYup, Loom will simplify a lot the Producer-Consumer pattern on I/O operation. With virtual threads, it's basically free to block on consumer threads, so you would need only 1 bounded pool for the consumers. Currently for efficiency, you would need at least 2 pools: 1 small bounded pool for dequeuing the requests and create the IO operation, and 1 unbounded pool for actually executing the IO operation.