8 ms·
Really excited to see this! Seeing some of the early comments here I think folks may not realize how awesome this would be in the server space. After all, a bi
by hn_throwaway_99 5y ago
Really excited to see this! Seeing some of the early comments here I think folks may not realize how awesome this would be in the server space.
After all, a big reason that NodeJS won a lot of popularity on the server is that, for many types of common webserver workloads (i.e. lots of IO, relatively minor CPU usage), NodeJS can actually scale much better than Java with its thread-per-request model.
With these virtual threads, though, you could get the best of all possible worlds - a webserver that scales like NodeJS, but without some of the "CPU starvation" issues you can hit in Node if one executing request doesn't yield, and also without having to worry about "function coloring" like you do in Node with async vs. non-async functions.
Really, really fantastic development, have been waiting to see when this would come out.
- tannhaeuser 5y ago> NodeJS [...] with its thread-per-request model Node.js doesn't create a thread per request; it's single-threaded with evented I/O. You can use node-cluster to start more than a single thread to saturate multi-core CPUs and load-balance HTTP requests across these, but that doesn't make it thread-per-request.
- an_account_name 5y agoSuspect they're talking about Java - a lot of frameworks do exactly create a thread per request.
- didibus 5y agoI can't think of any framework that still does one thread per request. Normally there is a a queue of incoming requests and they then get dispatched on a thread pool as threads return to the pool. The challenge is normally that if any of the threads in the pool, as part of processing a request, needs to itself make an IO call, it will block. Ideally you'd want to park the request processing, return the thread to the pool, pick up the next request, until the IO is done where then on the next thread available from the pool you'd resume that request instead of picking another one. This is what the virtual threads will make really easy I think.
- MikeTheGreat 5y agoI think my high school English teacher would agree with you that the sentence is written awkwardly (I can see the 'awk' note, in red, on my paper right now :) ). Here's how I parsed it: > a big reason that NodeJS won a lot of popularity on the server is that, for many types of common webserver workloads [...], NodeJS can actually scale much better than Java with ~~it's~~ [Java's] thread-per-request model.
- tannhaeuser 5y agoAh that makes sense; didn't get this reading at all!
- bradfitz 5y ago> ~~it's~~ Why are you calling that out? The original "its" was correct without the apostrophe.
- MikeTheGreat 5y agoTwisol was right - I was trying to imply strikethrough using the Markdown syntax in an attempt to depict the idea of replacing "its" with "Java's". It didn't work as well as I hoped. In my mind I can see more 'Awk' scribbles on my post, and looking at it I agree :) Adding in the 's is 100% my mistake. I've been guilty of using "it's" as the possessive form for most of my life, but that changes today! :)
- bradfitz 5y ago> but that changes today! :) Exciting! :)
- Twisol 5y agoTildes are used for strikethrough in some markup dialects (including markdown), so I think they meant to depict replacing "its" with "Java's". No clue on the apostrophe.
- 5y ago
- hn_throwaway_99 5y agoSorry, yes, my sentence was poorly written with the ambiguous antecedent. Most Java webservers use a thread-per-request model, which is why Node can usually scale to more concurrent requests.
- vlovich123 5y agoAm I understanding correctly that this is basically goroutines for Java?
- hn_throwaway_99 5y agoYes. I am not as familiar with the underlying implementation of goroutines, but this description in the linked JEP sounds exactly how I understand goroutines to work: > The JDK implements virtual threads by storing their state, including the stack, on the Java heap. Virtual threads are scheduled by a scheduler in the Java class libraries, whose worker threads mount virtual threads on their backs when the virtual threads are executing, thus becoming their carriers. When a virtual thread parks -- say, when it blocks on some I/O operation or a java.util.concurrent synchronization construct -- it suspends, and the virtual thread's carrier is free to run any other task. When a virtual thread is unparked -- say, by an I/O operation completing -- it is submitted to the scheduler, which, when available, will mount and resume the virtual thread on some carrier thread, not necessarily the same one it ran on previously. In this way, when a virtual thread performs a blocking operation, instead of parking an OS thread, it is suspended by the JVM and another one scheduled in its place, all without blocking any OS threads (see the Limitations section).
- pkulak 5y agoYes. But, since there are _two_ kinds of threads in Java (os and virtual), you still have to be very careful never to block a virtual thread. In Go/JavaScript/Beam, it doesn't matter because you literally can't block a thread (while idle). This is the kind of thing that's not terribly useful until nearly every library you interact with is using it as well. Also, there's no new syntax, so you're stuck with all the same thread pool concurrency we've been using for decades. EDIT: It looks like I'm wrong about this: > My understanding is that you won't have to worry about blocking a virtual thread, because all IO APIs are being modified to park when executed in the context of a virtual thread.
- foota 5y agoIf it's added to the VM then won't that allow languages like Kotlin to add support?
- vbezhenar 5y agoJava still eats like 5x ram compared to node, Java tools are slow, Java frameworks are gargantuan, even those claiming "lean". Java is fine if you don't care about RAM and start time, though.
- sandGorgon 5y agohttps://quarkus.io/guides/building-native-image https://quarkus.io/guides/building-native-image You should try Quarkus. It is a production framework built by Redhat. It uses Java-GraalVM under the cover to compile your entire webapp to an executable (like golang does). It's just as fast. Java is the highest performance and most tuned VM there is. I think you're really thinking of java from a long time ago, if ur thinking this
- geodel 5y agoAh that Supersonic-Subatomic-Java. Whatever Redhat lacks in quality side in Java frameworks, they more than compensate with corny marketing taglines.
- rbanffy 5y ago> Java is the highest performance and most tuned VM there is. Not defending the opposite argument, but V8 is also pretty impressive. It's rooted in work done for Smalltalk long before JavaScript was a thing.
- MaxBarraclough 5y ago> It's rooted in work done for Smalltalk long before JavaScript was a thing. The same is true of HotSpot. https://en.wikipedia.org/wiki/HotSpot_(virtual_machine)#History https://en.wikipedia.org/wiki/HotSpot_(virtual_machine)#Hist...
- sverhagen 5y agoIf you use gargantuan Java frameworks, you'll use a lot of RAM. Just don't do that. With Spring Boot and similar frameworks, the RAM usage is really just very modest. I'll give you startup times, since I am not a believer in Quarkus and Graal. And I wouldn't use Java for a serverless function that needs to spin up and respond quickly. But for a typical (blue/green-deployed) application in my world, startup time is still only a few seconds, which is fine for many applications. And I am not settling for "fine", just saying that the startup time isn't a big consideration, against a lot of things the Java (or Spring, in my case) ecosystem offers me.
- jayd16 5y ago>you could get the best of all possible worlds Maybe not _all_ possible worlds. You still have original Threads for things that need an actual OS thread. Its not a solution for UI threading. There will be code that needs a native thread or non-preemptive threading and shouldn't be run on a virtual thread. In that sense there is method coloring but its yet to be seen how common a problem that will be. Library writers and frameworks will need to sort out patterns for how to call Runnables in a safe way. Still, its a nice tool to have.
- kaba0 5y agoThat’s not function coloring, it is up to the caller whether to start it in a virt thread or a real one. Function coloring is having two methods do the same thing differing only in name and signature (eg. there is a blocking sleep and a non-blocking one).
- jayd16 5y ago>it is up to the caller whether to start it in a virt thread or a real one. Sorta kinda but not when you're working in a framework that will call your code or working in some library where the abstracted code is non-obvious or uneasy to configure. Maybe its not function coloring, although I wouldn't know what else to call it and I think its quite similar. What would you call the problem?
- native_samples 5y agoYes but you always need original/kernel threads, regardless of what approach to async you need. The concept of a thread and a stack is hard-wired into the CPU. W.R.T. code that needs a native thread: at the moment there's only two types of such code. One is code that uses Java's synchronized statement. That's supposedly just a, ehm, small matter of programming to fix. The other is calling into non-JVM controlled code. That's fundamental and no approach to scalable concurrency can fix it, not CPS/async/await or anything else because it's a foreign compiler. But fortunately the JVM has some really interesting tricks up its sleeve there. For instance you can compile your native code using LLVM and then execute the bitcode on the JVM. Well, OK, currently GraalVM doesn't support Loom but hopefully Graal will be upgraded to do so as Loom gets integrated into HotSpot. And when it does, you will be able to call into code written in C/C++/Objective-C/Rust as long as that code can be recompiled with your own toolchain and as long as you can tolerate it being JITCd, also whilst benefiting from Loom's scalability.
- fulafel 5y ago> For many types of common webserver workloads (i.e. lots of IO, relatively minor CPU usage), NodeJS can actually scale much better than Java with its thread-per-request model Linux can handle a ginormous amount of threads quite well, would be interesting to see a deeper investigation to this theory.
- jankotek 5y agoJava uses some memory for stack, about 1MB for each thread.
- fulafel 5y agoThis doesn't in practice limit scaling though as it's linear and small in absolute terms vs what you can put in a server.
- MrBuddyCasino 5y agoThis is configurable, and 1MB is very generous. I think the JVM automatically grows the stack size as needed nowadays and starts low.
- native_samples 5y agoThe problem with doing it all native is that stack sizes are quite variable, especially in managed languages where modularity and code reuse works better, so it's common to have tons of libraries in a single project. The kernel won't object to lots of threads, but once those threads have been running for a while a lot of stack space will be paged in and used. Loom solves this by moving stacks to and from the heap, where there's a compacting concurrent GC to clean up the unused space.