4 ms·

Also the space for the native thread stack ("C stack") is just allocated virtual memory (at least on *ix), not physical memory. When the program starts to touch

Also the space for the native thread stack ("C stack") is just allocated virtual memory (at least on *ix), not physical memory. When the program starts to touch stack pages, on first touch the user code will trap to the OS where the vm system will transparently fill in the needed physical pages as the usage grows. Virtual threads don't seem like a worthwhile complexity tradeoff unless you're trying to run lots of threads in 32-bit address space. I wonder if this got started in that era and just took time to mature.

jeroenhd 4y ago
Virtual threads usually don't suffer the overhead of asking the OS to spawn a thread. If your workload consists of many small concurrent tasks, spawning threads can easily become costlier than processing the workload itself. These "let's make our own threads" implementations all seem to stem from "I want threads, but I don't want to wait for the kernel to do its thing". This approach has some downsides for implementations (there's a reason the kernel takes a moment to spawn a thread and now you have to deal with the implications) but staying in userland also has some advantages in terms of pure performance. Such tasks could of course be done faster using thread pools and a manual division of the workload (or adding locking to a dynamic work queue, etc.) but the threading model can be easier to visualise and reason about. It sits somewhere in the middle between the performance of a custom threading solution and the ease of use of single threaded code. I imagine things like web servers, dealing with tons of different connections, will be able to use this mechanism quite effectively. If you're just batching through a dataset, I don't think you'll have much of an advantage using this model.
- xxs 4y ago
  >If your workload consists of many small concurrent tasks, spawning threads can easily become costlier than processing the workload itself. Executors have been in java since 1.5, and even before people used queues and thread pools. Starting threads on demand has never been an accepted way to write code aside, perhaps, some blog posts.
pdpi 4y ago
This sort of lightweight thread is at the core of things like Erlang or Go. You can just spin up processes/goroutines by the thousands without impacting performance too much. It just completely changes the way you write concurrency code.
- xxs 4y ago
  >concurrency code. Likely parallel but not concurrency, i.e. there are very few (if any) contention points.
BenoitP 4y ago
> unless you're trying to run lots of threads Well this is exactly the design point. The authors want to promote a coding style where spawning a new thread is ultra-cheap, possibly at a ratio of very few IO calls per virtual thread. The ideal application would be WhatsApp's use of Erlang [1][2]: 2.8M active connections per server (in 2012! 100GB RAM servers), each of them mostly idle with 200k msgs/sec. All of this while keeping the threading model, keeping your stacks intact for debugging, and possibly a hierarchy of threads where you can kill a whole branch and hot-reload it with new code. (which is a thing that's not easy to do with reactive programming / async await) [1] http://highscalability.com/blog/2014/2/26/the-whatsapp-architecture-facebook-bought-for-19-billion.html http://highscalability.com/blog/2014/2/26/the-whatsapp-archi... [2] https://web.archive.org/web/20221220020352/http://www.erlang-factory.com/upload/presentations/558/efsf2012-whatsapp-scaling.pdf https://web.archive.org/web/20221220020352/http://www.erlang...
- silvestrov 4y ago
  > 2.8M active connections per server I think this is a use-case best left for nginx and other proxies written i C. Java should be easy to use. Leave weird specialcase programming to special tools. Don't try to make java solve all problems. Java nio became a monster of complexity which made implementing an http server much more complex that it should be.
  - gpderetta 4y ago
    With modern hardware 2.8M doesn't seem a particularly large number. Modern Java VMs are pretty efficient (more so after project Loom), so I don't see why Java wouldn't be appropriate for this use case. In fact efficient virtual threads would allow avoiding nio and provide a fast and sane blocking interface. Note: I say this as a C++ programmer.
    - deleted 4y ago
      [deleted]
    - xxs 4y ago
      Having that many connections requires tons of buffering and effectively manual scheduling (or clients can experience latency issues). At the very least non-blocking io allows to tune the scheduling depending on the clients behavior and importance. Personal experience: tons of java nio and lock-free structures.

jeroenhd 4y ago

Virtual threads usually don't suffer the overhead of asking the OS to spawn a thread. If your workload consists of many small concurrent tasks, spawning threads can easily become costlier than processing the workload itself. These "let's make our own threads" implementations all seem to stem from "I want threads, but I don't want to wait for the kernel to do its thing". This approach has some downsides for implementations (there's a reason the kernel takes a moment to spawn a thread and now you have to deal with the implications) but staying in userland also has some advantages in terms of pure performance. Such tasks could of course be done faster using thread pools and a manual division of the workload (or adding locking to a dynamic work queue, etc.) but the threading model can be easier to visualise and reason about. It sits somewhere in the middle between the performance of a custom threading solution and the ease of use of single threaded code. I imagine things like web servers, dealing with tons of different connections, will be able to use this mechanism quite effectively. If you're just batching through a dataset, I don't think you'll have much of an advantage using this model.

xxs 4y ago
>If your workload consists of many small concurrent tasks, spawning threads can easily become costlier than processing the workload itself. Executors have been in java since 1.5, and even before people used queues and thread pools. Starting threads on demand has never been an accepted way to write code aside, perhaps, some blog posts.

pdpi 4y ago

This sort of lightweight thread is at the core of things like Erlang or Go. You can just spin up processes/goroutines by the thousands without impacting performance too much. It just completely changes the way you write concurrency code.

xxs 4y ago
>concurrency code. Likely parallel but not concurrency, i.e. there are very few (if any) contention points.

BenoitP 4y ago

> unless you're trying to run lots of threads Well this is exactly the design point. The authors want to promote a coding style where spawning a new thread is ultra-cheap, possibly at a ratio of very few IO calls per virtual thread. The ideal application would be WhatsApp's use of Erlang [1][2]: 2.8M active connections per server (in 2012! 100GB RAM servers), each of them mostly idle with 200k msgs/sec. All of this while keeping the threading model, keeping your stacks intact for debugging, and possibly a hierarchy of threads where you can kill a whole branch and hot-reload it with new code. (which is a thing that's not easy to do with reactive programming / async await) [1] http://highscalability.com/blog/2014/2/26/the-whatsapp-architecture-facebook-bought-for-19-billion.html http://highscalability.com/blog/2014/2/26/the-whatsapp-archi... [2] https://web.archive.org/web/20221220020352/http://www.erlang-factory.com/upload/presentations/558/efsf2012-whatsapp-scaling.pdf https://web.archive.org/web/20221220020352/http://www.erlang...

silvestrov 4y ago
> 2.8M active connections per server I think this is a use-case best left for nginx and other proxies written i C. Java should be easy to use. Leave weird specialcase programming to special tools. Don't try to make java solve all problems. Java nio became a monster of complexity which made implementing an http server much more complex that it should be.
- gpderetta 4y ago
  With modern hardware 2.8M doesn't seem a particularly large number. Modern Java VMs are pretty efficient (more so after project Loom), so I don't see why Java wouldn't be appropriate for this use case. In fact efficient virtual threads would allow avoiding nio and provide a fast and sane blocking interface. Note: I say this as a C++ programmer.
  - deleted 4y ago
    [deleted]
  - xxs 4y ago
    Having that many connections requires tons of buffering and effectively manual scheduling (or clients can experience latency issues). At the very least non-blocking io allows to tune the scheduling depending on the clients behavior and importance. Personal experience: tons of java nio and lock-free structures.