5 ms·

Loom sets out to give you a sane programming paradigm similar to what threads do (i.e. as opposed to programming asynchronous I/O in Java with some type of call

by zinxq 4y ago

Loom sets out to give you a sane programming paradigm similar to what threads do (i.e. as opposed to programming asynchronous I/O in Java with some type of callback) without the overhead of Operating System threads. That's a very cool and a noble pursuit. But the title of this article might as well have been "5M persistent connections with Linux" because that's where the magic 5M connections happen. I could also attempt 5M connections at the Java level using Netty and asynchronous IO - no threads or Loom. Again, it'd take more Linux configuration than anything else. If that configuration did happen though now you can also do it in C# async/await, javascript, I'm sure Erlang and anything else that does Asynchronous I/O whether it's masked by something like Loom/Async/Await or not.

pron 4y ago
It is true that the experiment exercises the OS, but that's only part of the point. The other part is that it uses a simple, blocking, thread-per-request model with Java 1.0 networking APIs. So this is "achieving 5M persistent connections with (essentially) 26-year-old code that's fully debuggable and observable by the platform." This stresses both the OS and the Java runtime. So while you could achieve 5M in other ways, those ways would not only be more complex, but also not really observable/debuggable by Java platform tools.
- cheradenine_uk 4y ago
  This. Writing the sort of applications that I get involved with, it's frequently the case whilst it's true that 1 OS thread/java thread was a theoretical scalability limitation - in practice we were never likely to hit it (and there was always the 'get a bigger computer'). But: the complexity mavens inside our company and projects we rely upon get bitten by an obsessive need to chase 'scalability' /at all costs/. Which is fine, but the downside to that is the negative consequences of coloured functions comes into play. We end up suffering having to deal with vert.x or kotlin or whatever flavour-of-the-month solution is that is /inherently/ harder to reason about than a linear piece of code. If you're in a c# project, the you get a library that's async, and boom, game over. If loom gets even within performance shouting distance of those other models, it's ought to kill (for all but the edgiest of edge-cases) reactive programming in the java space dead. You might be able to make a case - obviously depending on your use cases which are not mine - that extracting, say, 50% more scalability is worth the downsides. If that number is, say, 5%, then for the vast majority of projects the answer is going to be 'no'. I say 'ought to', as I fear the adage that "developers love complexity the way moths love flames - and often with the same results". I see both engineers and projects (Hibernate and keycloak, IIRC) have a great deal of themselves invested in their Rx position, and I already sense that they're not going to give it up without a fight. So: the headline number is less important than "for virtually everyone you will no longer have to trade simplicity for scalability". I can't wait!
  - mike_hearn 4y ago
    A couple of points to consider. 1. Demanding scalability for inappropriate projects and at any cost is something I've seen too, and on investigation it was usually related to former battle scars. A software system that stops scaling at the wrong time can be horrific for the business. Some of them never recover, the canonical example being MySpace, but I've heard of other examples that were less public. In finance entire multi-year IT projects by huge teams have failed and had to be scrapped because they didn't scale to even current business needs, let alone future needs. Emergency projects to make something "scale" because new customers have been on-boarded, or business requirements changed, are the sort of thing nobody wants to get caught up in. Over time these people graduate into senior management where they become architects who react to those bad experiences by insisting on making scalability a checkbox to tick. Of course there's also trying to make easy projects more challenging, resume-driven development etc too. It's not just that. But that's one way it can happen. 2. Rx type models aren't just about the cost of threads. An abstraction over a stream of events is useful in many contexts, for example, single-threaded GUIs.
    - lostcolony 4y ago
      One additional - as noted, it's been 26 years since Java's founding. Project Loom has been around since at least 2018 and still has no release date. It'll be cool for Java projects whenever it comes out, but I just...have a hard time caring right now. I can't use it for old codebases currently, and new codebases I'm not using one request per Java thread anyway (tbh - when it's my choice I'm not choosing the JVM at all). The space has moved, and continues to move. In no way to say the JVM shouldn't be adopting the good ideas that come along the way, that is one of the benefits of being as conservative and glacial in adoption as it is, but I just...don't get excited about them, or find myself in any position in relation to the JVM (Java specifically, but the fundamentals affect other languages) other than "ugh, this again".
      - Scarbutt 4y ago
        What has the space move to?
      - 4y ago
- zinxq 4y ago
  I think we're in agreement. Ignoring under the hood - Loom's programming paradigm (from the viewpoint of control flow) is the Threading programming paradigm. (Virtual)Thread-per-connection programming is easier and far more intuitive than asynchronous (i.e. callback-esque) programming. I still attest though - The 5M connections in this example is still a red herring. Can we get to 6M? Can we get to 10M? Is that a question for Loom or Java's asynchronous IO system? No - it's a question for the operating system. Loom and Java NIO can handle probably a billion connections as programmed. Java Threads cannot - although that too is a broken statement. "Linux Threads cannot" is the real statement. You can't have that many for resource reasons. Java Threads are just a thin abstraction on top of that. Linux out of the box can't do 5M connections (last I checked). It takes Linux tuning artistry to get it there. Don't get me wrong - I think Loom is cool. It's attempted to do the same thing as Async/Await tried - just better. But it is most definitely not the only way to achieve 5MM connections with Java or anything else. Possibly however, it's the most friendly and intuitive way to do it. *We typically vilify Java Threads for the Ram they consume. Something like 1M per thread or something (tunable). Loom must still use "some" ram per connection although surely far far less (and of course Linux must use some amount of kernel ram per connection too).
  - pron 4y ago
    > But it is most definitely not the only way to achieve 5MM connections with Java or anything else. Possibly however, it's the most friendly and intuitive way to do it. It is the only way to achieve that many connections with Java in a way that's debuggable and observable by the platform and its tools, regardless of its intuitiveness or friendliness to human programmers. It's important to understand that this is an objective technical difference, and one of the cornerstones of the project. Computations that are composed in the asynchronous style are invisible to the runtime. Your server could be overloaded with I/O, and yet your profile will show idle thread pools. Virtual threads don't just allow you to write something you could do anyway in some other way. They actually do work that has simply been impossible so far at that scale: they allow the runtime and its tools to understand how your program is composed and observe it at runtime in a meaningful and helpful way. One of the main reasons so many companies turn to Java for their most important server-side applications is that it offers unmatched observability into what the program is doing (at least among other languages/platforms with similar performance). But that ability was missing for high-scale concurrency. Virtual threads add it to the platform.
deleted 4y ago
[deleted]
simulate-me 4y ago
As the GP said, what's cool about this is how simple the code is. You might be able to achieve 5M connections in Java using an event loop based solution (eg Netty), but if the connection handlers need to do any async work, then they also need to be written using an event loop, which is not how most people write Java. Simply put, 5M connections was not possible using Java in the way most people write Java.

by zinxq 4y ago

pron 4y ago

It is true that the experiment exercises the OS, but that's only part of the point. The other part is that it uses a simple, blocking, thread-per-request model with Java 1.0 networking APIs. So this is "achieving 5M persistent connections with (essentially) 26-year-old code that's fully debuggable and observable by the platform." This stresses both the OS and the Java runtime. So while you could achieve 5M in other ways, those ways would not only be more complex, but also not really observable/debuggable by Java platform tools.

cheradenine_uk 4y ago
This. Writing the sort of applications that I get involved with, it's frequently the case whilst it's true that 1 OS thread/java thread was a theoretical scalability limitation - in practice we were never likely to hit it (and there was always the 'get a bigger computer'). But: the complexity mavens inside our company and projects we rely upon get bitten by an obsessive need to chase 'scalability' /at all costs/. Which is fine, but the downside to that is the negative consequences of coloured functions comes into play. We end up suffering having to deal with vert.x or kotlin or whatever flavour-of-the-month solution is that is /inherently/ harder to reason about than a linear piece of code. If you're in a c# project, the you get a library that's async, and boom, game over. If loom gets even within performance shouting distance of those other models, it's ought to kill (for all but the edgiest of edge-cases) reactive programming in the java space dead. You might be able to make a case - obviously depending on your use cases which are not mine - that extracting, say, 50% more scalability is worth the downsides. If that number is, say, 5%, then for the vast majority of projects the answer is going to be 'no'. I say 'ought to', as I fear the adage that "developers love complexity the way moths love flames - and often with the same results". I see both engineers and projects (Hibernate and keycloak, IIRC) have a great deal of themselves invested in their Rx position, and I already sense that they're not going to give it up without a fight. So: the headline number is less important than "for virtually everyone you will no longer have to trade simplicity for scalability". I can't wait!
- mike_hearn 4y ago
  A couple of points to consider. 1. Demanding scalability for inappropriate projects and at any cost is something I've seen too, and on investigation it was usually related to former battle scars. A software system that stops scaling at the wrong time can be horrific for the business. Some of them never recover, the canonical example being MySpace, but I've heard of other examples that were less public. In finance entire multi-year IT projects by huge teams have failed and had to be scrapped because they didn't scale to even current business needs, let alone future needs. Emergency projects to make something "scale" because new customers have been on-boarded, or business requirements changed, are the sort of thing nobody wants to get caught up in. Over time these people graduate into senior management where they become architects who react to those bad experiences by insisting on making scalability a checkbox to tick. Of course there's also trying to make easy projects more challenging, resume-driven development etc too. It's not just that. But that's one way it can happen. 2. Rx type models aren't just about the cost of threads. An abstraction over a stream of events is useful in many contexts, for example, single-threaded GUIs.
  - lostcolony 4y ago
    One additional - as noted, it's been 26 years since Java's founding. Project Loom has been around since at least 2018 and still has no release date. It'll be cool for Java projects whenever it comes out, but I just...have a hard time caring right now. I can't use it for old codebases currently, and new codebases I'm not using one request per Java thread anyway (tbh - when it's my choice I'm not choosing the JVM at all). The space has moved, and continues to move. In no way to say the JVM shouldn't be adopting the good ideas that come along the way, that is one of the benefits of being as conservative and glacial in adoption as it is, but I just...don't get excited about them, or find myself in any position in relation to the JVM (Java specifically, but the fundamentals affect other languages) other than "ugh, this again".
    - Scarbutt 4y ago
      What has the space move to?
    - 4y ago
zinxq 4y ago
I think we're in agreement. Ignoring under the hood - Loom's programming paradigm (from the viewpoint of control flow) is the Threading programming paradigm. (Virtual)Thread-per-connection programming is easier and far more intuitive than asynchronous (i.e. callback-esque) programming. I still attest though - The 5M connections in this example is still a red herring. Can we get to 6M? Can we get to 10M? Is that a question for Loom or Java's asynchronous IO system? No - it's a question for the operating system. Loom and Java NIO can handle probably a billion connections as programmed. Java Threads cannot - although that too is a broken statement. "Linux Threads cannot" is the real statement. You can't have that many for resource reasons. Java Threads are just a thin abstraction on top of that. Linux out of the box can't do 5M connections (last I checked). It takes Linux tuning artistry to get it there. Don't get me wrong - I think Loom is cool. It's attempted to do the same thing as Async/Await tried - just better. But it is most definitely not the only way to achieve 5MM connections with Java or anything else. Possibly however, it's the most friendly and intuitive way to do it. *We typically vilify Java Threads for the Ram they consume. Something like 1M per thread or something (tunable). Loom must still use "some" ram per connection although surely far far less (and of course Linux must use some amount of kernel ram per connection too).
- pron 4y ago
  > But it is most definitely not the only way to achieve 5MM connections with Java or anything else. Possibly however, it's the most friendly and intuitive way to do it. It is the only way to achieve that many connections with Java in a way that's debuggable and observable by the platform and its tools, regardless of its intuitiveness or friendliness to human programmers. It's important to understand that this is an objective technical difference, and one of the cornerstones of the project. Computations that are composed in the asynchronous style are invisible to the runtime. Your server could be overloaded with I/O, and yet your profile will show idle thread pools. Virtual threads don't just allow you to write something you could do anyway in some other way. They actually do work that has simply been impossible so far at that scale: they allow the runtime and its tools to understand how your program is composed and observe it at runtime in a meaningful and helpful way. One of the main reasons so many companies turn to Java for their most important server-side applications is that it offers unmatched observability into what the program is doing (at least among other languages/platforms with similar performance). But that ability was missing for high-scale concurrency. Virtual threads add it to the platform.

deleted 4y ago

[deleted]

simulate-me 4y ago

As the GP said, what's cool about this is how simple the code is. You might be able to achieve 5M connections in Java using an event loop based solution (eg Netty), but if the connection handlers need to do any async work, then they also need to be written using an event loop, which is not how most people write Java. Simply put, 5M connections was not possible using Java in the way most people write Java.