8 ms·

> The problem with platform threads is that they are expensive from a lot of points of view. First, they are costly to create. Whenever a platform thread is mad

by wahern 4y ago

> The problem with platform threads is that they are expensive from a lot of points of view. First, they are costly to create. Whenever a platform thread is made, the OS must allocate a large amount of memory (megabytes) in the stack to store the thread context, native, and Java call stacks. This is due to the not resizable nature of the stack. Moreover, whenever the scheduler preempts a thread from execution, this enormous amount of memory must be moved around. Scheduler pre-emption does not cause stack memory to be copied. Perhaps they're thinking of registers. (As an side, suspension and resumption of Java virtual threads does result in its stack being copied--saved and restored--as this was deemed less costly[1] than growable stacks, which is how Go works.) > As we can imagine, this is a costly operation, in space and time. In fact, the massive size of the stack frame limits the number of threads that can be created. We can reach an OutOfMemoryError quite easily in Java, continually instantiating new platform threads till the OS runs out of memory: Stack frame != stack. The author seems confused about some concepts. I didn't read beyond this so don't know whether that confusion effected any of their conclusions or advice. [1] EDIT: Whether less costly in terms of performance or development effort I'm not sure. A major reason JavaScript and many other languages don't implement stackfull coroutines is that the virtual machines--interpreters, JITs, etc--are written in a way that in-language function calls and recursion directly or indirectly rely on the underlying native "C" stack. This correspondence is not something you can typically remedy without completely rewriting the implementation from scratch. Language implementations like Go and Lua and were written from the beginning to avoid this correspondence. To accomplish stackfull coroutines languages like Java and, IIUC, OCaml really had no choice but to rely on some other tricks, though I think OCaml permitted some tricks not available to Java, because OCaml could do some transforms which Java couldn't given the nature of the JVM.

intelVISA 4y ago
People are worried about GPT4 producing nonsense huh... articles like this prove humans still have that market on lock.
- grimgrin 4y ago
  > The problem with platform threads is that they are expensive from a lot of points of view. First, they are costly to create. Whenever a platform thread is made, the OS must allocate a large amount of memory (megabytes) in the stack to store the thread context, native, and Java call stacks. This is due to the not resizable nature of the stack. Moreover, whenever the scheduler preempts a thread from execution, this enormous amount of memory must be moved around. There are a few issues and inaccuracies in this statement. While it is true that platform threads can be expensive in terms of resources, the claim that the OS must allocate "megabytes" of memory for the stack is an exaggeration. The actual size of the stack depends on the operating system and the specific implementation, but typical default values range from a few dozen kilobytes to a few hundred kilobytes, not megabytes. The statement implies that the entire stack is moved around when the scheduler preempts a thread from execution. This is not accurate. When a thread is preempted, the operating system saves the context of the thread, which is a relatively small amount of data, including the values of the CPU registers and the program counter. The stack itself is not moved around during this process. It is not correct to say that the stack is "not resizable." While the default stack size is set by the operating system, many programming languages and operating systems allow you to specify the stack size when creating a new thread. However, it is true that once a thread has been created, its stack size typically cannot be changed. Sent from OpenAI.
  - telotortium 4y ago
    > It is not correct to say that the stack is "not resizable." While the default stack size is set by the operating system, many programming languages and operating systems allow you to specify the stack size when creating a new thread. However, it is true that once a thread has been created, its stack size typically cannot be changed. Seems like a typical confused/pendantic Stack Overflow answer. I would say that stack size is "configurable", but not "resizable", because you indeed can't change the stack size (down, at least) of an already-created OS thread.
    - titzer 4y ago
      You can create pthreads with manually allocated stack segments and resize them later. https://www.ibm.com/docs/en/zos/2.2.0?topic=functions-pthread-attr-setstack-set-stack-attribute https://www.ibm.com/docs/en/zos/2.2.0?topic=functions-pthrea...
ameliaquining 4y ago
I think JavaScript doesn't want this because, semantically, the whole language is designed around the idea of a single thread of execution, that can't be suspended except explicitly with an await statement. So if you call a function, you know that it can't suspend and let some other thread take control and make arbitrary changes out from under you before control returns to you. Breaking this assumption would probably break too much existing code.
- avianlyric 4y ago
  What do stackfull coroutines have to do with cooperative vs non-cooperative concurrency? They’re entirely orthogonal problems. JavaScript absolutely does want stackfull coroutines (even if they’re not called coroutines, but just async stacks). That why chrome has so much magic inside it to reassemble async stack traces for exceptions. But it has to do that via all manner of complex bookkeeping and jiggery-pokery, which is often broken by libraries doing clever things. Having async functionality built on top of cooperative coroutines, all sharing a single system thread, would make async stacktraces trivial to produce accurately, and make it substantially easier to debug highly interwoven async code.
  - ameliaquining 4y ago
    The connection is that you can inefficiently simulate stackful coroutines with non-cooperative concurrency. Just give each coroutine its own OS thread, and block whenever the coroutine yields. It's slow and not a good way to build real systems, but the fact that it's possible at all means that libraries have to be designed to be able to handle it (e.g., that if they call into user code, it might yield the thread) and so not assume for correctness that their code will keep running in the order they control without interruption. JavaScript wasn't designed to support non-cooperative concurrency, and so a lot of existing code does assume that it's safe to, e.g., mutate global state in a way that other code doesn't expect or can't handle, run some operation, and then put the global state back how it was. Chrome's async stack traces don't undermine this because they are only for debugging; they can't in any way affect control flow. Stackful coroutines would break this assumption, the same way preemptive multithreading would. You can argue that the language should have included these features from the beginning, but it didn't, and changing it now would mess up the ecosystem.
fulafel 4y ago
Also the space for the native thread stack ("C stack") is just allocated virtual memory (at least on *ix), not physical memory. When the program starts to touch stack pages, on first touch the user code will trap to the OS where the vm system will transparently fill in the needed physical pages as the usage grows. Virtual threads don't seem like a worthwhile complexity tradeoff unless you're trying to run lots of threads in 32-bit address space. I wonder if this got started in that era and just took time to mature.
- jeroenhd 4y ago
  Virtual threads usually don't suffer the overhead of asking the OS to spawn a thread. If your workload consists of many small concurrent tasks, spawning threads can easily become costlier than processing the workload itself. These "let's make our own threads" implementations all seem to stem from "I want threads, but I don't want to wait for the kernel to do its thing". This approach has some downsides for implementations (there's a reason the kernel takes a moment to spawn a thread and now you have to deal with the implications) but staying in userland also has some advantages in terms of pure performance. Such tasks could of course be done faster using thread pools and a manual division of the workload (or adding locking to a dynamic work queue, etc.) but the threading model can be easier to visualise and reason about. It sits somewhere in the middle between the performance of a custom threading solution and the ease of use of single threaded code. I imagine things like web servers, dealing with tons of different connections, will be able to use this mechanism quite effectively. If you're just batching through a dataset, I don't think you'll have much of an advantage using this model.
  - xxs 4y ago
    >If your workload consists of many small concurrent tasks, spawning threads can easily become costlier than processing the workload itself. Executors have been in java since 1.5, and even before people used queues and thread pools. Starting threads on demand has never been an accepted way to write code aside, perhaps, some blog posts.
- pdpi 4y ago
  This sort of lightweight thread is at the core of things like Erlang or Go. You can just spin up processes/goroutines by the thousands without impacting performance too much. It just completely changes the way you write concurrency code.

by wahern 4y ago

intelVISA 4y ago

People are worried about GPT4 producing nonsense huh... articles like this prove humans still have that market on lock.

grimgrin 4y ago
> The problem with platform threads is that they are expensive from a lot of points of view. First, they are costly to create. Whenever a platform thread is made, the OS must allocate a large amount of memory (megabytes) in the stack to store the thread context, native, and Java call stacks. This is due to the not resizable nature of the stack. Moreover, whenever the scheduler preempts a thread from execution, this enormous amount of memory must be moved around. There are a few issues and inaccuracies in this statement. While it is true that platform threads can be expensive in terms of resources, the claim that the OS must allocate "megabytes" of memory for the stack is an exaggeration. The actual size of the stack depends on the operating system and the specific implementation, but typical default values range from a few dozen kilobytes to a few hundred kilobytes, not megabytes. The statement implies that the entire stack is moved around when the scheduler preempts a thread from execution. This is not accurate. When a thread is preempted, the operating system saves the context of the thread, which is a relatively small amount of data, including the values of the CPU registers and the program counter. The stack itself is not moved around during this process. It is not correct to say that the stack is "not resizable." While the default stack size is set by the operating system, many programming languages and operating systems allow you to specify the stack size when creating a new thread. However, it is true that once a thread has been created, its stack size typically cannot be changed. Sent from OpenAI.
- telotortium 4y ago
  > It is not correct to say that the stack is "not resizable." While the default stack size is set by the operating system, many programming languages and operating systems allow you to specify the stack size when creating a new thread. However, it is true that once a thread has been created, its stack size typically cannot be changed. Seems like a typical confused/pendantic Stack Overflow answer. I would say that stack size is "configurable", but not "resizable", because you indeed can't change the stack size (down, at least) of an already-created OS thread.
  - titzer 4y ago
    You can create pthreads with manually allocated stack segments and resize them later. https://www.ibm.com/docs/en/zos/2.2.0?topic=functions-pthread-attr-setstack-set-stack-attribute https://www.ibm.com/docs/en/zos/2.2.0?topic=functions-pthrea...

ameliaquining 4y ago

I think JavaScript doesn't want this because, semantically, the whole language is designed around the idea of a single thread of execution, that can't be suspended except explicitly with an await statement. So if you call a function, you know that it can't suspend and let some other thread take control and make arbitrary changes out from under you before control returns to you. Breaking this assumption would probably break too much existing code.

avianlyric 4y ago
What do stackfull coroutines have to do with cooperative vs non-cooperative concurrency? They’re entirely orthogonal problems. JavaScript absolutely does want stackfull coroutines (even if they’re not called coroutines, but just async stacks). That why chrome has so much magic inside it to reassemble async stack traces for exceptions. But it has to do that via all manner of complex bookkeeping and jiggery-pokery, which is often broken by libraries doing clever things. Having async functionality built on top of cooperative coroutines, all sharing a single system thread, would make async stacktraces trivial to produce accurately, and make it substantially easier to debug highly interwoven async code.
- ameliaquining 4y ago
  The connection is that you can inefficiently simulate stackful coroutines with non-cooperative concurrency. Just give each coroutine its own OS thread, and block whenever the coroutine yields. It's slow and not a good way to build real systems, but the fact that it's possible at all means that libraries have to be designed to be able to handle it (e.g., that if they call into user code, it might yield the thread) and so not assume for correctness that their code will keep running in the order they control without interruption. JavaScript wasn't designed to support non-cooperative concurrency, and so a lot of existing code does assume that it's safe to, e.g., mutate global state in a way that other code doesn't expect or can't handle, run some operation, and then put the global state back how it was. Chrome's async stack traces don't undermine this because they are only for debugging; they can't in any way affect control flow. Stackful coroutines would break this assumption, the same way preemptive multithreading would. You can argue that the language should have included these features from the beginning, but it didn't, and changing it now would mess up the ecosystem.

fulafel 4y ago

Also the space for the native thread stack ("C stack") is just allocated virtual memory (at least on *ix), not physical memory. When the program starts to touch stack pages, on first touch the user code will trap to the OS where the vm system will transparently fill in the needed physical pages as the usage grows. Virtual threads don't seem like a worthwhile complexity tradeoff unless you're trying to run lots of threads in 32-bit address space. I wonder if this got started in that era and just took time to mature.

jeroenhd 4y ago
Virtual threads usually don't suffer the overhead of asking the OS to spawn a thread. If your workload consists of many small concurrent tasks, spawning threads can easily become costlier than processing the workload itself. These "let's make our own threads" implementations all seem to stem from "I want threads, but I don't want to wait for the kernel to do its thing". This approach has some downsides for implementations (there's a reason the kernel takes a moment to spawn a thread and now you have to deal with the implications) but staying in userland also has some advantages in terms of pure performance. Such tasks could of course be done faster using thread pools and a manual division of the workload (or adding locking to a dynamic work queue, etc.) but the threading model can be easier to visualise and reason about. It sits somewhere in the middle between the performance of a custom threading solution and the ease of use of single threaded code. I imagine things like web servers, dealing with tons of different connections, will be able to use this mechanism quite effectively. If you're just batching through a dataset, I don't think you'll have much of an advantage using this model.
- xxs 4y ago
  >If your workload consists of many small concurrent tasks, spawning threads can easily become costlier than processing the workload itself. Executors have been in java since 1.5, and even before people used queues and thread pools. Starting threads on demand has never been an accepted way to write code aside, perhaps, some blog posts.
pdpi 4y ago
This sort of lightweight thread is at the core of things like Erlang or Go. You can just spin up processes/goroutines by the thousands without impacting performance too much. It just completely changes the way you write concurrency code.