4 ms·

> The point wasn't that you couldn't change this value, it's that for our purposes (lots of green threads, shallow stacks) a much smaller default stack size is

by leif 16y ago

> The point wasn't that you couldn't change this value, it's that for our purposes (lots of green threads, shallow stacks) a much smaller default stack size is more sensible. Slava, you're better than that. WHY is a smaller default stack size more sensible? If this is all you need, and you can change it manually, do that!

coffeemug 16y ago
WHY is a smaller default stack size more sensible? Because if you have a large number of threads on an architecture with a small address space, you take away memory space from the buffer cache which can have catastrophic effects on performance. If this is all you need, and you can change it manually, do that! This needs to be understood in the context of the previous posts. The issue with NPTL is significantly lower performance on large number of threads due to context switching. Dan implemented a solution that uses coroutines to address that problem. This post deals with the over-provisioning issue described above, and it starts out by describing the default configuration for pthreads. Adjusting this to a smaller number on the NPTL implementation isn't enough, we needed to adjust it for the coroutines implementation.
- cperciva 16y ago
  Because if you have a large number of threads on an architecture with a small address space, you take away memory space from the buffer cache which can have catastrophic effects on performance. Say again? Address space exhaustion is a problem, sure, but it has nothing to do with the buffer cache. The kernel's address space is completely separate from any particular process' address space.
  - coffeemug 16y ago
    Address space exhaustion is a problem, sure, but it has nothing to do with the buffer cache. The kernel's address space is completely separate from any particular process' address space. For various reasons we don't use the kernel's cache, we have our own cache implementation that's in the same address space as the rest of the process.
    - cperciva 16y ago
      Oh dear god. You have my sincere and profound sympathies.
      - barrkel 16y ago
        It's pretty unusual for databases to delegate something as vital to performance as caching to the OS.
        cperciva 16y ago
        Sure, a database needs to cache. But it doesn't automatically need to cache at that level -- usually they can get away with caching data rather than pages.
        strlen 16y ago
        If you have your own cache and you don't use direct I/O i.e., they go through the OS page cache, don't have you have the issue of the OS page cache and the database "fighting" each other?

by leif 16y ago

coffeemug 16y ago

WHY is a smaller default stack size more sensible? Because if you have a large number of threads on an architecture with a small address space, you take away memory space from the buffer cache which can have catastrophic effects on performance. If this is all you need, and you can change it manually, do that! This needs to be understood in the context of the previous posts. The issue with NPTL is significantly lower performance on large number of threads due to context switching. Dan implemented a solution that uses coroutines to address that problem. This post deals with the over-provisioning issue described above, and it starts out by describing the default configuration for pthreads. Adjusting this to a smaller number on the NPTL implementation isn't enough, we needed to adjust it for the coroutines implementation.

cperciva 16y ago
Because if you have a large number of threads on an architecture with a small address space, you take away memory space from the buffer cache which can have catastrophic effects on performance. Say again? Address space exhaustion is a problem, sure, but it has nothing to do with the buffer cache. The kernel's address space is completely separate from any particular process' address space.
- coffeemug 16y ago
  Address space exhaustion is a problem, sure, but it has nothing to do with the buffer cache. The kernel's address space is completely separate from any particular process' address space. For various reasons we don't use the kernel's cache, we have our own cache implementation that's in the same address space as the rest of the process.
  - cperciva 16y ago
    Oh dear god. You have my sincere and profound sympathies.
    - barrkel 16y ago
      It's pretty unusual for databases to delegate something as vital to performance as caching to the OS.
      - cperciva 16y ago
        Sure, a database needs to cache. But it doesn't automatically need to cache at that level -- usually they can get away with caching data rather than pages.
        strlen 16y ago
        If you have your own cache and you don't use direct I/O i.e., they go through the OS page cache, don't have you have the issue of the OS page cache and the database "fighting" each other?