4 ms·

Have you seen the performance of an application "just" creating 1000 Linux threads, it's pretty bad. Also you're talking about linux, the Go runtime runs on ma

by Thaxll 8y ago

Have you seen the performance of an application "just" creating 1000 Linux threads, it's pretty bad. Also you're talking about linux, the Go runtime runs on many platforms, therefore is not dependent on platform thread performance.

paulddraper 8y ago
Creating Linux threads is pretty fast. You should be able to "just" create 1000 threads in "only" 2ms or less.
- presscast 8y ago
  On Linux/x86-32, the default stack size for a new thread is 2 mb. The default stack for a goroutine is 2 kb. Those 1000 goroutines eat up as much memory as roughy one OS thread. Granted, I've somewhat shifted the goalpost, but goroutines are cheaper than OS threads or just about every sensible definition of "cheap".
  - masklinn 8y ago
    > On Linux/x86-32, the default stack size for a new thread is 2 mb. It's just reserved, not committed until actually used. > Granted, I've somewhat shifted the goalpost Shifted? You've strapped them to a rocket and put them in orbit.
    - presscast 8y ago
      >No shit The general point still stands. No need to be unpleasant.
      - romed 8y ago
        Your general point does not stand. It is totally incorrect. A thread does not commit its stack limit immediately. The limit is the limit. The initial size is a single page.
        masklinn 8y ago
        And even then, the original point wasn't "an OS thread is as cheap as a userland thread", it was "all things considered a native thread is pretty cheap".
        presscast 8y ago
        You misunderstood the point.
- Thaxll 8y ago
  I'm not talking about creating them, I'm talking about memory / context switch. There is a reason why none of the webservers on linux use one thread per connection.
  - Matthias247 8y ago
    I know what you mean (nginx & co), but I wouldn't bet whether the majority of websites and APIs use a thread per connection or event-driven IO. E.g. there is are a lot of PHP sites out there, which are thread-per-connection. Or which are powered by Java servlets, which had been synchronous until quite recently.
  - paulddraper 8y ago
    That was unclear. Yes, 1000 concurrent threads will take stack size * 1000.
    - masklinn 8y ago
      Thread stacks are reserved, but not allocated up-front (unless you're running on OSv apparently[0]). 1000 concurrent threads will not take stack size * 1000, they'll take used pages * 1000 (with used pages being at least 1) + the kernel overhead of the thread structures. That's trivial to check, even in a high-level language (with its own additional overhead) e.g. spawning 1000 threads in Python on OSX takes ~25MB. OSX uses 512k stacks for non-main threads. [0] https://github.com/cloudius-systems/osv/issues/143 https://github.com/cloudius-systems/osv/issues/143

by Thaxll 8y ago

paulddraper 8y ago

Creating Linux threads is pretty fast. You should be able to "just" create 1000 threads in "only" 2ms or less.

presscast 8y ago
On Linux/x86-32, the default stack size for a new thread is 2 mb. The default stack for a goroutine is 2 kb. Those 1000 goroutines eat up as much memory as roughy one OS thread. Granted, I've somewhat shifted the goalpost, but goroutines are cheaper than OS threads or just about every sensible definition of "cheap".
- masklinn 8y ago
  > On Linux/x86-32, the default stack size for a new thread is 2 mb. It's just reserved, not committed until actually used. > Granted, I've somewhat shifted the goalpost Shifted? You've strapped them to a rocket and put them in orbit.
  - presscast 8y ago
    >No shit The general point still stands. No need to be unpleasant.
    - romed 8y ago
      Your general point does not stand. It is totally incorrect. A thread does not commit its stack limit immediately. The limit is the limit. The initial size is a single page.
      - masklinn 8y ago
        And even then, the original point wasn't "an OS thread is as cheap as a userland thread", it was "all things considered a native thread is pretty cheap".
      - presscast 8y ago
        You misunderstood the point.
Thaxll 8y ago
I'm not talking about creating them, I'm talking about memory / context switch. There is a reason why none of the webservers on linux use one thread per connection.
- Matthias247 8y ago
  I know what you mean (nginx & co), but I wouldn't bet whether the majority of websites and APIs use a thread per connection or event-driven IO. E.g. there is are a lot of PHP sites out there, which are thread-per-connection. Or which are powered by Java servlets, which had been synchronous until quite recently.
- paulddraper 8y ago
  That was unclear. Yes, 1000 concurrent threads will take stack size * 1000.
  - masklinn 8y ago
    Thread stacks are reserved, but not allocated up-front (unless you're running on OSv apparently[0]). 1000 concurrent threads will not take stack size * 1000, they'll take used pages * 1000 (with used pages being at least 1) + the kernel overhead of the thread structures. That's trivial to check, even in a high-level language (with its own additional overhead) e.g. spawning 1000 threads in Python on OSX takes ~25MB. OSX uses 512k stacks for non-main threads. [0] https://github.com/cloudius-systems/osv/issues/143 https://github.com/cloudius-systems/osv/issues/143