7 ms·

> So, preemptive green-threads? This seems like a counter-productive and highly complicated re-implementation of what the OS thread model does, but inside the G

> So, preemptive green-threads? This seems like a counter-productive and highly complicated re-implementation of what the OS thread model does, but inside the Go runtime. You can't easily spawn 10.000 OS threads. > I would very much like for goroutines to remain cooperative. It is much simpler, naturally more performant, and quite easy to reason about. Cooperative is less simple to reason about, as the proposal explains. Preemption fully delivers what the developer already expects of goroutines.

vidarh 9y ago
> You can't easily spawn 10.000 OS threads. I just spawned 10.000 OS threads running separate mRuby instances in each thread on my laptop, to see if it'd cause any problems (because I happened to have an app sitting around that spawns separate threads; note: these are not Ruby threads, mRuby does not have built in threading). None at all. So yes, you can. I didn't check how much memory it'd take, however, or attempt to measure overheads in any way, so I'm not saying there aren't potential issues with it.
- jashmatthews 9y ago
  Simply using 1,000+ OS threads is often very close to as efficient as using coroutines or an event loop: https://www.slideshare.net/e456/tyma-paulmultithreaded1 https://www.slideshare.net/e456/tyma-paulmultithreaded1
  - zzzcpan 9y ago
    If your event loop is as slow as 1000+ coroutines or OS threads, you are doing it very very wrong.
    - jashmatthews 9y ago
      AFAIK the main benefit to coroutines is getting to manually manage tiny stacks. For example, in Go, they mark a goroutine's stack as "hasn't executed since the last time we marked", which makes GC with thousands of threads a lot faster.
    - arghwhat 9y ago
      [Citation needed]
      - jashmatthews 9y ago
        https://github.com/golang/go/issues/12061 https://github.com/golang/go/issues/12061 "Mark termination does not have to rescan stacks that haven’t executed since the last scan" With pre-emptive scheduling of OS threads, you potentially dirty a ton more stacks, and have to scan them all for pointers to heap memory. I'm not sure how VMs using lots of OS threads deal with this.
- coldtea 9y ago
  >So yes, you can. I didn't check how much memory it'd take, however, or attempt to measure overheads in any way, so I'm not saying there aren't potential issues with it. The "easily" part wasn't referring to feasibility, but to how effortless it is to do AND have performance and scalable code.
akvadrako 9y ago
> You can't easily spawn 10.000 OS threads. This is what I thought until last week, but it seems it's no big deal. 1M threads starts to be an issue.
arghwhat 9y ago
> You can't easily spawn 10.000 OS threads. Of course you can, without any problem at all. It's just more expensive than a green thread (they're full processes with stacks), and unlike green threads, it affects how processing time is sliced. It is, however, much cheaper than people tend to believe. > Cooperative is less simple to reason about, as the proposal explains. Cooperative scheduling is very simple to reason about. In the simplest form, you do not give up control unless you use I/O, a synchronization primitive, or explicitly give up control (runtime.Gosched() in Go). As a Go developer, and as a developer in any other language, this is exactly what I expect from any green thread implementation. That's how M:N scheduling has always been implemented, and how green threads have been implemented in other languages. Technically, preemptive is harder to reason about, but you save having to reason about it. However, if you expect preemption from goroutines, you have misunderstood the primitive.
- ngrilly 9y ago
  > In the simplest form, you do not give up control unless you use I/O, a synchronization primitive, or explicitly give up control (runtime.Gosched() in Go). This is not how Go works nowadays. There are other preemption points.
  - arghwhat 9y ago
    That's why I said "in the simplest form".
    - ngrilly 9y ago
      Yes, but it completely ruins the argument about "it's easy to reason about" ;-)
pcwalton 9y ago
> You can't easily spawn 10.000 OS threads. Of course you can. Go on, compile and run this program: #include <pthread.h> #include <stdlib.h> void *f(void *arg) { pthread_mutex_lock((pthread_mutex_t *)arg); pthread_mutex_unlock((pthread_mutex_t *)arg); return NULL; } int main() { pthread_t threads[10000]; pthread_mutex_t mutexes[10000]; for (unsigned i = 0; i < 10000; i++) { pthread_mutex_init(&mutexes[i], NULL); pthread_mutex_lock(&mutexes[i]); pthread_create(&threads[i], NULL, f, &mutexes[i]); } for (unsigned i = 0; i < 10000; i++) pthread_mutex_unlock(&mutexes[i]); for (unsigned i = 0; i < 10000; i++) pthread_join(threads[i], NULL); return 0; } Executes in a second for me, even on macOS.

vidarh 9y ago

> You can't easily spawn 10.000 OS threads. I just spawned 10.000 OS threads running separate mRuby instances in each thread on my laptop, to see if it'd cause any problems (because I happened to have an app sitting around that spawns separate threads; note: these are not Ruby threads, mRuby does not have built in threading). None at all. So yes, you can. I didn't check how much memory it'd take, however, or attempt to measure overheads in any way, so I'm not saying there aren't potential issues with it.

jashmatthews 9y ago
Simply using 1,000+ OS threads is often very close to as efficient as using coroutines or an event loop: https://www.slideshare.net/e456/tyma-paulmultithreaded1 https://www.slideshare.net/e456/tyma-paulmultithreaded1
- zzzcpan 9y ago
  If your event loop is as slow as 1000+ coroutines or OS threads, you are doing it very very wrong.
  - jashmatthews 9y ago
    AFAIK the main benefit to coroutines is getting to manually manage tiny stacks. For example, in Go, they mark a goroutine's stack as "hasn't executed since the last time we marked", which makes GC with thousands of threads a lot faster.
  - arghwhat 9y ago
    [Citation needed]
    - jashmatthews 9y ago
      https://github.com/golang/go/issues/12061 https://github.com/golang/go/issues/12061 "Mark termination does not have to rescan stacks that haven’t executed since the last scan" With pre-emptive scheduling of OS threads, you potentially dirty a ton more stacks, and have to scan them all for pointers to heap memory. I'm not sure how VMs using lots of OS threads deal with this.
coldtea 9y ago
>So yes, you can. I didn't check how much memory it'd take, however, or attempt to measure overheads in any way, so I'm not saying there aren't potential issues with it. The "easily" part wasn't referring to feasibility, but to how effortless it is to do AND have performance and scalable code.

akvadrako 9y ago

> You can't easily spawn 10.000 OS threads. This is what I thought until last week, but it seems it's no big deal. 1M threads starts to be an issue.

arghwhat 9y ago

> You can't easily spawn 10.000 OS threads. Of course you can, without any problem at all. It's just more expensive than a green thread (they're full processes with stacks), and unlike green threads, it affects how processing time is sliced. It is, however, much cheaper than people tend to believe. > Cooperative is less simple to reason about, as the proposal explains. Cooperative scheduling is very simple to reason about. In the simplest form, you do not give up control unless you use I/O, a synchronization primitive, or explicitly give up control (runtime.Gosched() in Go). As a Go developer, and as a developer in any other language, this is exactly what I expect from any green thread implementation. That's how M:N scheduling has always been implemented, and how green threads have been implemented in other languages. Technically, preemptive is harder to reason about, but you save having to reason about it. However, if you expect preemption from goroutines, you have misunderstood the primitive.

ngrilly 9y ago
> In the simplest form, you do not give up control unless you use I/O, a synchronization primitive, or explicitly give up control (runtime.Gosched() in Go). This is not how Go works nowadays. There are other preemption points.
- arghwhat 9y ago
  That's why I said "in the simplest form".
  - ngrilly 9y ago
    Yes, but it completely ruins the argument about "it's easy to reason about" ;-)

pcwalton 9y ago

> You can't easily spawn 10.000 OS threads. Of course you can. Go on, compile and run this program: #include <pthread.h> #include <stdlib.h> void *f(void *arg) { pthread_mutex_lock((pthread_mutex_t *)arg); pthread_mutex_unlock((pthread_mutex_t *)arg); return NULL; } int main() { pthread_t threads[10000]; pthread_mutex_t mutexes[10000]; for (unsigned i = 0; i < 10000; i++) { pthread_mutex_init(&mutexes[i], NULL); pthread_mutex_lock(&mutexes[i]); pthread_create(&threads[i], NULL, f, &mutexes[i]); } for (unsigned i = 0; i < 10000; i++) pthread_mutex_unlock(&mutexes[i]); for (unsigned i = 0; i < 10000; i++) pthread_join(threads[i], NULL); return 0; } Executes in a second for me, even on macOS.