3 ms·

> I do not think Nim code can run "in a goroutine". It does. Don't forget that this is gccgo so it is possible to use plain C functions as goroutines. Nim is t

> I do not think Nim code can run "in a goroutine". It does. Don't forget that this is gccgo so it is possible to use plain C functions as goroutines. Nim is translated to C and with the help of a macro I convert Nim functions with an arbitrary number of arguments into ones with a single void* arg that gccgo wants for its 'go' keyword implementation: extern void* __go_go(void (*f)(void *), void *);

jeremyjh 11y ago
Yes, but when the Go runtime calls that foreign function pointer, it is going to schedule an entire OS thread for its duration isn't it? If it doesn't, then nothing prevents foreign code from blocking other goroutines. How does the Nim code yield the thread for other goroutines, does it have to register a callback?
- stefantalpalaru 11y ago
  > when the Go runtime calls that foreign function pointer, it is going to schedule an entire OS thread for its duration isn't it? No. > How does the Nim code yield the thread for other goroutines, does it have to register a callback? There are no callbacks. Yielding happens automatically when launching another goroutine, when sending, receiving or selecting on a channel. You can also yield explicitly with go_yield() - the better named equivalent of Go's runtime.Gosched(). It's easier to understand if you realize that all those operations with goroutines and channels end up being done in the Go runtime.
  - nulltype 11y ago
    How does it handle making a blocking socket call in a go goroutine?
    - stefantalpalaru 11y ago
      It doesn't, yet.
  - jeremyjh 11y ago
    > > when the Go runtime calls that foreign function pointer, it is going to schedule an entire OS thread for its duration isn't it? >No. Are you sure? Once a thread enters cgo it is considered blocked and is removed from the thread pool according to these sources [1][2]. I previously found a thread where Ian Lance Taylor explained it more explicitly but I couldn't find that now. Is that not what is happening though when __go_go invokes your function pointer? I do not understand how the Nim code can live in the segmented stack of a goroutine, nor how the Go runtime could know it is time to grow that stack. [1] https://groups.google.com/forum/#!searchin/golang-nuts/goroutine$20OS$20thread$20C/golang-nuts/RTtMsgZi88Q/To5F9wWnpU8J https://groups.google.com/forum/#!searchin/golang-nuts/gorou... [2]http://stackoverflow.com/questions/27600587/why-my-program-of-golang-create-so-many-threads http://stackoverflow.com/questions/27600587/why-my-program-o...
    - stefantalpalaru 11y ago
      > Are you sure? Yes. See the chinese whispers benchmark with 500000 goroutines and a maximum resident set size of 5.4 GB on amd64. It has the same memory usage (and run time) as the Go version compiled with gccgo. > Once a thread enters cgo This has nothing to do with cgo. It's a different mechanism specific to gccgo. > I do not understand how the Nim code can live in the segmented stack of a goroutine Good thing you asked. I just ported to Nim the peano.go benchmark described as a "torture test for segmented stacks" and... it failed. The fix was to pass -fsplit-stack to gcc when it compiles the C code generated by nim. > nor how the Go runtime could know it is time to grow that stack From what I can tell it's done in __splitstack_makecontext() and friends from GCC's libgo.
      - jeremyjh 11y ago
        Well, cool! Great work :)

jeremyjh 11y ago

Yes, but when the Go runtime calls that foreign function pointer, it is going to schedule an entire OS thread for its duration isn't it? If it doesn't, then nothing prevents foreign code from blocking other goroutines. How does the Nim code yield the thread for other goroutines, does it have to register a callback?

stefantalpalaru 11y ago
> when the Go runtime calls that foreign function pointer, it is going to schedule an entire OS thread for its duration isn't it? No. > How does the Nim code yield the thread for other goroutines, does it have to register a callback? There are no callbacks. Yielding happens automatically when launching another goroutine, when sending, receiving or selecting on a channel. You can also yield explicitly with go_yield() - the better named equivalent of Go's runtime.Gosched(). It's easier to understand if you realize that all those operations with goroutines and channels end up being done in the Go runtime.
- nulltype 11y ago
  How does it handle making a blocking socket call in a go goroutine?
  - stefantalpalaru 11y ago
    It doesn't, yet.
- jeremyjh 11y ago
  > > when the Go runtime calls that foreign function pointer, it is going to schedule an entire OS thread for its duration isn't it? >No. Are you sure? Once a thread enters cgo it is considered blocked and is removed from the thread pool according to these sources [1][2]. I previously found a thread where Ian Lance Taylor explained it more explicitly but I couldn't find that now. Is that not what is happening though when __go_go invokes your function pointer? I do not understand how the Nim code can live in the segmented stack of a goroutine, nor how the Go runtime could know it is time to grow that stack. [1] https://groups.google.com/forum/#!searchin/golang-nuts/goroutine$20OS$20thread$20C/golang-nuts/RTtMsgZi88Q/To5F9wWnpU8J https://groups.google.com/forum/#!searchin/golang-nuts/gorou... [2]http://stackoverflow.com/questions/27600587/why-my-program-of-golang-create-so-many-threads http://stackoverflow.com/questions/27600587/why-my-program-o...
  - stefantalpalaru 11y ago
    > Are you sure? Yes. See the chinese whispers benchmark with 500000 goroutines and a maximum resident set size of 5.4 GB on amd64. It has the same memory usage (and run time) as the Go version compiled with gccgo. > Once a thread enters cgo This has nothing to do with cgo. It's a different mechanism specific to gccgo. > I do not understand how the Nim code can live in the segmented stack of a goroutine Good thing you asked. I just ported to Nim the peano.go benchmark described as a "torture test for segmented stacks" and... it failed. The fix was to pass -fsplit-stack to gcc when it compiles the C code generated by nim. > nor how the Go runtime could know it is time to grow that stack From what I can tell it's done in __splitstack_makecontext() and friends from GCC's libgo.
    - jeremyjh 11y ago
      Well, cool! Great work :)