5 ms·

This may be kind of a dumb question, but does Go use the stdlib C stack or are they talking about their own stack? I'm assuming that Go has a standard / vanill

by mox1 12y ago

This may be kind of a dumb question, but does Go use the stdlib C stack or are they talking about their own stack? I'm assuming that Go has a standard / vanilla / kernel aware C-style stack living somewhere right? Then the stack they are referring to is part of their run time environment...which is actually allocated on the heap somewhere. This is all so meta. :)

ufo 12y ago
Its not that surprising. If you want full coroutines that can yield from inside function calls then each coroutine is going to need to own a separate stack. Since you only have one C stack the coroutine stacks need to live on the heap. Some coroutine implementations copy the coroutine stacks back into the main C stack before resuming execution but usually its not worth the trouble to do that and you just run everything on the heap.
barrkel 12y ago
If you don't have any C in your executable or its libraries, then you don't need to have any C stack. For the purpose of system calls, the stack is just the memory pointed to by the stack pointer. It could be anywhere - heap, bss, a subrange of an existing stack allocation. If you want to call into C code (it's rather unrealistic not to have any C in any of your library dependencies), then you need to make a C-style stack available, following the conventions of the system (e.g. growing downwards vs upwards, using a guard page for incremental allocation, etc.) along with enough reserve for typical C maximum call stack depth. It needs to be set up before the call to C, but it doesn't need to hang around when C code is not in the chain of current procedure activations. The requirements of the stuff pointed to by the stack pointer for it to be considered stack is defined by the platform ABI, which in turn is heavily dependent on CPU architecture. But as long as you meet the rules, you're fine, with regard to the OS. Third party code may be more sniffy.
masklinn 12y ago
> This may be kind of a dumb question, but does Go use the stdlib C stack or are they talking about their own stack? Their own. > I'm assuming that Go has a standard / vanilla / kernel aware C-style stack living somewhere right? Not unless it has to call into C, which would require a C-style stack. > Then the stack they are referring to is part of their run time environment...which is actually allocated on the heap somewhere. Yes, as it is in C really. The difference is that the C stack is bounded and "fairly limited" (and highly variable, from 16k to a few MB[0]). [0] http://lists.gnu.org/archive/html/bug-coreutils/2009-10/msg00262.html http://lists.gnu.org/archive/html/bug-coreutils/2009-10/msg0...
- 4ad 12y ago
  > Yes, as it is in C really. Not really. What people understand by "the C stack" is the stack that automatically comes when new threads are created. This stack is not on the heap; in fact it's at the top of the user address space and it has some special properties, like the operating system setting guard pages for you. Pthreads allow you to set your own stack for a thread, but it is a feature seldom used; though it used by Go when cgo is employed.
  - rsc 12y ago
    Go does not choose the thread stack when using pthreads. Pthreads does.
    - 4ad 12y ago
      You're right, I was confused for a moment. Go doesn't do this. It is possible though, see pthread_attr_setstackaddr.
4ad 12y ago
There is no such thing as a "stdlib" stack. There is a thread stack, but with the exception of the initial thread, all Go stacks (both for Go code and for the scheduler) are allocated from "the heap" (rather Go is also responsible for mapping heap-like memory). That is, on anything except Windows, Solaris and Plan 9, where the stack given by the operating system is used for the scheduler stack, but user goroutine stacks are always allocated from the heap.
- rsc 12y ago
  Not true. System stacks are system allocated. See my reply above.
  - 4ad 12y ago
    Thanks for your note, but I'm confused by your statement. Take Linux go/src/runtime/os_linux.c:/^runtime·newosproc. It passes the already-allocated stack to clone(2). runtime·newosproc is called from go/src/runtime/proc.c:/^newm, which uses runtime·allocm to allocate a M. go/src/runtime/proc.c:/^runtime·allocm allocates this memory using runtime·malg on everything except Solaris, Windows, Plan 9, or when using cgo. The comment even reads: // In case of cgo or Solaris, pthread_create will make us a stack. // Windows and Plan 9 will layout sched stack on OS stack. For that class of systems, runtime·newosproc (or _cgo_sys_thread_start) indeed does use the system stacks, but that's what I said too. Let's take a look at runtime·malg again. runtime·malg calls go/src/runtime/stack.c:^/runtime·stackalloc which uses runtime·sysAlloc to allocate memory. Moving back to Linux, go/src/runtime/mem_linux.c:/^runtime·sysAlloc just calls mmap(2). So, the answer to "who allocates system stacks" seems to be "it depends", but I already qualified this in my original answer (omitting cgo; sorry about that). Now, is there anything wrong in my description?
    - rsc 12y ago
      It is true that when cgo and therefore pthreads is not involved, Go allocates all the stacks. But then the question of where 'system code' runs is not terribly relevant: it doesn't.
wahern 12y ago
Just to add what others have said, one of the Go developers has added support to GCC for segmented stacks with the -fsplit-stack flag. And it appears clang also supports this. The question remains how Go's new runtime deals with calling out to third-party libraries that don't obey their new stack protocol.
- rsc 12y ago
  The same way the old runtime did: call out on the pthread-provided stack.
  - wahern 12y ago
    Ah, right, Go is M:N threaded, so the scheduler runs multiple goroutines on a shared POSIX stack, mostly unused until calling out to C code. Presumably, then, only Go code can yield.

by mox1 12y ago

ufo 12y ago

Its not that surprising. If you want full coroutines that can yield from inside function calls then each coroutine is going to need to own a separate stack. Since you only have one C stack the coroutine stacks need to live on the heap. Some coroutine implementations copy the coroutine stacks back into the main C stack before resuming execution but usually its not worth the trouble to do that and you just run everything on the heap.

barrkel 12y ago

If you don't have any C in your executable or its libraries, then you don't need to have any C stack. For the purpose of system calls, the stack is just the memory pointed to by the stack pointer. It could be anywhere - heap, bss, a subrange of an existing stack allocation. If you want to call into C code (it's rather unrealistic not to have any C in any of your library dependencies), then you need to make a C-style stack available, following the conventions of the system (e.g. growing downwards vs upwards, using a guard page for incremental allocation, etc.) along with enough reserve for typical C maximum call stack depth. It needs to be set up before the call to C, but it doesn't need to hang around when C code is not in the chain of current procedure activations. The requirements of the stuff pointed to by the stack pointer for it to be considered stack is defined by the platform ABI, which in turn is heavily dependent on CPU architecture. But as long as you meet the rules, you're fine, with regard to the OS. Third party code may be more sniffy.

masklinn 12y ago

> This may be kind of a dumb question, but does Go use the stdlib C stack or are they talking about their own stack? Their own. > I'm assuming that Go has a standard / vanilla / kernel aware C-style stack living somewhere right? Not unless it has to call into C, which would require a C-style stack. > Then the stack they are referring to is part of their run time environment...which is actually allocated on the heap somewhere. Yes, as it is in C really. The difference is that the C stack is bounded and "fairly limited" (and highly variable, from 16k to a few MB[0]). [0] http://lists.gnu.org/archive/html/bug-coreutils/2009-10/msg00262.html http://lists.gnu.org/archive/html/bug-coreutils/2009-10/msg0...

4ad 12y ago
> Yes, as it is in C really. Not really. What people understand by "the C stack" is the stack that automatically comes when new threads are created. This stack is not on the heap; in fact it's at the top of the user address space and it has some special properties, like the operating system setting guard pages for you. Pthreads allow you to set your own stack for a thread, but it is a feature seldom used; though it used by Go when cgo is employed.
- rsc 12y ago
  Go does not choose the thread stack when using pthreads. Pthreads does.
  - 4ad 12y ago
    You're right, I was confused for a moment. Go doesn't do this. It is possible though, see pthread_attr_setstackaddr.

4ad 12y ago

There is no such thing as a "stdlib" stack. There is a thread stack, but with the exception of the initial thread, all Go stacks (both for Go code and for the scheduler) are allocated from "the heap" (rather Go is also responsible for mapping heap-like memory). That is, on anything except Windows, Solaris and Plan 9, where the stack given by the operating system is used for the scheduler stack, but user goroutine stacks are always allocated from the heap.

rsc 12y ago
Not true. System stacks are system allocated. See my reply above.
- 4ad 12y ago
  Thanks for your note, but I'm confused by your statement. Take Linux go/src/runtime/os_linux.c:/^runtime·newosproc. It passes the already-allocated stack to clone(2). runtime·newosproc is called from go/src/runtime/proc.c:/^newm, which uses runtime·allocm to allocate a M. go/src/runtime/proc.c:/^runtime·allocm allocates this memory using runtime·malg on everything except Solaris, Windows, Plan 9, or when using cgo. The comment even reads: // In case of cgo or Solaris, pthread_create will make us a stack. // Windows and Plan 9 will layout sched stack on OS stack. For that class of systems, runtime·newosproc (or _cgo_sys_thread_start) indeed does use the system stacks, but that's what I said too. Let's take a look at runtime·malg again. runtime·malg calls go/src/runtime/stack.c:^/runtime·stackalloc which uses runtime·sysAlloc to allocate memory. Moving back to Linux, go/src/runtime/mem_linux.c:/^runtime·sysAlloc just calls mmap(2). So, the answer to "who allocates system stacks" seems to be "it depends", but I already qualified this in my original answer (omitting cgo; sorry about that). Now, is there anything wrong in my description?
  - rsc 12y ago
    It is true that when cgo and therefore pthreads is not involved, Go allocates all the stacks. But then the question of where 'system code' runs is not terribly relevant: it doesn't.

wahern 12y ago

Just to add what others have said, one of the Go developers has added support to GCC for segmented stacks with the -fsplit-stack flag. And it appears clang also supports this. The question remains how Go's new runtime deals with calling out to third-party libraries that don't obey their new stack protocol.

rsc 12y ago
The same way the old runtime did: call out on the pthread-provided stack.
- wahern 12y ago
  Ah, right, Go is M:N threaded, so the scheduler runs multiple goroutines on a shared POSIX stack, mostly unused until calling out to C code. Presumably, then, only Go code can yield.