5 ms·

I feel this kind of thing should be remembered when people say they're going to reinvent decades-old technology. This bug exists because they wanted to reinven

I feel this kind of thing should be remembered when people say they're going to reinvent decades-old technology. This bug exists because they wanted to reinvent threading so you could have millions of threads, except when you actually use millions of threads for anything non-trivial it doesn't actually work out so well. To do that they couldn't use the standard since-1970 fixed stacks so they needed movable/growable stacks. They reinvented exceptions in a way that broke selinux. They reinvented the memory allocator in a way that breaks ulimit. They reinvented a javadoc/doxygen that's less useful. It's great to try to improve things, but a lot of times there's actually very good reasons for the old ways.

rounce 10y ago
Are you kidding? How is `go doc` anything like Javadoc aside from being automatically generated by machine from your sources. Might as well compare it to Jekyll.
- gravypod 10y ago
  https://docs.oracle.com/javase/8/docs/api/java/util/List.html#size-- https://docs.oracle.com/javase/8/docs/api/java/util/List.htm... https://golang.org/pkg/container/list/#List.Len https://golang.org/pkg/container/list/#List.Len
  - rounce 10y ago
    Note the backticks, I was referring to the `go doc` command built into the standard tooling. There are many generated HTML documentation generation tools for every language. It's Golang's general approach to documentation which many find appealing, not the fact they can output to HTML.
    - gravypod 10y ago
      My tooling does this too and I don't even need to remember a terminal command: http://i.imgur.com/YmAkBPG.png http://i.imgur.com/YmAkBPG.png I get what your saying but it's apples to oranges. They are in effect similar though.
      - rounce 10y ago
        I (and I guess many others) enjoy that Golang's documentation strategy is coupled far tighter to the language (almost as a component of the language), in contrast to JavaDoc's existence as more of a standard for writing comments, `javadoc` being the first of many implementations and variations on it. I'm not saying this is in any way empirically superior, in fact I think Go's approach owes a lot to Sun's original vision with JavaDoc. Some just prefer it and feel it's approach benefits them, and it's a preference that is often factored in when making the choice to use it. My original point was that Sun's approach is now so commonplace it's an 'apples and oranges' argument. Golang has made a legitimate step forward (I think), in how a language treats and encourages it's documentation: As a first class citizen from day 0, rather than a convenient afterthought. The parity of experience between godoc.org & `go doc` demonstrates this with ample effect.
_ph_ 10y ago
Go programs have roughly one active thread per CPU in the computer. I don't think they claimed they reinvented threading, they used a (known) model, where many green threads are mapped to hardware threads and added the primitives to the language. To make millions of goroutines on one machine feasible, they need small default stacks - 2kb currently.
nemothekid 10y ago
I'm not sure what you are implying here. Should we give up on creating a language that can have millions of threads because we might break selinux? Should we give up on programs with millions of threads because someone in the 1970's never foresaw that use case?
- pcwalton 10y ago
  > Should we give up on programs with millions of threads because someone in the 1970's never foresaw that use case? It's not that simple. I think there's a reasonable first-principles argument to be made that scheduling belongs in the kernel, which has a global view of the system and is better equipped to make those decisions.
  - ben0x539 10y ago
    Some Go users can probably make the argument that their one Go application is the single process using appreciable amounts of CPU time on their systems.
    - ben0x539 10y ago
      Well, or some Erlang users, I guess, fine
  - felixgallo 10y ago
    The kernel is necessarily so general purpose that it's hardly equipped at all to make correct, performant scheduling decisions at the app level.
  - toast0 10y ago
    I use erlang, not go, but it does similar things. On most of our production machines, the only OS process dong real work is the erlang VM, other processes are just things like crond, ntpd, sshd, syslogd; so all the kernel scheduler has to do is run the erlang OS threads (and they're pined to individual CPUs). Maybe the kernel scheduler would do a better job, but is it realistic to run 1M OS threads? It feels like the overhead per green thread is much lower, and the potential for being interrupted in a critical section is also much lower: An OS thread may be paused while holding a (vm internal) lock, but the vm scheduler would not preempt a green thread in the middle of doing something that requires a lock (ex: Delivering a message to another green thread)
sfifs 10y ago
A lot of the choices for Go become clearer when you realize it was primarily built to solve problems of servers and code base at scale. It supports millions of co-routines, not threads. There's typically one thread per CPU or so. The key insight is a lot of server type workloads primarily are constrained on I/O and multiplexing virtually unlimited co-routines on a thread automatically by the runtime makes for much simpler concurrent server code. Implementing high concurrency server code in almost any other mainstream language is non-trivial. Indeed I originally switched from Python to Go primarily because of that.
slrz 10y ago
Do you have a pointer to that SELinux issue? Can't remember anything related to exceptions. I think there was some issue with runtime code generation (used with closures IIRC) very early on but that's ancient history by now.

rounce 10y ago

Are you kidding? How is `go doc` anything like Javadoc aside from being automatically generated by machine from your sources. Might as well compare it to Jekyll.

gravypod 10y ago
https://docs.oracle.com/javase/8/docs/api/java/util/List.html#size-- https://docs.oracle.com/javase/8/docs/api/java/util/List.htm... https://golang.org/pkg/container/list/#List.Len https://golang.org/pkg/container/list/#List.Len
- rounce 10y ago
  Note the backticks, I was referring to the `go doc` command built into the standard tooling. There are many generated HTML documentation generation tools for every language. It's Golang's general approach to documentation which many find appealing, not the fact they can output to HTML.
  - gravypod 10y ago
    My tooling does this too and I don't even need to remember a terminal command: http://i.imgur.com/YmAkBPG.png http://i.imgur.com/YmAkBPG.png I get what your saying but it's apples to oranges. They are in effect similar though.
    - rounce 10y ago
      I (and I guess many others) enjoy that Golang's documentation strategy is coupled far tighter to the language (almost as a component of the language), in contrast to JavaDoc's existence as more of a standard for writing comments, `javadoc` being the first of many implementations and variations on it. I'm not saying this is in any way empirically superior, in fact I think Go's approach owes a lot to Sun's original vision with JavaDoc. Some just prefer it and feel it's approach benefits them, and it's a preference that is often factored in when making the choice to use it. My original point was that Sun's approach is now so commonplace it's an 'apples and oranges' argument. Golang has made a legitimate step forward (I think), in how a language treats and encourages it's documentation: As a first class citizen from day 0, rather than a convenient afterthought. The parity of experience between godoc.org & `go doc` demonstrates this with ample effect.

_ph_ 10y ago

Go programs have roughly one active thread per CPU in the computer. I don't think they claimed they reinvented threading, they used a (known) model, where many green threads are mapped to hardware threads and added the primitives to the language. To make millions of goroutines on one machine feasible, they need small default stacks - 2kb currently.

nemothekid 10y ago

I'm not sure what you are implying here. Should we give up on creating a language that can have millions of threads because we might break selinux? Should we give up on programs with millions of threads because someone in the 1970's never foresaw that use case?

pcwalton 10y ago
> Should we give up on programs with millions of threads because someone in the 1970's never foresaw that use case? It's not that simple. I think there's a reasonable first-principles argument to be made that scheduling belongs in the kernel, which has a global view of the system and is better equipped to make those decisions.
- ben0x539 10y ago
  Some Go users can probably make the argument that their one Go application is the single process using appreciable amounts of CPU time on their systems.
  - ben0x539 10y ago
    Well, or some Erlang users, I guess, fine
- felixgallo 10y ago
  The kernel is necessarily so general purpose that it's hardly equipped at all to make correct, performant scheduling decisions at the app level.
- toast0 10y ago
  I use erlang, not go, but it does similar things. On most of our production machines, the only OS process dong real work is the erlang VM, other processes are just things like crond, ntpd, sshd, syslogd; so all the kernel scheduler has to do is run the erlang OS threads (and they're pined to individual CPUs). Maybe the kernel scheduler would do a better job, but is it realistic to run 1M OS threads? It feels like the overhead per green thread is much lower, and the potential for being interrupted in a critical section is also much lower: An OS thread may be paused while holding a (vm internal) lock, but the vm scheduler would not preempt a green thread in the middle of doing something that requires a lock (ex: Delivering a message to another green thread)

sfifs 10y ago

A lot of the choices for Go become clearer when you realize it was primarily built to solve problems of servers and code base at scale. It supports millions of co-routines, not threads. There's typically one thread per CPU or so. The key insight is a lot of server type workloads primarily are constrained on I/O and multiplexing virtually unlimited co-routines on a thread automatically by the runtime makes for much simpler concurrent server code. Implementing high concurrency server code in almost any other mainstream language is non-trivial. Indeed I originally switched from Python to Go primarily because of that.

slrz 10y ago

Do you have a pointer to that SELinux issue? Can't remember anything related to exceptions. I think there was some issue with runtime code generation (used with closures IIRC) very early on but that's ancient history by now.