9 ms·

> That said, I think Node is not the best system to build a massive server web. I would definitely use Go for that. And honestly, that's basically the reason wh

by s_kilk 9y ago

> That said, I think Node is not the best system to build a massive server web. I would definitely use Go for that. And honestly, that's basically the reason why I left Node. It was the realization that: oh, actually, this is not the best server side system ever. Really interesting. I can imagine others would refuse to give up on the thing they'd worked so hard on. But Dahl has the self-awareness to just step back and say "huh, guess this isn't so great after all". Huge props to the guy.

jgrahamc 9y ago
Agree. It takes a lot to walk away from something and say you were wrong. But, sadly, part of the saga of nodejs could have been avoided by people looking at the history of computing. If you look at Go's concurrency model it was entirely mapped out in the 1970s in CSP. There wasn't really any need for go the 'single thread, non-blocking' route that nodejs took and evangelize it as nirvana. There was a ton of distributed systems work done in the 1970s. Just look at Lamport's clocks paper. I'm not criticizing Dahl directly here, more a sense that people ignore history in a field that has only had history for far less than a century.
- spraak 9y ago
  It is clear however that JS itself has benefited from Node in that it really increased attention on the language. It seems doubtful we'd have ES6 etc. without it
- jrs95 9y ago
  While you are sort of right, the non-blocking thing was basically necessary to get JS on the server working. And while he was focused on the I/O thing, he accomplished a lot of things by accident, including making a really convenient server platform for JS.
  - thehardsphere 9y ago
    Right, but the whole reason for JS on the server was because JS had no I/O, so he could do the non-blocking thing. JS on the server was the means, not the end.
- rpeden 9y ago
  It might be because it's relatively easy to end up doing something computing related without studying it directly. This was the case for Dahl and many others. It's definitely possible to catch up on the history of the field while you're working in it, but not everyone can find the time to do so.
- peterbraden 9y ago
  While I agree that non-blocking isn't a panacea, I think that your attitude that it was an unnecessary experiment is a little dismissive. The fact is that green threads aren't a panacea either, and exploring the callback based pattern led to some really interesting development, evidenced by the explosion of javascript on the server . At the time that node took off, a huge amount of server code was python with the GIL, and there needed to be something to shake up the status quo.
- deleted 9y ago
  [deleted]
- amasad 9y ago
  The fact that Node survives as a platform for Fullstack JavaScript shows how serendipity works in invention: he set out to build a high performance network server and ended up building one of the largest platforms for building web applications. The reasons it ended being very big are clear in hindsight but are hard to predict: - Sharing code between server and client - Server side rendering - Lower barrier to entry than other platforms (learn once write anywhere) - Well-designed package system It was also helped by the fact that folks decided to write JS tooling in JS, be it minifier, transpilers, or test runners, without nodejs we would've built then on top of something else.
  - thehardsphere 9y ago
    What do people mean when they keep saying "server side rendering" in this specific Javascript/Node.js context? I ask because I've seen quite a few people tout that term as a recent innovation. It was my understanding that "server side rendering" was the way the web worked since the beginning, with the server generating the markup that is provided to the browser to render the page. Nobody called it "server side rendering" until we had "client side rendering", where Javascript executed in the browser provided the markup instead of whatever program was running on the server. I'm not trying to lecture or be snarky here; I'm genuinely confused as to what people mean when they say "server side rendering" as a new thing. Like, is there something novel that I'm missing because the term is overloaded with a definition I don't know?
    - amasad 9y ago
      It means you take your client side JavaScript code and render it on the server for performance and SEO. Previously you had to either duplicate the code in different languages or pick one or the other (either use angular or rails).
      - steveklabnik 9y ago
        Not just that, but to accomplish it means that you need to provide the correct client-side APIs on the server side, so that your code renders the same in both places. Doing this nicely is a lot of work.
- erokar 9y ago
  > If you look at Go's concurrency model it was entirely mapped out in the 1970s Isn't that Go's problem, though — that there isn't a single thing in the language that wasn't entierly mapped out in the 1970s?
  - geodel 9y ago
    It is only a problem to someone whose thought process dictates anything old is bad and anything new is good.
    - camus2 9y ago
      > It is only a problem to someone whose thought process dictates anything old is bad and anything new is good. What do you call new? generic programming, nullable types, type classes? all pioneered around 1973? Can you explain what "new" is?
  - spraak 9y ago
    Using ideas from history isn't a problem. And Go is unique in that it combined all these learned lessons into something new.
    - camus2 9y ago
      > Using ideas from history isn't a problem. And Go is unique in that it combined all these learned lessons into something new. what learned lessons, like pragmas and struct tags?
      - jerf 9y ago
        A lot of is more in what was taken away rather than what was added. As someone who is occasionally accused of being too pro-Go on HN... I actually have no problems thinking of Go as a really nice and refined 1990s language. Another type of "Java done right". (I say "another" because I think C# has a really good claim to that as well, albeit in a very different direction.) There's a place for that in the world. As much as I love the cutting edge of programming languages too, I'm not sure that we're anywhere near as far along on knowing how to build really big systems with them as a lot of people think we are. (There's only one way to get there though, and I absolutely encourage people to keep trying.)
- willvarfar 9y ago
  You couldn't have got CSP from writing a web server API to run with V8. To get CSP you needed green threading built into the runtime, and that's a much taller order. So it's hard to say that the not-critizing-directly Dahl should have learned from the past. We'd probably never have heard of him if he'd set out to build a new esoteric CSP language or runtime, just like we haven't heard of anyone around that time other than the golang team.
- bbatha 9y ago
  > But, sadly, part of the saga of nodejs could have been avoided by people looking at the history of computing. If you look at Go's concurrency model it was entirely mapped out in the 1970s in CSP. There wasn't really any need for go the 'single thread, non-blocking' route that nodejs took and evangelize it as nirvana. There was a ton of distributed systems work done in the 1970s. Just look at Lamport's clocks paper. Non-Blocking IO is still the best we have because its a limitation in syscall interfaces. Node.js, nginx, go, etc all use epoll and their ilk under the hood. In fact, in go goroutines only ran by default on a single OS thread until go 1.5. The ergonomics of call-back based non-blocking io is the issue here, not non-blocking io itself. Because of javascript's history on the browser where callback based events are the only apis it made sense to copy this style of api for IO so stuff like `setTimeout` worked in both cases. Node's selling point with regard to non-blocking io was never about ergonomics, it was the fact that you had a dynamic scripting language where the entire ecosystem was using non-blocking io using the same standardized event loop. Contrast to the other popular "web" languages of the day, Ruby and Python which have had some non-blocking io for years but usage is isolated because most of the ecosystem doesn't support it at all, and when they do they completely different and incompatible low-level io stacks.
- TheAceOfHearts 9y ago
  I agree that we regularly fail to learn from history. However, it's worth recognizing that trying to learn from history can be incredibly challenging. There's a mountains of research to filter through. Sometimes you discover a relevant document, only to find out it's not readily available or that you have to pay an exorbitant amount of money for access. You might pay a lot of money for something that doesn't even end up being useful to your problem. When there have been multiple attempts at solving the problem, each with mixed results, how do you actually identify the good parts from the bad ones? A lot of tech knowledge isn't easily discoverable. Heck, sometimes things don't even make sense without understanding their historical background. The filesystem layout in the unixes is a great example, on my system I have: /bin, /sbin, /usr/bin, /usr/sbin, /usr/local/bin, /usr/local/sbin.
  - komali2 9y ago
    A good example - I have no idea why unix filesystems are like that, and after some idle googling I still don't know. Can't even figure out where to start to find the history of those decisions. Certainly am finding what kind of stuff goes where.
    - nomel 9y ago
      Some old Unix devs/users ran out of disk space, so they mounted more disk to those points. This caused boot problems since some tools (like mount) needed to be available. All of that together, along with people blindly following convention, and you have the mess of the unix filesystem. Yep... [1] [1] http://lists.busybox.net/pipermail/busybox/2010-December/074114.html http://lists.busybox.net/pipermail/busybox/2010-December/074...
      - komali2 9y ago
        oh my god, that's awful
- zzzcpan 9y ago
  Lamport's paper was only the very beginning of distributed systems. And CSP was invented by mathematicians for themselves, not for real life programming. As opposed to Erlang, which was invented for real life programming of telephony applications later in 1980s. But people keep ignoring history and think that CSP is suitable somehow, and not actor model. CSP is an even bigger mistake, don't try to make it look like it isn't. Learn from history.
- pcwalton 9y ago
  More than that. Go's model is just threads with a particularly idiosyncratic implementation, in userspace instead of in the kernel. This itself is nothing new, as it was tried by the Linux NGPT project in the early '90s. (NGPT was abandoned for being inferior to plain old 1:1 threads, which suggests that Go's approach is not the end state either.)
  - jabl 9y ago
    AFAIU a lot of the complexity and slowness of "traditional" M:N threading approaches were due to the requirement to follow POSIX semantics (signal handling, preemptivity, etc.) within a purely library/OS based approach (no changes in the code generated by the compiler). If you're doing a green threads implementation for a high level language runtime, those restrictions don't apply, to an extent. I think e.g. Erlang, Haskell or Go are examples of "green threads done right". (Now, I do think Rust did the right thing in getting rid of green threads, but that IMHO is more a result of the space Rust is in rather than a general indictment on the utility of green threads)
    - pcwalton 9y ago
      Go still has a lot of issues around the inability to preempt except at function call boundaries, which is an issue that doesn't exist in 1:1 threading. Fairness and priority inversion are likewise issues with M:N that apply equally well to Go's implementation. Signal handling is too, although any GC language pretty much has to sacrifice POSIX signal handling already... Green threads done right strikes me as something like Windows' user mode scheduling (or Google's switchto patch that sadly never made it upstream), in which threads really are 1:1 as far as the kernel is concerned, but manually scheduled by userspace. This requires kernel support, but it fixes every issue except preemption, which is better to just not fix. (Few Go programs actually benefit from multicore CPU scaling; for CPU bound tasks a better use of optimization time is just not writing in Go, which tends to result in better speedups than trying to scale Go due to Go's compiler being relatively immature.) In my ideal world, kernel-scheduled and user-scheduled threads would exist in the same language, and programmers could choose which one they want on a thread-by-thread basis. I/O calls would be equally compatible with either threading model (which could be done in a zero-overhead fashion with the proper kernel support), eliminating the problem of sync/async incompatibility. This can only really be done today on Windows, unfortunately...
gggboxing 9y ago
That was me and my thesis. I defended it, graduated, and the next day I was like "I don't want to ever look at this pile of trash again".
devmunchies 9y ago
Well he does work for Google. We don't know if there is any pressure for him to be be a Govangelist.
- spraak 9y ago
  I see what you mean, but as a similar example, I never thought anyone at Google wouldn't use Google for search, but some use DuckDuckGo -- https://news.ycombinator.com/item?id=15068047 https://news.ycombinator.com/item?id=15068047 Plus, he said he was using Go before he joined Google.
- pier25 9y ago
  TJ moved to Go too, and he doesn't work at Google.
  - devmunchies 9y ago
    Was I defending Node.js or something? Just stating the obvious.
tedunangst 9y ago
I think that happens fairly often. The first person to solve the problem learns a lot in the process. The people who come later don't see the problem, only the solution.
jondubois 9y ago
He doesn't give any specific reason why he thinks that Go is better. I've used both Go and Node.js and I came to the opposite conclusion. I'm not a huge fan of Goroutines spawning threads in the background. I think that spawning threads and processes should be explicit because there is a big performance penalty when multiple threads have to share a CPU core because of context switching. With Node.js, you have more control over the process count so you can minimize the amount of CPU context switching that happens by making sure that each process gets its own CPU core. That said, I do think that on a conceptual level, Go feels cleaner than Node.js... But since async/await was introduced I feel that Node.js has the upper hand again.
- ploxiln 9y ago
  Well the thing is, Goroutines are not threads, they're coroutines. The go runtime uses up to a real thread per cpu core, and all your goroutines are run on those (fewer) real threads. The runtime also manages a shared thread pool for blocking syscalls e.g. local disk access and some kinds of name resolution depending on OS. But starting 10 goroutines is much cheaper than spawning 10 threads.

by s_kilk 9y ago

jgrahamc 9y ago

Agree. It takes a lot to walk away from something and say you were wrong. But, sadly, part of the saga of nodejs could have been avoided by people looking at the history of computing. If you look at Go's concurrency model it was entirely mapped out in the 1970s in CSP. There wasn't really any need for go the 'single thread, non-blocking' route that nodejs took and evangelize it as nirvana. There was a ton of distributed systems work done in the 1970s. Just look at Lamport's clocks paper. I'm not criticizing Dahl directly here, more a sense that people ignore history in a field that has only had history for far less than a century.

spraak 9y ago
It is clear however that JS itself has benefited from Node in that it really increased attention on the language. It seems doubtful we'd have ES6 etc. without it
jrs95 9y ago
While you are sort of right, the non-blocking thing was basically necessary to get JS on the server working. And while he was focused on the I/O thing, he accomplished a lot of things by accident, including making a really convenient server platform for JS.
- thehardsphere 9y ago
  Right, but the whole reason for JS on the server was because JS had no I/O, so he could do the non-blocking thing. JS on the server was the means, not the end.
rpeden 9y ago
It might be because it's relatively easy to end up doing something computing related without studying it directly. This was the case for Dahl and many others. It's definitely possible to catch up on the history of the field while you're working in it, but not everyone can find the time to do so.
peterbraden 9y ago
While I agree that non-blocking isn't a panacea, I think that your attitude that it was an unnecessary experiment is a little dismissive. The fact is that green threads aren't a panacea either, and exploring the callback based pattern led to some really interesting development, evidenced by the explosion of javascript on the server . At the time that node took off, a huge amount of server code was python with the GIL, and there needed to be something to shake up the status quo.
deleted 9y ago
[deleted]
amasad 9y ago
The fact that Node survives as a platform for Fullstack JavaScript shows how serendipity works in invention: he set out to build a high performance network server and ended up building one of the largest platforms for building web applications. The reasons it ended being very big are clear in hindsight but are hard to predict: - Sharing code between server and client - Server side rendering - Lower barrier to entry than other platforms (learn once write anywhere) - Well-designed package system It was also helped by the fact that folks decided to write JS tooling in JS, be it minifier, transpilers, or test runners, without nodejs we would've built then on top of something else.
- thehardsphere 9y ago
  What do people mean when they keep saying "server side rendering" in this specific Javascript/Node.js context? I ask because I've seen quite a few people tout that term as a recent innovation. It was my understanding that "server side rendering" was the way the web worked since the beginning, with the server generating the markup that is provided to the browser to render the page. Nobody called it "server side rendering" until we had "client side rendering", where Javascript executed in the browser provided the markup instead of whatever program was running on the server. I'm not trying to lecture or be snarky here; I'm genuinely confused as to what people mean when they say "server side rendering" as a new thing. Like, is there something novel that I'm missing because the term is overloaded with a definition I don't know?
  - amasad 9y ago
    It means you take your client side JavaScript code and render it on the server for performance and SEO. Previously you had to either duplicate the code in different languages or pick one or the other (either use angular or rails).
    - steveklabnik 9y ago
      Not just that, but to accomplish it means that you need to provide the correct client-side APIs on the server side, so that your code renders the same in both places. Doing this nicely is a lot of work.
erokar 9y ago
> If you look at Go's concurrency model it was entirely mapped out in the 1970s Isn't that Go's problem, though — that there isn't a single thing in the language that wasn't entierly mapped out in the 1970s?
- geodel 9y ago
  It is only a problem to someone whose thought process dictates anything old is bad and anything new is good.
  - camus2 9y ago
    > It is only a problem to someone whose thought process dictates anything old is bad and anything new is good. What do you call new? generic programming, nullable types, type classes? all pioneered around 1973? Can you explain what "new" is?
- spraak 9y ago
  Using ideas from history isn't a problem. And Go is unique in that it combined all these learned lessons into something new.
  - camus2 9y ago
    > Using ideas from history isn't a problem. And Go is unique in that it combined all these learned lessons into something new. what learned lessons, like pragmas and struct tags?
    - jerf 9y ago
      A lot of is more in what was taken away rather than what was added. As someone who is occasionally accused of being too pro-Go on HN... I actually have no problems thinking of Go as a really nice and refined 1990s language. Another type of "Java done right". (I say "another" because I think C# has a really good claim to that as well, albeit in a very different direction.) There's a place for that in the world. As much as I love the cutting edge of programming languages too, I'm not sure that we're anywhere near as far along on knowing how to build really big systems with them as a lot of people think we are. (There's only one way to get there though, and I absolutely encourage people to keep trying.)
willvarfar 9y ago
You couldn't have got CSP from writing a web server API to run with V8. To get CSP you needed green threading built into the runtime, and that's a much taller order. So it's hard to say that the not-critizing-directly Dahl should have learned from the past. We'd probably never have heard of him if he'd set out to build a new esoteric CSP language or runtime, just like we haven't heard of anyone around that time other than the golang team.
bbatha 9y ago
> But, sadly, part of the saga of nodejs could have been avoided by people looking at the history of computing. If you look at Go's concurrency model it was entirely mapped out in the 1970s in CSP. There wasn't really any need for go the 'single thread, non-blocking' route that nodejs took and evangelize it as nirvana. There was a ton of distributed systems work done in the 1970s. Just look at Lamport's clocks paper. Non-Blocking IO is still the best we have because its a limitation in syscall interfaces. Node.js, nginx, go, etc all use epoll and their ilk under the hood. In fact, in go goroutines only ran by default on a single OS thread until go 1.5. The ergonomics of call-back based non-blocking io is the issue here, not non-blocking io itself. Because of javascript's history on the browser where callback based events are the only apis it made sense to copy this style of api for IO so stuff like `setTimeout` worked in both cases. Node's selling point with regard to non-blocking io was never about ergonomics, it was the fact that you had a dynamic scripting language where the entire ecosystem was using non-blocking io using the same standardized event loop. Contrast to the other popular "web" languages of the day, Ruby and Python which have had some non-blocking io for years but usage is isolated because most of the ecosystem doesn't support it at all, and when they do they completely different and incompatible low-level io stacks.
TheAceOfHearts 9y ago
I agree that we regularly fail to learn from history. However, it's worth recognizing that trying to learn from history can be incredibly challenging. There's a mountains of research to filter through. Sometimes you discover a relevant document, only to find out it's not readily available or that you have to pay an exorbitant amount of money for access. You might pay a lot of money for something that doesn't even end up being useful to your problem. When there have been multiple attempts at solving the problem, each with mixed results, how do you actually identify the good parts from the bad ones? A lot of tech knowledge isn't easily discoverable. Heck, sometimes things don't even make sense without understanding their historical background. The filesystem layout in the unixes is a great example, on my system I have: /bin, /sbin, /usr/bin, /usr/sbin, /usr/local/bin, /usr/local/sbin.
- komali2 9y ago
  A good example - I have no idea why unix filesystems are like that, and after some idle googling I still don't know. Can't even figure out where to start to find the history of those decisions. Certainly am finding what kind of stuff goes where.
  - nomel 9y ago
    Some old Unix devs/users ran out of disk space, so they mounted more disk to those points. This caused boot problems since some tools (like mount) needed to be available. All of that together, along with people blindly following convention, and you have the mess of the unix filesystem. Yep... [1] [1] http://lists.busybox.net/pipermail/busybox/2010-December/074114.html http://lists.busybox.net/pipermail/busybox/2010-December/074...
    - komali2 9y ago
      oh my god, that's awful
zzzcpan 9y ago
Lamport's paper was only the very beginning of distributed systems. And CSP was invented by mathematicians for themselves, not for real life programming. As opposed to Erlang, which was invented for real life programming of telephony applications later in 1980s. But people keep ignoring history and think that CSP is suitable somehow, and not actor model. CSP is an even bigger mistake, don't try to make it look like it isn't. Learn from history.
pcwalton 9y ago
More than that. Go's model is just threads with a particularly idiosyncratic implementation, in userspace instead of in the kernel. This itself is nothing new, as it was tried by the Linux NGPT project in the early '90s. (NGPT was abandoned for being inferior to plain old 1:1 threads, which suggests that Go's approach is not the end state either.)
- jabl 9y ago
  AFAIU a lot of the complexity and slowness of "traditional" M:N threading approaches were due to the requirement to follow POSIX semantics (signal handling, preemptivity, etc.) within a purely library/OS based approach (no changes in the code generated by the compiler). If you're doing a green threads implementation for a high level language runtime, those restrictions don't apply, to an extent. I think e.g. Erlang, Haskell or Go are examples of "green threads done right". (Now, I do think Rust did the right thing in getting rid of green threads, but that IMHO is more a result of the space Rust is in rather than a general indictment on the utility of green threads)
  - pcwalton 9y ago
    Go still has a lot of issues around the inability to preempt except at function call boundaries, which is an issue that doesn't exist in 1:1 threading. Fairness and priority inversion are likewise issues with M:N that apply equally well to Go's implementation. Signal handling is too, although any GC language pretty much has to sacrifice POSIX signal handling already... Green threads done right strikes me as something like Windows' user mode scheduling (or Google's switchto patch that sadly never made it upstream), in which threads really are 1:1 as far as the kernel is concerned, but manually scheduled by userspace. This requires kernel support, but it fixes every issue except preemption, which is better to just not fix. (Few Go programs actually benefit from multicore CPU scaling; for CPU bound tasks a better use of optimization time is just not writing in Go, which tends to result in better speedups than trying to scale Go due to Go's compiler being relatively immature.) In my ideal world, kernel-scheduled and user-scheduled threads would exist in the same language, and programmers could choose which one they want on a thread-by-thread basis. I/O calls would be equally compatible with either threading model (which could be done in a zero-overhead fashion with the proper kernel support), eliminating the problem of sync/async incompatibility. This can only really be done today on Windows, unfortunately...

gggboxing 9y ago

That was me and my thesis. I defended it, graduated, and the next day I was like "I don't want to ever look at this pile of trash again".

devmunchies 9y ago

Well he does work for Google. We don't know if there is any pressure for him to be be a Govangelist.

spraak 9y ago
I see what you mean, but as a similar example, I never thought anyone at Google wouldn't use Google for search, but some use DuckDuckGo -- https://news.ycombinator.com/item?id=15068047 https://news.ycombinator.com/item?id=15068047 Plus, he said he was using Go before he joined Google.
pier25 9y ago
TJ moved to Go too, and he doesn't work at Google.
- devmunchies 9y ago
  Was I defending Node.js or something? Just stating the obvious.

tedunangst 9y ago

I think that happens fairly often. The first person to solve the problem learns a lot in the process. The people who come later don't see the problem, only the solution.

jondubois 9y ago

He doesn't give any specific reason why he thinks that Go is better. I've used both Go and Node.js and I came to the opposite conclusion. I'm not a huge fan of Goroutines spawning threads in the background. I think that spawning threads and processes should be explicit because there is a big performance penalty when multiple threads have to share a CPU core because of context switching. With Node.js, you have more control over the process count so you can minimize the amount of CPU context switching that happens by making sure that each process gets its own CPU core. That said, I do think that on a conceptual level, Go feels cleaner than Node.js... But since async/await was introduced I feel that Node.js has the upper hand again.

ploxiln 9y ago
Well the thing is, Goroutines are not threads, they're coroutines. The go runtime uses up to a real thread per cpu core, and all your goroutines are run on those (fewer) real threads. The runtime also manages a shared thread pool for blocking syscalls e.g. local disk access and some kinds of name resolution depending on OS. But starting 10 goroutines is much cheaper than spawning 10 threads.