8 ms·

Libchan: Like Go channels over the network

171 points by akerl_ 12y ago

stephanos2k 12y ago
I would love to see a small example/snippet illustrating how I'd use this.
- patrickaljord 12y ago
  Here are Solomon's slides on the topic. http://www.slideshare.net/shykes/docker-the-road-ahead http://www.slideshare.net/shykes/docker-the-road-ahead
- shykes 12y ago
  Here's how you would implement basic RPC-style request/response. On the client: var ch libchan.Sender // Send a message, indicate that we want a return channel to be automatically created ret1, err := ch.Send(&libchan.Message{Data: []byte("request 1!"), Ret: libchan.RetPipe}) // Send another message on the same channel ret2, err := ch.Send(&libchan.Message{Data: []byte("request 2!"), Ret: libchan.RetPipe}) // Wait for an answer from the first request. Set flags to zero // to indicate we don't want a nested return channel msg, err := ret1.Receive(0) fmt.Printf("Received answer: %s\n", msg.Data) On the server: var ch libchan.Receiver // Wait for messages in a loop // Set the return channel flag, // to indicate we want to receive nested channels (if any). // Note: we don't send a nested return channel, but we could. for { msg, err := ch.Receive(libchan.Ret) msg.Ret.Send(&libchan.Message{Data: []byte("this is an utterly useless response")}) }
- deleted 12y ago
  [deleted]
- shurcooL 12y ago
  I came here to say the same thing. Neither the README nor the examples folder had a very short, basic example. Thanks for the answer (below/above).
rubiquity 12y ago
I'd like to know why to use this over ZeroMQ. Yes, ZeroMQ has its own protocol (which libchan's README points out), but I haven't found that protocol to get in the way, because in the end you're just sending text down the ZeroMQ socket and ZeroMQ's protocol is almost entirely "behind the scenes" so to speak. I do understand why Docker would create this for their own use, because having a dependency on messaging libraries (where people tend to have varying opinions) wouldn't be the greatest for a wildly popular open source project. I'm just trying to figure out why others would want to use this. I'm all for more tools in this space, though I like to make sure a tool exists for reasons other than "not invented here" before I learn yet another messaging library.
- shykes 12y ago
  Hey, we (Dotcloud now Docker) have used zeromq a LOT in the past. The entire Dotcloud platform is based on Zerorpc (http://github.com/dotcloud/zerorpc http://github.com/dotcloud/zerorpc) which is based on ZMQ. Libchan is the spiritual child of zerorpc, so it is definitely informed by our experience using zeromq. The main experience was: it's awesome but does too much, too low in the stack, and it's too much of a black box to remove the parts we don't need. Also, the inability to safely expose it on the public internet was a big problem. I know this has been addressed since then, but we don't want to use a separate protocol stack for that: we want to use the existing HTTP (soon HTTP2) and TLS middleware infrastructure. That's why libchan uses spdy/tls as its main network transport.
  - sneak 12y ago
    > Also, the inability to safely expose it on the public internet was a big problem. Deus, why? Why on Earth would parts of your critical internal infrastructure be exposed to the public Internet?!
    - shykes 12y ago
      Who said anything about "internal"? We want every piece of our infrastructure to use a common, unified communication layer. If parts of it should be authenticated, tunneled, throttled or otherwise protected from nefarious outsiders, then we want to add the appropriate off-the-shelf middleware, or possibly write our own middleware. It makes no sense, in my mind, to artificially separate "internal" and "external" by implementing completely different protocol stacks and tools. In the end this hurts security because everything gets more complicated and you lose visibility. Just my 2c based on our own production experience.
baq 12y ago
"like Go channels over the network" um, sockets anyone?
- zimbatm 12y ago
  It's possible to send a channel over a channel in go. This allows to for example to pass in the response channel of a message. It's possible to pass in file descriptors on unix sockets but it only works locally.
  - shykes 12y ago
    Note that, when using the Unix socket transport, libchan actually utilizes the ability to pass file descriptors.
- shykes 12y ago
  You can't send a tcp socket over a tcp socket :) (just one among many reasons why raw sockets are necessary but not sufficient).
  - mbreese 12y ago
    And I suppose you can with a channel? Or are you just sending a proxy to the socket (which you could also do with a traditional socket and an appropriate protocol). I'm not trying to diminish the importance of an easy to use library that abstracts away the socket-level code with a good protocol. I'm just saying that sending a socket over a socket isn't a good way to pitch it.
  - tlrobinson 12y ago
    How does this work? If A has a channel to B, and sends that channel to C, do B and C now communicate directly, or is it proxied through A? EDIT: I see you mentioned the Unix socket transport passes around file descriptors. How about the inter-host transports? Obviously two browsers can't directly connect to each other (or could they with WebRTC...?) what about, say, TCP between hosts on the same local network?
  - jvermillard 12y ago
kitd 12y ago
Back in the days of DCE, DCOM, RMI, etc, everyone used to say that "location transparency" was a myth. Has that changed?
- shykes 12y ago
  This is actually the opposite pattern: instead of pretending everything is a local function even over the network (which turned out to be a bad idea), what if we did it the other way around? Pretend your components are communicating over a network even when they aren't. This is made possible by very efficient lightweight threads (goroutines in go, gevent in Python etc.) and message-oriented patterns popularized by Go channels (and Erlang/OTP before it). Now your application always checks for IO errors, and the underlying "plumbing" is exposed and available for the developer to tweak at will: timeouts, caching, failover, fan-in, fan-out, etc. become programmable components just like the rest of your app. TLDR: this is not born-again RPC. It's the anti-RPC.
  - lclarkmichalek 12y ago
    Essentially, instead of fighting against concurrency, by building big fancy black boxes of state, we now embrace concurrency, and let it permeate through the (green) fabric of the program.
  - tbrownaw 12y ago
    Pretend your components are communicating over a network even when they aren't Now your application always checks for IO errors, and the underlying "plumbing" is exposed and available for the developer to tweak at will: timeouts, caching, failover, fan-in, fan-out, etc The big problem is that the bandwidth and latency numbers are vastly different over an actual network vs between processes or OS threads on a single machine vs between green threads within a process. The problem is that sometimes you require high performance, to a degree that is not possible across an actual network link. And if your coding style doesn't distinguish between an actual network link and an imaginary in-process link (or worse, deliberately makes them indistinguishable and silently interchangeable), sooner or later someone will refactor it or change the config file or something and your microsecond-scale latency that you were assuming and relying on has suddenly become multiple-millisecond latency and everything grinds to a halt.
shykes 12y ago
We introduced this at Dockercon today, along with a few other projects. Here are the slides for the presentation: http://www.slideshare.net/shykes/docker-the-road-ahead http://www.slideshare.net/shykes/docker-the-road-ahead Note: the illustrator who helped me with this is @laurelcomics on Twitter, she is awesome and available for contracting :)
nkozyra 12y ago
I wonder if the authors are aware of Go'circuit: https://github.com/gocircuit/circuit https://github.com/gocircuit/circuit
swah 12y ago
Nice, I've read some about a netchan lib that existed in previous go versions. How is the problem w/ netchan described here https://groups.google.com/d/msg/Golang-Nuts/kK8XqkaaVuU/ZQdK01Sjil0J https://groups.google.com/d/msg/Golang-Nuts/kK8XqkaaVuU/ZQdK... handled?
- jerf 12y ago
  Based on the code snippet above, by not trying to truly replicate the channel API. Which is a good thing, because it is simply too simple for something that may be a network communication. It's only trying to truly replicate the channel API over the network that causes problems; "channel-like" abstractions themselves have no particular complication.
stcredzero 12y ago
How about this? Make a new language that compiles to Javascript with Go like functionality, but instead of the "go" keyword to fire a coroutine, use something like "fore." Then enable single argument code blocks (lambdas) to be attached to a channel, also making it a function. Channels would have the same abbreviation as in Go. I think there is a lot one could do with these two features. Of course, one would have to name such a language after its two major features.
- jvermillard 12y ago
  clojurescript + core.async?
  - stcredzero 12y ago
    Yes, but then you'd lose the pun.
gangster_dave 12y ago
And down goes erlang!
swdunlop 12y ago
The README looks interesting, but with zero API documentation, it is just a curiosity; see https://godoc.org/github.com/docker/libchan https://godoc.org/github.com/docker/libchan for how this package looks to GoDoc. Usage information and context is essential for encouraging other developers to adopt a dependency and start returning contributions.
lsllc 12y ago
Looks very interesting, I've been testing nsq vs 0mq; I like nsq's durability, but it's missing basic stuff such as a reply queue (you have to manage that yourself). 0mq seems ok, but encryption doesn't appear to be supported in the go client. Will take a good look at this. Thanks!
- politician 12y ago
  Here's an NSQ RPC server written in Erlang: https://github.com/project-fifo/ensq_rpc https://github.com/project-fifo/ensq_rpc I've been meaning to spend an afternoon porting it to Go...
ww520 12y ago
OT: Is there a PR campaign running by the Docker company now? There are awfully lots of Docker related news lately, not just the product release but all these side projects. It seems the side projects are announced coincided with the product release to prolong the buzz. Just curious.
- mtalantikite 12y ago
  DockerCon has been going on: http://www.dockercon.com/ http://www.dockercon.com/
- akerl_ 12y ago
  They seem to be engaged in the best kind of PR: shipping lots of cool code :) As the other responder pointed out, DockerCon is currently in progress, so most of the projects that have hit HN were just announced publicly as part of that. On a related note, I recommend folks keep an eye out for the videos of the talks. The Red Hat talk on geard/atomic and shykes's talk on the future of Docker (and these other projects) were especially great.
- zobzu 12y ago
  HN is a pretty good PR channel for such news. its easy to get in front too, get 10 ppl to upvote you and done you're front page material. Its a little annoying when they spam it but meh, they work on stuff we need and its open source - could be worse.
iand675 12y ago
While I can understand that the Docker team operates in a very Go-centric environment, it’s a shame that some of these libraries don’t export a C interface to enable other languages to use them. That being said, it’s a neat project that I’m tempted to port the ideas from.
- shykes 12y ago
  We explicitly optimized for ease of portability, because Docker needs a solid implementation in every major language (so that we can use libchan as a transport for remote access to Docker, and for introspection by containers). Would you like to work with us on a C implementation ? :)
  - iand675 12y ago
    I'm not stellar with C, but I'd be happy to contribute!
    - shykes 12y ago
      No need to be stellar :) Feel free to join #libswarm on Freenode, that's where the devs hang out!
CMCDragonkai 12y ago
Is this designed for more higher level protocols to be built on top of? For example, could ZeroMQ, NanoMSG, WebSockets... rely on this? Or is this meant to supplant all the existing protocols that we already have. Would erlang have a use for this?
tete 12y ago
Appears to be heavily inspired by Node.js Streams[1]. Would love this pattern to be copied to other languages aswell. [1] https://github.com/substack/stream-handbook https://github.com/substack/stream-handbook
dragonwriter 12y ago
From the readme: > An explicit goal of libchan is simplicity of implementation and clarity of spec. Porting it to any language should be as effortless as humanly possible. Where is the spec?
mveety 12y ago
I wrote something similar to this a while back for Plan 9. Having channels over the network is really awesome, and it's the only thing I think that Go is badly missing. It would be cool if the author could have this make use of the normal Go syntax for message passing.

171 points by akerl_ 12y ago

stephanos2k 12y ago

I would love to see a small example/snippet illustrating how I'd use this.

patrickaljord 12y ago
Here are Solomon's slides on the topic. http://www.slideshare.net/shykes/docker-the-road-ahead http://www.slideshare.net/shykes/docker-the-road-ahead
shykes 12y ago
Here's how you would implement basic RPC-style request/response. On the client: var ch libchan.Sender // Send a message, indicate that we want a return channel to be automatically created ret1, err := ch.Send(&libchan.Message{Data: []byte("request 1!"), Ret: libchan.RetPipe}) // Send another message on the same channel ret2, err := ch.Send(&libchan.Message{Data: []byte("request 2!"), Ret: libchan.RetPipe}) // Wait for an answer from the first request. Set flags to zero // to indicate we don't want a nested return channel msg, err := ret1.Receive(0) fmt.Printf("Received answer: %s\n", msg.Data) On the server: var ch libchan.Receiver // Wait for messages in a loop // Set the return channel flag, // to indicate we want to receive nested channels (if any). // Note: we don't send a nested return channel, but we could. for { msg, err := ch.Receive(libchan.Ret) msg.Ret.Send(&libchan.Message{Data: []byte("this is an utterly useless response")}) }
deleted 12y ago
[deleted]
shurcooL 12y ago
I came here to say the same thing. Neither the README nor the examples folder had a very short, basic example. Thanks for the answer (below/above).

rubiquity 12y ago

I'd like to know why to use this over ZeroMQ. Yes, ZeroMQ has its own protocol (which libchan's README points out), but I haven't found that protocol to get in the way, because in the end you're just sending text down the ZeroMQ socket and ZeroMQ's protocol is almost entirely "behind the scenes" so to speak. I do understand why Docker would create this for their own use, because having a dependency on messaging libraries (where people tend to have varying opinions) wouldn't be the greatest for a wildly popular open source project. I'm just trying to figure out why others would want to use this. I'm all for more tools in this space, though I like to make sure a tool exists for reasons other than "not invented here" before I learn yet another messaging library.

shykes 12y ago
Hey, we (Dotcloud now Docker) have used zeromq a LOT in the past. The entire Dotcloud platform is based on Zerorpc (http://github.com/dotcloud/zerorpc http://github.com/dotcloud/zerorpc) which is based on ZMQ. Libchan is the spiritual child of zerorpc, so it is definitely informed by our experience using zeromq. The main experience was: it's awesome but does too much, too low in the stack, and it's too much of a black box to remove the parts we don't need. Also, the inability to safely expose it on the public internet was a big problem. I know this has been addressed since then, but we don't want to use a separate protocol stack for that: we want to use the existing HTTP (soon HTTP2) and TLS middleware infrastructure. That's why libchan uses spdy/tls as its main network transport.
- sneak 12y ago
  > Also, the inability to safely expose it on the public internet was a big problem. Deus, why? Why on Earth would parts of your critical internal infrastructure be exposed to the public Internet?!
  - shykes 12y ago
    Who said anything about "internal"? We want every piece of our infrastructure to use a common, unified communication layer. If parts of it should be authenticated, tunneled, throttled or otherwise protected from nefarious outsiders, then we want to add the appropriate off-the-shelf middleware, or possibly write our own middleware. It makes no sense, in my mind, to artificially separate "internal" and "external" by implementing completely different protocol stacks and tools. In the end this hurts security because everything gets more complicated and you lose visibility. Just my 2c based on our own production experience.

baq 12y ago

"like Go channels over the network" um, sockets anyone?

zimbatm 12y ago
It's possible to send a channel over a channel in go. This allows to for example to pass in the response channel of a message. It's possible to pass in file descriptors on unix sockets but it only works locally.
- shykes 12y ago
  Note that, when using the Unix socket transport, libchan actually utilizes the ability to pass file descriptors.
shykes 12y ago
You can't send a tcp socket over a tcp socket :) (just one among many reasons why raw sockets are necessary but not sufficient).
- mbreese 12y ago
  And I suppose you can with a channel? Or are you just sending a proxy to the socket (which you could also do with a traditional socket and an appropriate protocol). I'm not trying to diminish the importance of an easy to use library that abstracts away the socket-level code with a good protocol. I'm just saying that sending a socket over a socket isn't a good way to pitch it.
- tlrobinson 12y ago
  How does this work? If A has a channel to B, and sends that channel to C, do B and C now communicate directly, or is it proxied through A? EDIT: I see you mentioned the Unix socket transport passes around file descriptors. How about the inter-host transports? Obviously two browsers can't directly connect to each other (or could they with WebRTC...?) what about, say, TCP between hosts on the same local network?
- jvermillard 12y ago

kitd 12y ago

Back in the days of DCE, DCOM, RMI, etc, everyone used to say that "location transparency" was a myth. Has that changed?

shykes 12y ago
This is actually the opposite pattern: instead of pretending everything is a local function even over the network (which turned out to be a bad idea), what if we did it the other way around? Pretend your components are communicating over a network even when they aren't. This is made possible by very efficient lightweight threads (goroutines in go, gevent in Python etc.) and message-oriented patterns popularized by Go channels (and Erlang/OTP before it). Now your application always checks for IO errors, and the underlying "plumbing" is exposed and available for the developer to tweak at will: timeouts, caching, failover, fan-in, fan-out, etc. become programmable components just like the rest of your app. TLDR: this is not born-again RPC. It's the anti-RPC.
- lclarkmichalek 12y ago
  Essentially, instead of fighting against concurrency, by building big fancy black boxes of state, we now embrace concurrency, and let it permeate through the (green) fabric of the program.
- tbrownaw 12y ago
  Pretend your components are communicating over a network even when they aren't Now your application always checks for IO errors, and the underlying "plumbing" is exposed and available for the developer to tweak at will: timeouts, caching, failover, fan-in, fan-out, etc The big problem is that the bandwidth and latency numbers are vastly different over an actual network vs between processes or OS threads on a single machine vs between green threads within a process. The problem is that sometimes you require high performance, to a degree that is not possible across an actual network link. And if your coding style doesn't distinguish between an actual network link and an imaginary in-process link (or worse, deliberately makes them indistinguishable and silently interchangeable), sooner or later someone will refactor it or change the config file or something and your microsecond-scale latency that you were assuming and relying on has suddenly become multiple-millisecond latency and everything grinds to a halt.

shykes 12y ago

We introduced this at Dockercon today, along with a few other projects. Here are the slides for the presentation: http://www.slideshare.net/shykes/docker-the-road-ahead http://www.slideshare.net/shykes/docker-the-road-ahead Note: the illustrator who helped me with this is @laurelcomics on Twitter, she is awesome and available for contracting :)

nkozyra 12y ago

I wonder if the authors are aware of Go'circuit: https://github.com/gocircuit/circuit https://github.com/gocircuit/circuit

swah 12y ago

Nice, I've read some about a netchan lib that existed in previous go versions. How is the problem w/ netchan described here https://groups.google.com/d/msg/Golang-Nuts/kK8XqkaaVuU/ZQdK01Sjil0J https://groups.google.com/d/msg/Golang-Nuts/kK8XqkaaVuU/ZQdK... handled?

jerf 12y ago
Based on the code snippet above, by not trying to truly replicate the channel API. Which is a good thing, because it is simply too simple for something that may be a network communication. It's only trying to truly replicate the channel API over the network that causes problems; "channel-like" abstractions themselves have no particular complication.

stcredzero 12y ago

How about this? Make a new language that compiles to Javascript with Go like functionality, but instead of the "go" keyword to fire a coroutine, use something like "fore." Then enable single argument code blocks (lambdas) to be attached to a channel, also making it a function. Channels would have the same abbreviation as in Go. I think there is a lot one could do with these two features. Of course, one would have to name such a language after its two major features.

jvermillard 12y ago
clojurescript + core.async?
- stcredzero 12y ago
  Yes, but then you'd lose the pun.

gangster_dave 12y ago

And down goes erlang!

swdunlop 12y ago

The README looks interesting, but with zero API documentation, it is just a curiosity; see https://godoc.org/github.com/docker/libchan https://godoc.org/github.com/docker/libchan for how this package looks to GoDoc. Usage information and context is essential for encouraging other developers to adopt a dependency and start returning contributions.

lsllc 12y ago

Looks very interesting, I've been testing nsq vs 0mq; I like nsq's durability, but it's missing basic stuff such as a reply queue (you have to manage that yourself). 0mq seems ok, but encryption doesn't appear to be supported in the go client. Will take a good look at this. Thanks!

politician 12y ago
Here's an NSQ RPC server written in Erlang: https://github.com/project-fifo/ensq_rpc https://github.com/project-fifo/ensq_rpc I've been meaning to spend an afternoon porting it to Go...

ww520 12y ago

OT: Is there a PR campaign running by the Docker company now? There are awfully lots of Docker related news lately, not just the product release but all these side projects. It seems the side projects are announced coincided with the product release to prolong the buzz. Just curious.

mtalantikite 12y ago
DockerCon has been going on: http://www.dockercon.com/ http://www.dockercon.com/
akerl_ 12y ago
They seem to be engaged in the best kind of PR: shipping lots of cool code :) As the other responder pointed out, DockerCon is currently in progress, so most of the projects that have hit HN were just announced publicly as part of that. On a related note, I recommend folks keep an eye out for the videos of the talks. The Red Hat talk on geard/atomic and shykes's talk on the future of Docker (and these other projects) were especially great.
zobzu 12y ago
HN is a pretty good PR channel for such news. its easy to get in front too, get 10 ppl to upvote you and done you're front page material. Its a little annoying when they spam it but meh, they work on stuff we need and its open source - could be worse.

iand675 12y ago

While I can understand that the Docker team operates in a very Go-centric environment, it’s a shame that some of these libraries don’t export a C interface to enable other languages to use them. That being said, it’s a neat project that I’m tempted to port the ideas from.

shykes 12y ago
We explicitly optimized for ease of portability, because Docker needs a solid implementation in every major language (so that we can use libchan as a transport for remote access to Docker, and for introspection by containers). Would you like to work with us on a C implementation ? :)
- iand675 12y ago
  I'm not stellar with C, but I'd be happy to contribute!
  - shykes 12y ago
    No need to be stellar :) Feel free to join #libswarm on Freenode, that's where the devs hang out!

CMCDragonkai 12y ago

Is this designed for more higher level protocols to be built on top of? For example, could ZeroMQ, NanoMSG, WebSockets... rely on this? Or is this meant to supplant all the existing protocols that we already have. Would erlang have a use for this?

tete 12y ago

Appears to be heavily inspired by Node.js Streams[1]. Would love this pattern to be copied to other languages aswell. [1] https://github.com/substack/stream-handbook https://github.com/substack/stream-handbook

dragonwriter 12y ago

From the readme: > An explicit goal of libchan is simplicity of implementation and clarity of spec. Porting it to any language should be as effortless as humanly possible. Where is the spec?

mveety 12y ago

I wrote something similar to this a while back for Plan 9. Having channels over the network is really awesome, and it's the only thing I think that Go is badly missing. It would be cool if the author could have this make use of the normal Go syntax for message passing.