5 ms·

Gotalk

132 points by tbassetto 12y ago

farslan 12y ago
I'm one of the Core authors of the Kite library (library for writing micro services). I wrote an extensive introduction here on GopherAcademy: http://blog.gopheracademy.com/birthday-bash-2014/kite-microservice-library/ http://blog.gopheracademy.com/birthday-bash-2014/kite-micros... I think Gotalk and Kite have a lot of common (for example we have also kite.js, to talk with browsers). The mission of the Kite library is the same as with Gotalk, mainly to communicate two pieces of software in an easy way. However in order to achieve this Kite also has an inbuilt service discovery mechanism. I think this is really important because with time you are app number will increase and service discovery is a must have. There are many differences too, for example protocol (we use JSON). Congrats on shipping this, I'm really looking forward to see how it evolves and how you solve upcoming problems (such as poor network conditions where Websocket badly fails).
- zimbatm 12y ago
  Wouldn't it be more fair to compare Gotalk to Kite's underlying dnode-protocol[1] ? Personally Gotalk's protocol seems much better thought out; size-prefixed messaging, rpc & streaming support, multiplexing (especially important with WebSocket). The only missing bit would be some form of heart-beating. * [1] https://github.com/substack/dnode-protocol https://github.com/substack/dnode-protocol
  - jra_samba 12y ago
    Messages need to be encrypted by default. No more cleartext protocols on the network anymore please, for obvious reasons.
    - rasmusfabbe 12y ago
      With Gotalk, you can use the byte-buffer request and response functionality (instead of the JSON-encoding versions) and e.g. dec.XORKeyStream(inbuf, payload) in your handler when receiving a payload followed by enc.XORKeyStream(payload, outbuf) when responding.
      - irq-1 12y ago
        You could add symmetric encryption before and after putting anything on the network, and let users turn it on with a simple setting. Here's the fun part: you could use go generators to create a local key when the package is compiled. Then you wouldn't need to embed a key, or ask the user to generate and pass in a key.
- rasmusfabbe 12y ago
  Hello Farslan, I'm the author of Gotalk and just read your article on Kite — very interesting! In the scenario of composing distributing systems where you'd e.g. want to "map-reduce this data on some number of cores" service discovery is important. However I wrote Gotalk to be as small as possible, meaning I hope it can become one of those small packages I feel comfortable using in any of my projects. The "small" part here is key: that it feels small. Perhaps there's an approach where a completely decoupled service-discovery package exists which can be combined with Kite or Gotalk to form auto-discovering clusters of programs. Thanks for bringing Kite to our attention.
tobz 12y ago
What's the major difference between Gotalk and Socket.IO?
- zimbatm 12y ago
  Socket.IO is for pubsub only, Gotalk is much more versatile than that.
  - jhgg 12y ago
    It most definitely is not. http://socket.io/docs/#sending-and-getting-data-(acknowledgements) http://socket.io/docs/#sending-and-getting-data-(acknowledge... Could be used to serve a similar purpose to this lib.
mjs 12y ago
The author was Spotify's original designer: http://rsms.me/about/spotify/ http://rsms.me/about/spotify/
fishnchips 12y ago
Awesome work, congrats! IMO it would be great if the serialisation/pipelining protocol was decoupled from this library so that you could easily create compatible clients and servers in a language other than Go.
- rasmusfabbe 12y ago
  If you have a look at protocol.go as well as the "Wire format" section of README.md, you'll find that the Gotalk protocol is decoupled from the Sock type, handlers and other parts of the Go package. The JavaScript implementation similarly provides protocol functions in js/gotalk/protocol.js — I hope I didn't misunderstand your question :)
toxik 12y ago
Nice thingy, I always loved the writing style of Rasmus. Good work, and stellar exposition.
- rasmusfabbe 12y ago
  Thanks!
Animats 12y ago
This looks a lot like Open RPC, formerly Sun RPC (https://en.wikipedia.org/wiki/Open_Network_Computing_Remote_Procedure_Call https://en.wikipedia.org/wiki/Open_Network_Computing_Remote_...) It's a binary-format remote procedure call. Why not just implement that for Go? Then you can interoperate with existing implementations for other languages. There are lots of standards for this sort of thing, from SOAP to Google Protocol Buffers. Opening this up for connection to potentially hostile nodes requires more design. What about denial of service problems? Can someone open a connection and tie up resources?
- otterley 12y ago
  The complementary wire format for OpenRPC is XDR, which is an RFC: http://tools.ietf.org/html/rfc4506.html http://tools.ietf.org/html/rfc4506.html
gtaylor 12y ago
If anyone is interested, I've started playing around with a Python implementation here: https://github.com/gtaylor/python-gotalk https://github.com/gtaylor/python-gotalk I'm mostly focused on the marshalling/unmarshalling of the various message types right now.
sneak 12y ago
What's wrong with json-rpc?
rtpg 12y ago
I worked on a project using Akka (Scala), and this seems to have the same core aspects as that (well, it's just message passing but easy to set up and thel like). Hope this catches on, seems like a very promising library
- rasmusfabbe 12y ago
  Akka makes some impressive claims, like 400 bytes of memory usage for an entire actor "coroutine." I'm not well-versed in Scala, but it looks like a pretty novel API (i.e. the GreetingActor example et al.)
  - Gurkenmaster 12y ago
    400bytes? Seems to be the absoulute minimum because Parallel Universe also claims that Quasar's fibers only use 400 bytes.
kbd 12y ago
Does this improve at all upon Cap'n Proto RPC? https://capnproto.org/rpc.html https://capnproto.org/rpc.html Why not use that?
xcombelle 12y ago
Only 3 bytes for requestid ? is not too few ?
thebyrd 12y ago
Nice project! The callbacks on event listeners reminds me a lot of javascript and can get messy. Go handles this with channels. Why did you decide not to use channels to emit events?

132 points by tbassetto 12y ago

I'm one of the Core authors of the Kite library (library for writing micro services). I wrote an extensive introduction here on GopherAcademy: http://blog.gopheracademy.com/birthday-bash-2014/kite-microservice-library/ http://blog.gopheracademy.com/birthday-bash-2014/kite-micros... I think Gotalk and Kite have a lot of common (for example we have also kite.js, to talk with browsers). The mission of the Kite library is the same as with Gotalk, mainly to communicate two pieces of software in an easy way. However in order to achieve this Kite also has an inbuilt service discovery mechanism. I think this is really important because with time you are app number will increase and service discovery is a must have. There are many differences too, for example protocol (we use JSON). Congrats on shipping this, I'm really looking forward to see how it evolves and how you solve upcoming problems (such as poor network conditions where Websocket badly fails).

zimbatm 12y ago
Wouldn't it be more fair to compare Gotalk to Kite's underlying dnode-protocol[1] ? Personally Gotalk's protocol seems much better thought out; size-prefixed messaging, rpc & streaming support, multiplexing (especially important with WebSocket). The only missing bit would be some form of heart-beating. * [1] https://github.com/substack/dnode-protocol https://github.com/substack/dnode-protocol
- jra_samba 12y ago
  Messages need to be encrypted by default. No more cleartext protocols on the network anymore please, for obvious reasons.
  - rasmusfabbe 12y ago
    With Gotalk, you can use the byte-buffer request and response functionality (instead of the JSON-encoding versions) and e.g. dec.XORKeyStream(inbuf, payload) in your handler when receiving a payload followed by enc.XORKeyStream(payload, outbuf) when responding.
    - irq-1 12y ago
      You could add symmetric encryption before and after putting anything on the network, and let users turn it on with a simple setting. Here's the fun part: you could use go generators to create a local key when the package is compiled. Then you wouldn't need to embed a key, or ask the user to generate and pass in a key.
rasmusfabbe 12y ago
Hello Farslan, I'm the author of Gotalk and just read your article on Kite — very interesting! In the scenario of composing distributing systems where you'd e.g. want to "map-reduce this data on some number of cores" service discovery is important. However I wrote Gotalk to be as small as possible, meaning I hope it can become one of those small packages I feel comfortable using in any of my projects. The "small" part here is key: that it feels small. Perhaps there's an approach where a completely decoupled service-discovery package exists which can be combined with Kite or Gotalk to form auto-discovering clusters of programs. Thanks for bringing Kite to our attention.

tobz 12y ago

What's the major difference between Gotalk and Socket.IO?

zimbatm 12y ago
Socket.IO is for pubsub only, Gotalk is much more versatile than that.
- jhgg 12y ago
  It most definitely is not. http://socket.io/docs/#sending-and-getting-data-(acknowledgements) http://socket.io/docs/#sending-and-getting-data-(acknowledge... Could be used to serve a similar purpose to this lib.

mjs 12y ago

The author was Spotify's original designer: http://rsms.me/about/spotify/ http://rsms.me/about/spotify/

fishnchips 12y ago

Awesome work, congrats! IMO it would be great if the serialisation/pipelining protocol was decoupled from this library so that you could easily create compatible clients and servers in a language other than Go.

rasmusfabbe 12y ago
If you have a look at protocol.go as well as the "Wire format" section of README.md, you'll find that the Gotalk protocol is decoupled from the Sock type, handlers and other parts of the Go package. The JavaScript implementation similarly provides protocol functions in js/gotalk/protocol.js — I hope I didn't misunderstand your question :)

toxik 12y ago

Nice thingy, I always loved the writing style of Rasmus. Good work, and stellar exposition.

rasmusfabbe 12y ago
Thanks!

Animats 12y ago

This looks a lot like Open RPC, formerly Sun RPC (https://en.wikipedia.org/wiki/Open_Network_Computing_Remote_Procedure_Call https://en.wikipedia.org/wiki/Open_Network_Computing_Remote_...) It's a binary-format remote procedure call. Why not just implement that for Go? Then you can interoperate with existing implementations for other languages. There are lots of standards for this sort of thing, from SOAP to Google Protocol Buffers. Opening this up for connection to potentially hostile nodes requires more design. What about denial of service problems? Can someone open a connection and tie up resources?

otterley 12y ago
The complementary wire format for OpenRPC is XDR, which is an RFC: http://tools.ietf.org/html/rfc4506.html http://tools.ietf.org/html/rfc4506.html

gtaylor 12y ago

If anyone is interested, I've started playing around with a Python implementation here: https://github.com/gtaylor/python-gotalk https://github.com/gtaylor/python-gotalk I'm mostly focused on the marshalling/unmarshalling of the various message types right now.

sneak 12y ago

What's wrong with json-rpc?

rtpg 12y ago

I worked on a project using Akka (Scala), and this seems to have the same core aspects as that (well, it's just message passing but easy to set up and thel like). Hope this catches on, seems like a very promising library

rasmusfabbe 12y ago
Akka makes some impressive claims, like 400 bytes of memory usage for an entire actor "coroutine." I'm not well-versed in Scala, but it looks like a pretty novel API (i.e. the GreetingActor example et al.)
- Gurkenmaster 12y ago
  400bytes? Seems to be the absoulute minimum because Parallel Universe also claims that Quasar's fibers only use 400 bytes.

kbd 12y ago

Does this improve at all upon Cap'n Proto RPC? https://capnproto.org/rpc.html https://capnproto.org/rpc.html Why not use that?

xcombelle 12y ago

Only 3 bytes for requestid ? is not too few ?

thebyrd 12y ago

Nice project! The callbacks on event listeners reminds me a lot of javascript and can get messy. Go handles this with channels. Why did you decide not to use channels to emit events?