4 ms·

Go Circuit – Distributed Go Programming Framework

121 points by Rooki 13y ago

vanderZwan 13y ago
For those who, like me, were looking for a more in-depth "About" page, the "Big picture: problems and solutions" link under "Research" is what you want: http://www.gocircuit.org/big.html http://www.gocircuit.org/big.html (also, to the maintainer: the "Ideal architecture" header at the bottom is on the wrong column)
RyanZAG 13y ago
This application is temporarily over its serving quota. Please try again later. Seems to be running on AppEngine by the look of it. AppEngine has the limits far too low for basic stuff like this, they should really re-look at their service offering if they want to actually compete in the market.
- GilbertErik 13y ago
  I think App Engine can be just fine depending on your needs. It's great for prototyping ideas. I just find that at some point you have to draw a line as to whether you're going to pay GAE to serve your project or if you're going to take care of that overhead yourself.
  - RyanZAG 13y ago
    I get what you're saying in that it can be used for something currently, but the opportunity being missed here is enormous. When Gmail launched with 1GB of storage when everyone else was doing 5mb, this was a pretty massive cost at the time. However, it was one of the best things Google ever did as it kick-started their entire online application business. Google has a similar possibility here - they can really take over the webapp hosting environment and create massive ease of use and integration for their various web platforms and APIs by making it easy to integrate them into AppEngine. The requirement for this is massive adoption, and the requirement for massive adoption is, at the very least, providing similar cost/performance as a random dedicated host. AppEngine/Google compute engine are barely competitive with other very expensive, high margin services such as AWS. Google is missing out massively here because they want to squeeze out traffic penalties with a terrible business model that gives users across the web a nice error message. They've at least had the sense to be guilty enough to remove their branding from the error page though.
    - jonpeda 13y ago
      1GB was a huge gimmick, because 99.9% of people couldn't get 1GB into their accounts if they tried. They just threw the huge number out there because they knew they could handle all actual current email demand.
    - crypto5 13y ago
      So, what's better currently available alternative to GAE in terms of free quota?
    - georgemcbay 13y ago
      When GAE first launched the free quotas were much nicer than they are now, they got scaled back significantly during Google's big return-of-Larry-as-CEO shift towards heavy monetization.
- threeseed 13y ago
  When it comes to GAE it really doesn't seem like Google's heart is in it. Which given their history is a bad, bad sign.
  - salimmadjd 13y ago
    I'm guessing like everything else google, it will be shutdown leaving many developers stranded. We built an entire site around GAE and it was the biggest mistake of my professional career!
    - reeses 13y ago
      You have a great opportunity to start a GAE-compatible service provider! I imagine a decent percentage of people would migrate just for insurance. (Yes, I suspect this would be the new biggest mistake of your professional career. :-))
GilbertErik 13y ago
Quick popularity like making it to the front page of HN, Reddit, Slashdot, etc. can become quite the curse when you're running a small app on App Engine. Of course, after throwing a few buckets of money down the well, you'll quickly learn to start optimizing for the platform. I don't know what this framework does, but it might be useful to link to a static page in the interim and then have people click through to your app.
- mseepgood 13y ago
  > I don't know what this framework does Excerpt from my browser cache: "The circuit extends the reach of Go's linguistic environment to multi-host/multi-process applications. In simple terms, the Go Circuit was born from the desire to be able to write: feedback := make(chan int) circuit.Spawn("host25.datacenter.net", func() { feedback <- 1 }) <-feedback println("Roundtrip complete.") The Spawn operator will start its argument function on a desired remote host in a new goroutine, while making it possible to communicate between the parent and child goroutines using the same Go code that you would use to communicate between traditional goroutines. Above, the channel feedback is transparently “stretched” between the parent goroutine, executing locally, and the child goroutine, executing remotely and hosting the anonymous function execution."
  - reddit_clone 13y ago
    Though interesting, it brings back up the age old question if this is the right thing to do? Should an RPC actually look indistinguishable from a local call? I have heard arguments from both sides starting from the days of CORBA and DCOM. But never had a convincing answer either way. It usually boils down to 'depends on use case'.
paulsmith 13y ago
https://github.com/tumblr/gocircuit https://github.com/tumblr/gocircuit
- deleted 13y ago
  [deleted]
davidw 13y ago
Sounds like Go adding more Erlang style features. Off the top of my head, some nice things that Erlang lacks in that department might include: * A bit more security by default. Erlang was built for something where they presumed they were walled off from bad stuff. * Some kind of routing mechanism, so that you could have A connected to B, and B connected to C, and the system would be able to send messages from A to C. I have no idea if this is actually a good idea, as this is not the sort of thing I use Erlang for myself. But it seems like it might be cool.
- dualogy 13y ago
  > Sounds like Go adding more Erlang style features. Nitpick: this is simply an independent 3rd-party package / external lib for Go developers. Not a language or runtime feature or addition.
  - elviejo 13y ago
    So maybe like erlang's OTP ?
deleted 13y ago
[deleted]
peterwaller 13y ago
Can someone explain how `Spawn` ships the closure for execution to a remote host? I'm quite curious to know. I had a grep through the source, but I only found one [1] implementation of `Spawn` accepting a function as a parameter in the code base. This hides the true implementation, which I'm struggling to find. Any hints? [1] https://github.com/tumblr/gocircuit/blob/master/src/circuit/use/circuit/bind.go#L84 https://github.com/tumblr/gocircuit/blob/master/src/circuit/...
- sesteel 13y ago
  I am thinking it translates your program into multiple go programs, compiles them, and then deploys them.
- jbp 13y ago
  http://www.gocircuit.org/spawn.html http://www.gocircuit.org/spawn.html
  - peterwaller 13y ago
    I found that earlier - unless I'm missing something, it doesn't actually explain how the closure is packaged. Does the worker need compiled-in knowledge of what the master wants to execute in advance, for example? It's not clear to me.
    - jbp 13y ago
      It looks like at this time Spawn doesn't support the closures yet. http://www.gocircuit.org/lang.html#worker-functions http://www.gocircuit.org/lang.html#worker-functions: "The need for boilerplate will be removed in an upcoming Circuit Compiler" From https://github.com/tumblr/gocircuit/blob/master/src/circuit/use/circuit/bind.go#L75 https://github.com/tumblr/gocircuit/blob/master/src/circuit/...: "The worker function fn, whose type must have previously been registered with RegisterFunc" And it looks like only function type id is communicated to worker: https://github.com/tumblr/gocircuit/blob/master/src/circuit/sys/lang/func.go#L115 https://github.com/tumblr/gocircuit/blob/master/src/circuit/...
jzelinskie 13y ago
This looks like tumblr took their hand at making "netchan" (and more). Pretty interesting; I'll have to check this out. I like how they tried to make it similar to Erlang. However, the way they've organized the source isn't "go-get"able which is a pet peeve of mine.

121 points by Rooki 13y ago

For those who, like me, were looking for a more in-depth "About" page, the "Big picture: problems and solutions" link under "Research" is what you want: http://www.gocircuit.org/big.html http://www.gocircuit.org/big.html (also, to the maintainer: the "Ideal architecture" header at the bottom is on the wrong column)

RyanZAG 13y ago

This application is temporarily over its serving quota. Please try again later. Seems to be running on AppEngine by the look of it. AppEngine has the limits far too low for basic stuff like this, they should really re-look at their service offering if they want to actually compete in the market.

GilbertErik 13y ago
I think App Engine can be just fine depending on your needs. It's great for prototyping ideas. I just find that at some point you have to draw a line as to whether you're going to pay GAE to serve your project or if you're going to take care of that overhead yourself.
- RyanZAG 13y ago
  I get what you're saying in that it can be used for something currently, but the opportunity being missed here is enormous. When Gmail launched with 1GB of storage when everyone else was doing 5mb, this was a pretty massive cost at the time. However, it was one of the best things Google ever did as it kick-started their entire online application business. Google has a similar possibility here - they can really take over the webapp hosting environment and create massive ease of use and integration for their various web platforms and APIs by making it easy to integrate them into AppEngine. The requirement for this is massive adoption, and the requirement for massive adoption is, at the very least, providing similar cost/performance as a random dedicated host. AppEngine/Google compute engine are barely competitive with other very expensive, high margin services such as AWS. Google is missing out massively here because they want to squeeze out traffic penalties with a terrible business model that gives users across the web a nice error message. They've at least had the sense to be guilty enough to remove their branding from the error page though.
  - jonpeda 13y ago
    1GB was a huge gimmick, because 99.9% of people couldn't get 1GB into their accounts if they tried. They just threw the huge number out there because they knew they could handle all actual current email demand.
  - crypto5 13y ago
    So, what's better currently available alternative to GAE in terms of free quota?
  - georgemcbay 13y ago
    When GAE first launched the free quotas were much nicer than they are now, they got scaled back significantly during Google's big return-of-Larry-as-CEO shift towards heavy monetization.
threeseed 13y ago
When it comes to GAE it really doesn't seem like Google's heart is in it. Which given their history is a bad, bad sign.
- salimmadjd 13y ago
  I'm guessing like everything else google, it will be shutdown leaving many developers stranded. We built an entire site around GAE and it was the biggest mistake of my professional career!
  - reeses 13y ago
    You have a great opportunity to start a GAE-compatible service provider! I imagine a decent percentage of people would migrate just for insurance. (Yes, I suspect this would be the new biggest mistake of your professional career. :-))

GilbertErik 13y ago

Quick popularity like making it to the front page of HN, Reddit, Slashdot, etc. can become quite the curse when you're running a small app on App Engine. Of course, after throwing a few buckets of money down the well, you'll quickly learn to start optimizing for the platform. I don't know what this framework does, but it might be useful to link to a static page in the interim and then have people click through to your app.

mseepgood 13y ago
> I don't know what this framework does Excerpt from my browser cache: "The circuit extends the reach of Go's linguistic environment to multi-host/multi-process applications. In simple terms, the Go Circuit was born from the desire to be able to write: feedback := make(chan int) circuit.Spawn("host25.datacenter.net", func() { feedback <- 1 }) <-feedback println("Roundtrip complete.") The Spawn operator will start its argument function on a desired remote host in a new goroutine, while making it possible to communicate between the parent and child goroutines using the same Go code that you would use to communicate between traditional goroutines. Above, the channel feedback is transparently “stretched” between the parent goroutine, executing locally, and the child goroutine, executing remotely and hosting the anonymous function execution."
- reddit_clone 13y ago
  Though interesting, it brings back up the age old question if this is the right thing to do? Should an RPC actually look indistinguishable from a local call? I have heard arguments from both sides starting from the days of CORBA and DCOM. But never had a convincing answer either way. It usually boils down to 'depends on use case'.

paulsmith 13y ago

https://github.com/tumblr/gocircuit https://github.com/tumblr/gocircuit

deleted 13y ago
[deleted]

davidw 13y ago

Sounds like Go adding more Erlang style features. Off the top of my head, some nice things that Erlang lacks in that department might include: * A bit more security by default. Erlang was built for something where they presumed they were walled off from bad stuff. * Some kind of routing mechanism, so that you could have A connected to B, and B connected to C, and the system would be able to send messages from A to C. I have no idea if this is actually a good idea, as this is not the sort of thing I use Erlang for myself. But it seems like it might be cool.

dualogy 13y ago
> Sounds like Go adding more Erlang style features. Nitpick: this is simply an independent 3rd-party package / external lib for Go developers. Not a language or runtime feature or addition.
- elviejo 13y ago
  So maybe like erlang's OTP ?

deleted 13y ago

[deleted]

peterwaller 13y ago

Can someone explain how `Spawn` ships the closure for execution to a remote host? I'm quite curious to know. I had a grep through the source, but I only found one [1] implementation of `Spawn` accepting a function as a parameter in the code base. This hides the true implementation, which I'm struggling to find. Any hints? [1] https://github.com/tumblr/gocircuit/blob/master/src/circuit/use/circuit/bind.go#L84 https://github.com/tumblr/gocircuit/blob/master/src/circuit/...

sesteel 13y ago
I am thinking it translates your program into multiple go programs, compiles them, and then deploys them.
jbp 13y ago
http://www.gocircuit.org/spawn.html http://www.gocircuit.org/spawn.html
- peterwaller 13y ago
  I found that earlier - unless I'm missing something, it doesn't actually explain how the closure is packaged. Does the worker need compiled-in knowledge of what the master wants to execute in advance, for example? It's not clear to me.
  - jbp 13y ago
    It looks like at this time Spawn doesn't support the closures yet. http://www.gocircuit.org/lang.html#worker-functions http://www.gocircuit.org/lang.html#worker-functions: "The need for boilerplate will be removed in an upcoming Circuit Compiler" From https://github.com/tumblr/gocircuit/blob/master/src/circuit/use/circuit/bind.go#L75 https://github.com/tumblr/gocircuit/blob/master/src/circuit/...: "The worker function fn, whose type must have previously been registered with RegisterFunc" And it looks like only function type id is communicated to worker: https://github.com/tumblr/gocircuit/blob/master/src/circuit/sys/lang/func.go#L115 https://github.com/tumblr/gocircuit/blob/master/src/circuit/...

jzelinskie 13y ago

This looks like tumblr took their hand at making "netchan" (and more). Pretty interesting; I'll have to check this out. I like how they tried to make it similar to Erlang. However, the way they've organized the source isn't "go-get"able which is a pet peeve of mine.