4 ms·

I don't really "know" golang but this seems like an area where it would have been useful. Am I think about this the right way? Would channels have helped handl

I don't really "know" golang but this seems like an area where it would have been useful. Am I think about this the right way? Would channels have helped handle some of the load effectively? Any gophers care to comment? :D

Zariel 11y ago
I know bitly have an implementation of statsd in golang [0]. Ive not used it in production but have contributed to it. Go is fast to a point, it is not as fast as C but it is easier to write fast Go than fast C. [0] https://github.com/bitly/statsdaemon https://github.com/bitly/statsdaemon
sagichmal 11y ago
Choosing C for this project is definitely... strange. Keith Rarick calls it "downright irresponsible" [1] — I'm not sure I disagree. [1] https://twitter.com/krarick/status/610502413007503360 https://twitter.com/krarick/status/610502413007503360
- jeremyjh 11y ago
  Does he think there were no Garbage Collected languages available in 2007 when he started beanstalkd ? The arrogance is breathtaking.
  - brazzledazzle 11y ago
    He actually replied to a tweet that brought that up: "I was young(er) and stupid(er) then.."
- DasIch 11y ago
  What is the alternative? Rust isn't there yet when it comes to IO and there are certainly no other languages that qualify and allow a similar level of control.
  - sagichmal 11y ago
    Go is the obvious choice.
    - DasIch 11y ago
      Go is more high level and doesn't offer a similar level of control. Go is an obvious candidate that should be evaluated but it's not an obvious choice the same way that Rust is going to be and C is.
  - rakoo 11y ago
    What about nim ? There are facilities to run an evented loop, to listen on an udp socket, can be tuned for more realtime capabilities, is compiled to C...
    - DasIch 11y ago
      Nim doesn't have the same popularity. There aren't a lot of people that know the language, that makes hiring people that do difficult, it means there may be nobody at Github that does either and due to it's lack of popularity people will probably be less willing to learn it than Rust or Go.
- rtomayko 11y ago
  I'm as keen to see memory safe systems languages become commonplace as anyone but some 90% of our stack is C - Linux, MySQL, Git, MRI, nginx, haproxy, memcached, redis, and many other internal components that we'll be talking about on the engineering blog soon. We like C. We're going to need a few more years of research, real production experience, and language/library maturity before betting critical infrastructure on something else by default.
  - sagichmal 11y ago
    > We like C. We're going to need a few more years of > research, real production experience, and > language/library maturity before betting critical > infrastructure on something else by default. That's a really bizarre opinion to hold in the year 2015. Don't get me wrong: I love C, too. But certainly not for home-grown infrastructure at a web company — the incentives just don't align. And hiding behind the "we just don't know" bugbear doesn't parse, either. Go, for example, powers gargantuan-scale infrastructure at Google, and has for half a decade. And there's whole fleets of organizations as big or bigger than GitHub that report the same experience.
    - vidarh 11y ago
      Google also has a gargantuan-scale dev team that includes the people behind Go. It's ridiculous to compare. If Github does not yet themselves have sufficient people with sufficient experience supporting large services in Go, betting critical infrastructure on it would be irresponsible. Note that he is not saying they're not open to alternatives, nor that they're not experimenting with alternatives, but that they need more experience first before "betting critical infrastructure on something else by default".
      - sagichmal 11y ago
        > Google also has a gargantuan-scale dev team that > includes the people behind Go. It's ridiculous to > compare. It's not ridiculous. There are probably reasonable arguments against using Go for things like this, but "insufficient developer capacity" isn't one of them. Becoming a Go expert is a task measured in weeks.
kasey_junk 11y ago
I'm not sure what you are asking about specifically but in general it's important to remember that channels are a pretty thin abstraction around a mutex (and a buffer). There is nothing about that abstraction that would make you think it would be better for this use case than spinlocks and a lock free table, and quite a bit that would make you think it is much worse. One of the first things people abandon when scaling golang servers is the channel abstraction and I expect the same would be true in this case.

Zariel 11y ago

I know bitly have an implementation of statsd in golang [0]. Ive not used it in production but have contributed to it. Go is fast to a point, it is not as fast as C but it is easier to write fast Go than fast C. [0] https://github.com/bitly/statsdaemon https://github.com/bitly/statsdaemon

sagichmal 11y ago

Choosing C for this project is definitely... strange. Keith Rarick calls it "downright irresponsible" [1] — I'm not sure I disagree. [1] https://twitter.com/krarick/status/610502413007503360 https://twitter.com/krarick/status/610502413007503360

jeremyjh 11y ago
Does he think there were no Garbage Collected languages available in 2007 when he started beanstalkd ? The arrogance is breathtaking.
- brazzledazzle 11y ago
  He actually replied to a tweet that brought that up: "I was young(er) and stupid(er) then.."
DasIch 11y ago
What is the alternative? Rust isn't there yet when it comes to IO and there are certainly no other languages that qualify and allow a similar level of control.
- sagichmal 11y ago
  Go is the obvious choice.
  - DasIch 11y ago
    Go is more high level and doesn't offer a similar level of control. Go is an obvious candidate that should be evaluated but it's not an obvious choice the same way that Rust is going to be and C is.
- rakoo 11y ago
  What about nim ? There are facilities to run an evented loop, to listen on an udp socket, can be tuned for more realtime capabilities, is compiled to C...
  - DasIch 11y ago
    Nim doesn't have the same popularity. There aren't a lot of people that know the language, that makes hiring people that do difficult, it means there may be nobody at Github that does either and due to it's lack of popularity people will probably be less willing to learn it than Rust or Go.
rtomayko 11y ago
I'm as keen to see memory safe systems languages become commonplace as anyone but some 90% of our stack is C - Linux, MySQL, Git, MRI, nginx, haproxy, memcached, redis, and many other internal components that we'll be talking about on the engineering blog soon. We like C. We're going to need a few more years of research, real production experience, and language/library maturity before betting critical infrastructure on something else by default.
- sagichmal 11y ago
  > We like C. We're going to need a few more years of > research, real production experience, and > language/library maturity before betting critical > infrastructure on something else by default. That's a really bizarre opinion to hold in the year 2015. Don't get me wrong: I love C, too. But certainly not for home-grown infrastructure at a web company — the incentives just don't align. And hiding behind the "we just don't know" bugbear doesn't parse, either. Go, for example, powers gargantuan-scale infrastructure at Google, and has for half a decade. And there's whole fleets of organizations as big or bigger than GitHub that report the same experience.
  - vidarh 11y ago
    Google also has a gargantuan-scale dev team that includes the people behind Go. It's ridiculous to compare. If Github does not yet themselves have sufficient people with sufficient experience supporting large services in Go, betting critical infrastructure on it would be irresponsible. Note that he is not saying they're not open to alternatives, nor that they're not experimenting with alternatives, but that they need more experience first before "betting critical infrastructure on something else by default".
    - sagichmal 11y ago
      > Google also has a gargantuan-scale dev team that > includes the people behind Go. It's ridiculous to > compare. It's not ridiculous. There are probably reasonable arguments against using Go for things like this, but "insufficient developer capacity" isn't one of them. Becoming a Go expert is a task measured in weeks.

kasey_junk 11y ago

I'm not sure what you are asking about specifically but in general it's important to remember that channels are a pretty thin abstraction around a mutex (and a buffer). There is nothing about that abstraction that would make you think it would be better for this use case than spinlocks and a lock free table, and quite a bit that would make you think it is much worse. One of the first things people abandon when scaling golang servers is the channel abstraction and I expect the same would be true in this case.