7 ms·

tl;dr We rewrote some internal Python software in Go, it is now faster. Unfortunately, there is no technical content (which seems endemic to the recent storm o

tl;dr We rewrote some internal Python software in Go, it is now faster. Unfortunately, there is no technical content (which seems endemic to the recent storm of Go articles). Is it faster because is a static language producing machine code? Was their choice of Python packages wrong? How much are the performance improvements caused by knowing better where the bottlenecks are? Were other languages with larger ecosystems considered?

mattrobenolt 13y ago
This was partially intention on my part. A lot of the problems that we had with our Python backends were a bit unknown. We had some guesses, and not the right instrumentation to dig into it. There are memory leaks in gevent, and the CPU that was being consumed was ridiculous. Internally, we had each played with Go for a little bit, and felt very natural to us. There weren't any other considerations really. For comparison, we're still dark writing to the old realtime just to make sure that nothing is broken, and old realtime is consuming easily 16x resources and technically doing a bit less since it's not really publishing anymore.
- vanderZwan 13y ago
  My first compsci teacher was Harm Bakker aka Harm the Almighty Recursion Master, and anyone who followed his classes has the following quote imprinted in their brains: "... and then it's tempting to start kludging. I have a suggestion: get it right the first time." Whenever I'm programming I can still feel his disappointed stare of disapproval when applying a quick and dirty hack.
  - mattrobenolt 13y ago
    Fortunately, it's not as black and white in the real world. Quick and dirty hacks are a real thing that will never go away. If anything, having the ability to do those under stressful scenarios is a good quality to have.
    - vanderZwan 13y ago
      Oh, I know. But I still think it's good to feel guilty about it so you feel the need to fix it properly later. EDIT: And of course, there's learning from your mistakes so you can get it right on the first try the next time you have to do a similar thing.
  - druiid 13y ago
    Someone already responded with hacks being a fact of life. There's basically CS programming, and then 'real life'. If there wasn't a need to generally get things out the door I imagine there would be far fewer exploits of software out there.
  - saidajigumi 13y ago
    > "... and then it's tempting to start kludging. I have a suggestion: get it right the first time." Where this falls down in the real world is that oftentimes your understanding of the problem is quite limited at first. You'll ship a demo or an MVP, and that will teach you a little. As you continue to explore the problem through your work you'll often redefine it. Or it will redefine itself, as happens in so many war stories of service scaling, product maturation, and so on.
  - coldtea 13y ago
    Yes, if you're working in the academia, you have the luxury to say such BS. Let's see Harm Bakker "get it right the first time" under changing conditions, sloppy inherited code, a pointy haired boss, etc...
noelwelsh 13y ago
This. Is it really a surprise that the Python VM is not performant? I thought everyone knew that by now. Any reasonably well engineered VM (e.g. V8, JVM, Go) will blow it out of the water. This should be well known to anyone who calls themselves a software engineer. The Disqus system could have been written in a large number of other languages and have performed more or less the same as the Go system. The only advantage I can see of Go is that it's reasonably close to Python/Ruby in terms of semantics. If Python or Ruby is your only tool then Go might be a good choice. Not so if, for example, you're more into the functional style. Edit: Ok, so Go doesn't use a VM. It still has a runtime, as all languages must. Substitute runtime for VM above as appropriate. Point still stands.
- voidlogic 13y ago
  >>Any reasonably well engineered VM (e.g. V8, JVM, Go) will blow it out of the water. FYI, Go does not use a VM like V8 or the JVM; Go code compiles to native binary. However, like many VM based languages and unlike (a lot of) C, Go should be write once, (compile) and run everywhere(Win,Lin,BSD,Mac). Also like many VM languages Go is type/memory safe and is garbage collected. Edit: It appears V8 is also not a VM, it is a interpreter that performs just-in-time (JIT) compilation of source with no intermediate representation such as bytecode (Java instead JITs bytecode). This makes sense since Java programs are distributed as bytecode and Javascript programs are distributed as source. @calinet6: Thanks for pointing this out. Life-cycles: Developer's machine | Runtime -------------------------------------------------------- Go: source->binary | exec binary Java: source->bytecode | interp. bytecode / JIT & exec bin JS: source | interp. source / JIT & exec bin
  - pjmlp 13y ago
    It is going to take ages to clear young developer minds that safe strong typed languages don't require a VM and we have to thank Sun and Microsoft to have spread that misconception.
    - pstuart 13y ago
      As an older developer, I'm ok with that :-)
lysium 13y ago
> We rewrote some internal Python software in Go ...and dropped a link (Thoonk) in the processing chain...
xb95 13y ago
For me, the really interesting point of the article was less "yay, it's faster than Python" (which is generally undisputed), but more that (to paraphrase): "I didn't know anything about Go other than 'it sounds cool', so I decided to rewrite this very important bit of Disqus' infrastructure. In one week, from start to finish, I had it implemented, tested, and deployed and improved latencies by several orders of magnitude and reduced the capacity requirement from four fully loaded servers to 20% of one!" To be able to say that is pretty impressive, worth writing about, and definitely interesting to me as someone who wants to see Go used more. Disqus is well known, has lots of traffic, and so a recommendation from them carries a lot more weight than some random benchmarks. (As cool as they are.)
ExpiredLink 13y ago
> the recent storm of Go articles Maybe an insider could shed some light on this campaign. Does Google marketing dedicate people to cater for social networking sites and news aggregators? How is the 'storm' organized? Obviously, many startups try to spread the word through 'cheap' channels. Only few are really successful.
- eridius 13y ago
  I don't think this is organized by Google. I think it's just a natural consequence of more people checking out and using Go for production work. For what it's worth, outside of the FAQ, I don't think golang.org even mentions Google. In general, Go is kept very distinct from Google, so it wouldn't make much sense for Google to embark on a Go marketing campaign.
- voidlogic 13y ago
  You do realize that Go is an open-source project with contributors from both inside and outside of Google? I imagine more user's outside of Google use Go than inside.
orangethirty 13y ago
1. Yes, that is one of the reasons for Go's speed. 2. You don't have many options aside from Gevent. 3. I can't talk for them, but my Python bottlenecks were mainly memory related. I was simply using up too much of it, due to how it was handling each request. My pattern was a basic Data/Handler/Json one. Which is as simple as they come. Yet, I was having issues (at a small scale). 4. Yes, of course. I considered and built parts using other languages. Did some testing, too. But Go stood out as the best choice given my needs.

mattrobenolt 13y ago

This was partially intention on my part. A lot of the problems that we had with our Python backends were a bit unknown. We had some guesses, and not the right instrumentation to dig into it. There are memory leaks in gevent, and the CPU that was being consumed was ridiculous. Internally, we had each played with Go for a little bit, and felt very natural to us. There weren't any other considerations really. For comparison, we're still dark writing to the old realtime just to make sure that nothing is broken, and old realtime is consuming easily 16x resources and technically doing a bit less since it's not really publishing anymore.

vanderZwan 13y ago
My first compsci teacher was Harm Bakker aka Harm the Almighty Recursion Master, and anyone who followed his classes has the following quote imprinted in their brains: "... and then it's tempting to start kludging. I have a suggestion: get it right the first time." Whenever I'm programming I can still feel his disappointed stare of disapproval when applying a quick and dirty hack.
- mattrobenolt 13y ago
  Fortunately, it's not as black and white in the real world. Quick and dirty hacks are a real thing that will never go away. If anything, having the ability to do those under stressful scenarios is a good quality to have.
  - vanderZwan 13y ago
    Oh, I know. But I still think it's good to feel guilty about it so you feel the need to fix it properly later. EDIT: And of course, there's learning from your mistakes so you can get it right on the first try the next time you have to do a similar thing.
- druiid 13y ago
  Someone already responded with hacks being a fact of life. There's basically CS programming, and then 'real life'. If there wasn't a need to generally get things out the door I imagine there would be far fewer exploits of software out there.
- saidajigumi 13y ago
  > "... and then it's tempting to start kludging. I have a suggestion: get it right the first time." Where this falls down in the real world is that oftentimes your understanding of the problem is quite limited at first. You'll ship a demo or an MVP, and that will teach you a little. As you continue to explore the problem through your work you'll often redefine it. Or it will redefine itself, as happens in so many war stories of service scaling, product maturation, and so on.
- coldtea 13y ago
  Yes, if you're working in the academia, you have the luxury to say such BS. Let's see Harm Bakker "get it right the first time" under changing conditions, sloppy inherited code, a pointy haired boss, etc...

noelwelsh 13y ago

This. Is it really a surprise that the Python VM is not performant? I thought everyone knew that by now. Any reasonably well engineered VM (e.g. V8, JVM, Go) will blow it out of the water. This should be well known to anyone who calls themselves a software engineer. The Disqus system could have been written in a large number of other languages and have performed more or less the same as the Go system. The only advantage I can see of Go is that it's reasonably close to Python/Ruby in terms of semantics. If Python or Ruby is your only tool then Go might be a good choice. Not so if, for example, you're more into the functional style. Edit: Ok, so Go doesn't use a VM. It still has a runtime, as all languages must. Substitute runtime for VM above as appropriate. Point still stands.

voidlogic 13y ago
>>Any reasonably well engineered VM (e.g. V8, JVM, Go) will blow it out of the water. FYI, Go does not use a VM like V8 or the JVM; Go code compiles to native binary. However, like many VM based languages and unlike (a lot of) C, Go should be write once, (compile) and run everywhere(Win,Lin,BSD,Mac). Also like many VM languages Go is type/memory safe and is garbage collected. Edit: It appears V8 is also not a VM, it is a interpreter that performs just-in-time (JIT) compilation of source with no intermediate representation such as bytecode (Java instead JITs bytecode). This makes sense since Java programs are distributed as bytecode and Javascript programs are distributed as source. @calinet6: Thanks for pointing this out. Life-cycles: Developer's machine | Runtime -------------------------------------------------------- Go: source->binary | exec binary Java: source->bytecode | interp. bytecode / JIT & exec bin JS: source | interp. source / JIT & exec bin
- pjmlp 13y ago
  It is going to take ages to clear young developer minds that safe strong typed languages don't require a VM and we have to thank Sun and Microsoft to have spread that misconception.
  - pstuart 13y ago
    As an older developer, I'm ok with that :-)

lysium 13y ago

> We rewrote some internal Python software in Go ...and dropped a link (Thoonk) in the processing chain...

xb95 13y ago

For me, the really interesting point of the article was less "yay, it's faster than Python" (which is generally undisputed), but more that (to paraphrase): "I didn't know anything about Go other than 'it sounds cool', so I decided to rewrite this very important bit of Disqus' infrastructure. In one week, from start to finish, I had it implemented, tested, and deployed and improved latencies by several orders of magnitude and reduced the capacity requirement from four fully loaded servers to 20% of one!" To be able to say that is pretty impressive, worth writing about, and definitely interesting to me as someone who wants to see Go used more. Disqus is well known, has lots of traffic, and so a recommendation from them carries a lot more weight than some random benchmarks. (As cool as they are.)

ExpiredLink 13y ago

> the recent storm of Go articles Maybe an insider could shed some light on this campaign. Does Google marketing dedicate people to cater for social networking sites and news aggregators? How is the 'storm' organized? Obviously, many startups try to spread the word through 'cheap' channels. Only few are really successful.

eridius 13y ago
I don't think this is organized by Google. I think it's just a natural consequence of more people checking out and using Go for production work. For what it's worth, outside of the FAQ, I don't think golang.org even mentions Google. In general, Go is kept very distinct from Google, so it wouldn't make much sense for Google to embark on a Go marketing campaign.
voidlogic 13y ago
You do realize that Go is an open-source project with contributors from both inside and outside of Google? I imagine more user's outside of Google use Go than inside.

orangethirty 13y ago

1. Yes, that is one of the reasons for Go's speed. 2. You don't have many options aside from Gevent. 3. I can't talk for them, but my Python bottlenecks were mainly memory related. I was simply using up too much of it, due to how it was handling each request. My pattern was a basic Data/Handler/Json one. Which is as simple as they come. Yet, I was having issues (at a small scale). 4. Yes, of course. I considered and built parts using other languages. Did some testing, too. But Go stood out as the best choice given my needs.