8 ms·

Using Go at The New York Times [video]

140 points by somerandomness 11y ago

hueving 11y ago
Sometimes I really wonder if the choice of the language is just a way to filter out 'boring' engineers. It doesn't seem like they particularly needed to use Go, but look at the attention it gets them for being 'cutting edge'.
- spoutn1k 11y ago
  Could you elaborate on what you mean by "boring engineers"?
  - ARCarr 11y ago
    Engineers who don't want to try new things.
  - hueving 11y ago
    Sadly it's people who don't keep up with what's cool on hackernews.
  - mkawia 11y ago
    I read/heard yahoo suffers from this problem ,one of the reason Marissa Mayer is struggling.
  - thrwwy9 11y ago
    The ones who actually know their history and consequently, when they hear "NoSQL," they think of IBM's IMS and run as fast as they can in the opposite direction.
- meowface 11y ago
  pg touches on this in his Blub essay: http://www.paulgraham.com/avg.html http://www.paulgraham.com/avg.html >The first thing I would do, after checking to see if they had a live online demo, was look at their job listings. After a couple years of this I could tell which companies to worry about and which not to. The more of an IT flavor the job descriptions had, the less dangerous the company was. The safest kind were the ones that wanted Oracle experience. You never had to worry about those. You were also safe if they said they wanted C++ or Java developers. If they wanted Perl or Python programmers, that would be a bit frightening-- that's starting to sound like a company where the technical side, at least, is run by real hackers. If I had ever seen a job posting looking for Lisp hackers, I would have been really worried. I think it's a valid recruitment strategy, though it does sometimes cause languages' hype to outweigh their actual value, which many would argue is the case for Go (even though Go is generally a nice language).
  - illicium 11y ago
    pg, being a Lisp hacker, is really stroking his ego there.
  - bootload 11y ago
    "I think it's a valid recruitment strategy, though it does sometimes cause languages' hype to outweigh their actual value" Not always, found this example yesterday: Farmlogs, Clojure and image processing ~ https://news.ycombinator.com/item?id=9522397 https://news.ycombinator.com/item?id=9522397
  - mahyarm 11y ago
    TBH, I feel like go is the next java almost. But I'm an objective-c programmer, so rust's non-gc memory management is ok with me.
- emehrkay 11y ago
  If I'm not mistaken, he did mention that one server's memory requirements shrank from 4gb to roughly 1. It's kinda amazing that they use so many different tools
  - nkozyra 11y ago
    That's true of any very big organization and news companies are very compartmentalized and segregated. I recall quite a few different dev groups not including R&D at the Times that used dissonant tools - I didn't even see RoR listed, which is used extensively in news teams within the Times.
- nkozyra 11y ago
  As a big fan and daily user of Go, I agree with this. I've used it to solve concurrency problems or problems wherein I could quickly replace a slower long-running process in Python with something just as simple but more performant. I see a lot of codebase refactoring in Go that doesn't really need to be written in Go. I'm not sure what the impetus is if not to just do it or to reposition a staff's codebase. The argument for these sorts of things should be better expressed, in my opinion. "We moved to language X because ..." should be a simple 1-line explanation, even if it is indeed "we want to work in the language or attract a certain type of developer."
- brnstz 11y ago
  (Disclaimer: I'm a former engineer at NYT and was a Go proponent there.) The reality of the situation at NYT is that every dollar counts. Go is an effective language that enables a programmer to do both "scripting" and "real" tasks without switching between different environments. The email system described sends more email than many companies that only send email send, and have many more employees working on said system. It is one of the few systems at NYT that pays for itself with zero doubt. The speaker did not know Go before being hired by NYT, and famously solved a Java programming task in his interview without an IDE and barely a typo (the only time I'd ever seen it done successfully, some people can't even run javac or vim). It certainly doesn't hurt in hiring to say that you get to program Go, but filtering out "boring" engineers is not a chase of "coolness". It is seeking out engineers who want to take a risk and want to be productive.
  - hueving 11y ago
    No, I get that you want to filter out boring engineers. It's just unfortunate that it seems the only way our industry can do this is by changing up the tools and seeing who makes it through. Some of the smartest engineers I know are not interested in go because they think it solved the wrong problems and prefer rust for something new.
    - robmccoll 11y ago
      I've been writing in Go at work nearly every day for the past year. I think that of the problems both Go and Rust attempt to solve, Rust solves them better in a technical and academic sense. As someone who prefers C and C++ over Python and Java, I personally would rather use Rust, but I have to say that Go strikes a great balance for general-purpose use. It is as fast to develop and as flexible and extendable as Python. Similar performance to Java but with much less code, better interoperability, and a more robust standard library. More features than C without the needless complexity of C++. Entirely too opinionated and the build system and dependency management are 80% solutions / hacks, but for the most part you can work around them. Most importantly, I think Go can be easily picked up and used by C, C++, Java, PHP, JavaScript, Python, C#, Objective-C, XYZ, ETC programmers alike with very little learning curve (some more than others perhaps). I think with Rust, you are going to have a hard time training someone who has depended on GC for so long that they have to be explicit about the ownership and transfer of ownership of memory and that it's elegant and cool and performant to do so. You are going to have trouble explaining traits and composability and the syntax around it to someone who is used to using more verbose syntax to implement complicated inheritance trees. Pattern matching and enums are fantastic, but also foreign to most of the programming public. This is why Go is spreading - good middle ground for productivity and performance, easy to pick up, seemingly bright future, Google backing, good documentation (no doc is perfect - Sphinx is frequently not great, Javadoc is terrible, Doxygen is meh, MSDN is overgrown with cruft - but Godoc is decent), etc.
- nemothekid 11y ago
  I recall reading something similar wrt Scala & Twitter. At the time Twitter really needed to scale up they chose Scala and had the effect of attracting people who wanted to work at a company because they used Scala. I can't find the link however
- thrwwy9 11y ago
  Granted, you did put "cutting edge" in quotes, but it's still worth stating explicitly, Go isn't cutting edge, and neither are Ruby, Node.js, NoSQL, or most of the other technological rehashes ignorant developers get excited over. These people would never pay any attention to old, boring technology like Lisp or Smalltalk or even (given the success of NoSQL) ACID-compliant relational databases like PostgreSQL. But repackage some of the good ideas from them and pass them off as your own, fire up your hype machine, and these clueless devs will eat it up. Why, look at the cult-like following Matz enjoys for creating a half-baked Smalltalk and Perl knock-off.
  - brightball 11y ago
    90% of the value created by using Ruby is because of the exemplary tools put out there by the people IN that cult-like following. There's a lot of programming languages and programmers totally focussed on application performance as well as squeezing the limits out of their hardware. The ruby community is focussed on developer performance and squeezing the limits out of their people. If more languages and communities spent as much time on streamlining everything around time to market, Ruby would not be nearly as big as it is. Because most are not focussed on that, Ruby stands out significantly when it comes to getting big things done with fewer people. That's why so many start ups use it. That's also why so many start ups eventually come back and refactor the slower parts of their system with something like Go...but the important thing is that they got to market, built enough of a customer base and revenue stream that they can both a) afford to refactor and b) need to refactor because so many people are using it. Building a language and tools around people rather than process is a unique undertaking. Because of that Ruby and Matz deserve every ounce of credit that they get. Doesn't have to be right for everything, but what they try to do well they do exceptionally well.
- taf2 11y ago
  it actually makes good business sense for a new organization to capitalize on these decisions. It is similar to a consulting agency - show off the use of a cool/upandcoming programming language or approach and earn more eyes or potential customers. Side benefit- the team also gets to improve their skills and enjoy the job more.
ScotterC 11y ago
Slides: https://speakerdeck.com/nytdevs/using-go-at-the-new-york-times https://speakerdeck.com/nytdevs/using-go-at-the-new-york-tim...
vinceyuan 11y ago
Are Go servers at New York Times or other companies impacted by the Golang GC pause? I am learning Golang and want to know if Golang GC pause is a significant issue in production.
- _ak 11y ago
  We're doing RTB with Google DoubleClick AdX, and our bidder (written in Go) can easily handle 4000 QPS per server with only a 0.3 % error rate.
  - vinceyuan 11y ago
    Is 0.3% error rate caused by GC pause?
    - _ak 11y ago
      From what we can tell, yes. But, according to our Google contacts, we're among the top performers, most parties taking part in RTB perform a lot worse.
mrfusion 11y ago
Can we change the title to "Golang"?
agumonkey 11y ago
It's a relief to see post-redesign diagrams. But at the same time I have tiny 'bus factor' fear by relying on a single language, is it (that) absurd ?

140 points by somerandomness 11y ago

Sometimes I really wonder if the choice of the language is just a way to filter out 'boring' engineers. It doesn't seem like they particularly needed to use Go, but look at the attention it gets them for being 'cutting edge'.

spoutn1k 11y ago
Could you elaborate on what you mean by "boring engineers"?
- ARCarr 11y ago
  Engineers who don't want to try new things.
- hueving 11y ago
  Sadly it's people who don't keep up with what's cool on hackernews.
- mkawia 11y ago
  I read/heard yahoo suffers from this problem ,one of the reason Marissa Mayer is struggling.
- thrwwy9 11y ago
  The ones who actually know their history and consequently, when they hear "NoSQL," they think of IBM's IMS and run as fast as they can in the opposite direction.
meowface 11y ago
pg touches on this in his Blub essay: http://www.paulgraham.com/avg.html http://www.paulgraham.com/avg.html >The first thing I would do, after checking to see if they had a live online demo, was look at their job listings. After a couple years of this I could tell which companies to worry about and which not to. The more of an IT flavor the job descriptions had, the less dangerous the company was. The safest kind were the ones that wanted Oracle experience. You never had to worry about those. You were also safe if they said they wanted C++ or Java developers. If they wanted Perl or Python programmers, that would be a bit frightening-- that's starting to sound like a company where the technical side, at least, is run by real hackers. If I had ever seen a job posting looking for Lisp hackers, I would have been really worried. I think it's a valid recruitment strategy, though it does sometimes cause languages' hype to outweigh their actual value, which many would argue is the case for Go (even though Go is generally a nice language).
- illicium 11y ago
  pg, being a Lisp hacker, is really stroking his ego there.
- bootload 11y ago
  "I think it's a valid recruitment strategy, though it does sometimes cause languages' hype to outweigh their actual value" Not always, found this example yesterday: Farmlogs, Clojure and image processing ~ https://news.ycombinator.com/item?id=9522397 https://news.ycombinator.com/item?id=9522397
- mahyarm 11y ago
  TBH, I feel like go is the next java almost. But I'm an objective-c programmer, so rust's non-gc memory management is ok with me.
emehrkay 11y ago
If I'm not mistaken, he did mention that one server's memory requirements shrank from 4gb to roughly 1. It's kinda amazing that they use so many different tools
- nkozyra 11y ago
  That's true of any very big organization and news companies are very compartmentalized and segregated. I recall quite a few different dev groups not including R&D at the Times that used dissonant tools - I didn't even see RoR listed, which is used extensively in news teams within the Times.
nkozyra 11y ago
As a big fan and daily user of Go, I agree with this. I've used it to solve concurrency problems or problems wherein I could quickly replace a slower long-running process in Python with something just as simple but more performant. I see a lot of codebase refactoring in Go that doesn't really need to be written in Go. I'm not sure what the impetus is if not to just do it or to reposition a staff's codebase. The argument for these sorts of things should be better expressed, in my opinion. "We moved to language X because ..." should be a simple 1-line explanation, even if it is indeed "we want to work in the language or attract a certain type of developer."
brnstz 11y ago
(Disclaimer: I'm a former engineer at NYT and was a Go proponent there.) The reality of the situation at NYT is that every dollar counts. Go is an effective language that enables a programmer to do both "scripting" and "real" tasks without switching between different environments. The email system described sends more email than many companies that only send email send, and have many more employees working on said system. It is one of the few systems at NYT that pays for itself with zero doubt. The speaker did not know Go before being hired by NYT, and famously solved a Java programming task in his interview without an IDE and barely a typo (the only time I'd ever seen it done successfully, some people can't even run javac or vim). It certainly doesn't hurt in hiring to say that you get to program Go, but filtering out "boring" engineers is not a chase of "coolness". It is seeking out engineers who want to take a risk and want to be productive.
- hueving 11y ago
  No, I get that you want to filter out boring engineers. It's just unfortunate that it seems the only way our industry can do this is by changing up the tools and seeing who makes it through. Some of the smartest engineers I know are not interested in go because they think it solved the wrong problems and prefer rust for something new.
  - robmccoll 11y ago
    I've been writing in Go at work nearly every day for the past year. I think that of the problems both Go and Rust attempt to solve, Rust solves them better in a technical and academic sense. As someone who prefers C and C++ over Python and Java, I personally would rather use Rust, but I have to say that Go strikes a great balance for general-purpose use. It is as fast to develop and as flexible and extendable as Python. Similar performance to Java but with much less code, better interoperability, and a more robust standard library. More features than C without the needless complexity of C++. Entirely too opinionated and the build system and dependency management are 80% solutions / hacks, but for the most part you can work around them. Most importantly, I think Go can be easily picked up and used by C, C++, Java, PHP, JavaScript, Python, C#, Objective-C, XYZ, ETC programmers alike with very little learning curve (some more than others perhaps). I think with Rust, you are going to have a hard time training someone who has depended on GC for so long that they have to be explicit about the ownership and transfer of ownership of memory and that it's elegant and cool and performant to do so. You are going to have trouble explaining traits and composability and the syntax around it to someone who is used to using more verbose syntax to implement complicated inheritance trees. Pattern matching and enums are fantastic, but also foreign to most of the programming public. This is why Go is spreading - good middle ground for productivity and performance, easy to pick up, seemingly bright future, Google backing, good documentation (no doc is perfect - Sphinx is frequently not great, Javadoc is terrible, Doxygen is meh, MSDN is overgrown with cruft - but Godoc is decent), etc.
nemothekid 11y ago
I recall reading something similar wrt Scala & Twitter. At the time Twitter really needed to scale up they chose Scala and had the effect of attracting people who wanted to work at a company because they used Scala. I can't find the link however
thrwwy9 11y ago
Granted, you did put "cutting edge" in quotes, but it's still worth stating explicitly, Go isn't cutting edge, and neither are Ruby, Node.js, NoSQL, or most of the other technological rehashes ignorant developers get excited over. These people would never pay any attention to old, boring technology like Lisp or Smalltalk or even (given the success of NoSQL) ACID-compliant relational databases like PostgreSQL. But repackage some of the good ideas from them and pass them off as your own, fire up your hype machine, and these clueless devs will eat it up. Why, look at the cult-like following Matz enjoys for creating a half-baked Smalltalk and Perl knock-off.
- brightball 11y ago
  90% of the value created by using Ruby is because of the exemplary tools put out there by the people IN that cult-like following. There's a lot of programming languages and programmers totally focussed on application performance as well as squeezing the limits out of their hardware. The ruby community is focussed on developer performance and squeezing the limits out of their people. If more languages and communities spent as much time on streamlining everything around time to market, Ruby would not be nearly as big as it is. Because most are not focussed on that, Ruby stands out significantly when it comes to getting big things done with fewer people. That's why so many start ups use it. That's also why so many start ups eventually come back and refactor the slower parts of their system with something like Go...but the important thing is that they got to market, built enough of a customer base and revenue stream that they can both a) afford to refactor and b) need to refactor because so many people are using it. Building a language and tools around people rather than process is a unique undertaking. Because of that Ruby and Matz deserve every ounce of credit that they get. Doesn't have to be right for everything, but what they try to do well they do exceptionally well.
taf2 11y ago
it actually makes good business sense for a new organization to capitalize on these decisions. It is similar to a consulting agency - show off the use of a cool/upandcoming programming language or approach and earn more eyes or potential customers. Side benefit- the team also gets to improve their skills and enjoy the job more.

ScotterC 11y ago

Slides: https://speakerdeck.com/nytdevs/using-go-at-the-new-york-times https://speakerdeck.com/nytdevs/using-go-at-the-new-york-tim...

vinceyuan 11y ago

Are Go servers at New York Times or other companies impacted by the Golang GC pause? I am learning Golang and want to know if Golang GC pause is a significant issue in production.

_ak 11y ago
We're doing RTB with Google DoubleClick AdX, and our bidder (written in Go) can easily handle 4000 QPS per server with only a 0.3 % error rate.
- vinceyuan 11y ago
  Is 0.3% error rate caused by GC pause?
  - _ak 11y ago
    From what we can tell, yes. But, according to our Google contacts, we're among the top performers, most parties taking part in RTB perform a lot worse.

mrfusion 11y ago

Can we change the title to "Golang"?

agumonkey 11y ago

It's a relief to see post-redesign diagrams. But at the same time I have tiny 'bus factor' fear by relying on a single language, is it (that) absurd ?