4 ms·

> What was the stupid decision "made back then"? Using PHP to begin with was a bad technical decision. Failing to establish a reasonable migration strategy was

> What was the stupid decision "made back then"? Using PHP to begin with was a bad technical decision. Failing to establish a reasonable migration strategy was a bad business decision likely rooted in bad engineering management that fell out of starting with bad technical decisions. It's much harder to hire people that can pull you out of a mess like PHP when, at the same time, you have to hire people that can keep writing PHP for you. > That the initial team didn't rewrite Facebook in Python/Perl/Ruby/Haskell during the fast growth phase? That would have been a good time to bring on new engineering blood as part of scaling out, which would have provided opportunities to enact mitigation and transition strategies. Imagine if the massive amount of talent currently devoted to HHVM had been devoted to Facebook's actual business? There are migration strategies other than "rewrite everything immediately", and in fact, I'd bet that's exactly what HHVM is. It's just a shame they waited so long that the most cost-effective migration strategy was to tackle an enormously difficult computer science and engineering problem that the world's biggest software companies already invest hundreds of millions of dollars on and provide to the world largely for free. > If you have ever experienced the growth phase, you understand how ludicrous the idea of rewrite would be. Yes, and I've also (repeatedly) been the team brought in to rewrite the mess of a code base that was about to torpedo the growth phase. There's not much correlation between funding, initial success, and engineering talent. Which is why you so often wind up with a mess that has to be cleaned up once you can hire people who know what the hell they're doing, instead of the ones you happened to be stuck with because you didn't know how to grow an engineering team. > We underestimate how much hidden wisdom a production code base has and that the messiness is often there for good reasons. Messiness is never there for good reasons other than that replacing it is more expensive than not touching it. You don't strive for hidden wisdom and inescapable messiness -- that's just what you get when you let engineering slip up.

nbm 13y ago
As much as I prefer not to engage trolls... Which language would have been a good technical decision in 2002-2003? It needs to be fast enough in terms of iteration. It needs to not require more resources than PHP. It must be easy to onboard people who don't know it onto. It needs to be easy to operate, and not be costly to deploy on the tens, hundreds, and then thousands of servers necessary. (Spending time learning a new technology that others think is cool or which seems cool, while trying to also build a product, has sunk more than a few startups...) What was bad about the decision to keep the reasonably well-performing and reasonably suited-to-purpose PHP code for front-end code, and peel off suitable tasks into services like the feed, typeahead, messages, and so forth into languages like C++, Java, and so forth. What was bad with the decision to let hundreds of software engineers continue to build the PHP code-base while a much smaller group of people work on projects to improve the efficiency of both the execution environment but also the tooling and developer efficiency on that code-base? Their contribution there is multiplied out by the improved efficiency of those hundreds of developers and the code they wrote. Seemingly Facebook survived its growth phase fairly well and didn't need a tiger team from outside to handle it - and without the reliability problems others who chose to use languages other than PHP for their front-end and decided to rewrite their much smaller surface area in other languages. As much as people may dislike PHP (and I'm one of them), it was definitely "good enough". Many languages may not have been, even if they are nicer languages in some objective way. (Disclaimer: I actually have to write code in Facebook's PHP code-base every once in a while. But most of the code I work with is Python, Java, or C++.)
- teacup50 13y ago
  > Which language would have been a good technical decision in 2002-2003? It must be easy to onboard people who don't know it onto. It needs to be easy to operate, and not be costly to deploy on the tens, hundreds, and then thousands of servers necessary. You mean, like the JVM? 2003 wasn't the pliocene epoch, we had a working JVM. If you remember back to the last bubble, in the 90s, we were shipping "easy to operate, not costly to deploy" software on Java post-1998. Java 1.4 was released in 2002, and Java 1.5 -- what most people would say is modern Java -- was only 2004. Scala 2.0 was released in 2006, Clojure in 2007 -- that's 8 and 7 years ago, respectively. You really don't think there were alternatives during that long period? > What was bad about the decision to keep the reasonably well-performing and reasonably suited-to-purpose PHP code for front-end code, and peel off suitable tasks into services like the feed, typeahead, messages, and so forth into languages like C++, Java, and so forth. In 2004? Nothing. In 2005-2008? Things should have been reassessed, especially before building out a millstone of an engineering team around PHP. Instead, Facebook doubled-down on an actively bad language with HPHP, and the results were hilarious: HPHPc required a very different push process, requiring a bigger than 1 GB binary to be compiled and distributed to many machines in short order. So then in 2010, Facebook decides to embark on HHVM, and now four years later, we can run one of the most correctness-hostile programming languages around, quickly, with optional static typing. That's a span of 6 years, and at the end of it, Facebook has functionality they could have gotten for free in 2003. On top of that, the intervening years allowed the PHP mess to become only more entrenched -- who on earth do you think the engineers are that accept a job writing PHP, for Facebook or otherwise? If I had to hazard a guess, I'd guess that HHVM exists because of a large amount of political inertia in the organization that has everything to lose by PHP being eliminated entirely, and the lack of a strong hand by upper management. I'd guess that lack of a strong hand by upper management came in no small part from hiring straight-out-of-college graduate Adam D'Angelo -- who had literally zero experience -- to serve as CTO from 2006-2008. By the time FriendFeed was acquired and Bret Taylor along with it (2009), my guess would be entrenched interests made for a very difficult position for anyone wanting to change the ship's course.
  - Lennie 13y ago
    "the intervening years allowed the PHP mess to become only more entrenched" They specifically choose to do it and not move to an other language. One of the reasons was: PHP programmers are cheap and plenty full and can do quick iterations. Sounds a bit like you are complaining about other peoples choices, it really is their choice. :-) I'm not saying it is possible to move to an other language. Just look at Paypal they moved their customer-facing code from Java to node.js and got a very large productivity increase: http://www.youtube.com/watch?v=V5yk5SZxWX4 http://www.youtube.com/watch?v=V5yk5SZxWX4 Obviously the reason Paypal choose node.js are similar to why Facebook choose PHP: quick iterations, means more iterations, which means more experimentation and better results.
bluesnowmonkey 13y ago
> Using PHP to begin with was a bad technical decision. Failing to establish a reasonable migration strategy was a bad business decision likely rooted in bad engineering management that fell out of starting with bad technical decisions. Facebook is a ten year old company with a market capitalization of $170 billion, so "bad" is probably not the most accurate word to describe their technical and business decisions.
- teacup50 13y ago
  > Facebook is a ten year old company with a market capitalization of $170 billion, so "bad" is probably not the most accurate word to describe their technical and business decisions. How does that follow, exactly? They haven't failed, so any inefficient or sub-optimal decisions were the correct decisions?
  - curiousquestion 13y ago
    I guess we should consider your decisions as the "optimal and correct decisions"? Are you a billionaire too?

nbm 13y ago

As much as I prefer not to engage trolls... Which language would have been a good technical decision in 2002-2003? It needs to be fast enough in terms of iteration. It needs to not require more resources than PHP. It must be easy to onboard people who don't know it onto. It needs to be easy to operate, and not be costly to deploy on the tens, hundreds, and then thousands of servers necessary. (Spending time learning a new technology that others think is cool or which seems cool, while trying to also build a product, has sunk more than a few startups...) What was bad about the decision to keep the reasonably well-performing and reasonably suited-to-purpose PHP code for front-end code, and peel off suitable tasks into services like the feed, typeahead, messages, and so forth into languages like C++, Java, and so forth. What was bad with the decision to let hundreds of software engineers continue to build the PHP code-base while a much smaller group of people work on projects to improve the efficiency of both the execution environment but also the tooling and developer efficiency on that code-base? Their contribution there is multiplied out by the improved efficiency of those hundreds of developers and the code they wrote. Seemingly Facebook survived its growth phase fairly well and didn't need a tiger team from outside to handle it - and without the reliability problems others who chose to use languages other than PHP for their front-end and decided to rewrite their much smaller surface area in other languages. As much as people may dislike PHP (and I'm one of them), it was definitely "good enough". Many languages may not have been, even if they are nicer languages in some objective way. (Disclaimer: I actually have to write code in Facebook's PHP code-base every once in a while. But most of the code I work with is Python, Java, or C++.)

teacup50 13y ago
> Which language would have been a good technical decision in 2002-2003? It must be easy to onboard people who don't know it onto. It needs to be easy to operate, and not be costly to deploy on the tens, hundreds, and then thousands of servers necessary. You mean, like the JVM? 2003 wasn't the pliocene epoch, we had a working JVM. If you remember back to the last bubble, in the 90s, we were shipping "easy to operate, not costly to deploy" software on Java post-1998. Java 1.4 was released in 2002, and Java 1.5 -- what most people would say is modern Java -- was only 2004. Scala 2.0 was released in 2006, Clojure in 2007 -- that's 8 and 7 years ago, respectively. You really don't think there were alternatives during that long period? > What was bad about the decision to keep the reasonably well-performing and reasonably suited-to-purpose PHP code for front-end code, and peel off suitable tasks into services like the feed, typeahead, messages, and so forth into languages like C++, Java, and so forth. In 2004? Nothing. In 2005-2008? Things should have been reassessed, especially before building out a millstone of an engineering team around PHP. Instead, Facebook doubled-down on an actively bad language with HPHP, and the results were hilarious: HPHPc required a very different push process, requiring a bigger than 1 GB binary to be compiled and distributed to many machines in short order. So then in 2010, Facebook decides to embark on HHVM, and now four years later, we can run one of the most correctness-hostile programming languages around, quickly, with optional static typing. That's a span of 6 years, and at the end of it, Facebook has functionality they could have gotten for free in 2003. On top of that, the intervening years allowed the PHP mess to become only more entrenched -- who on earth do you think the engineers are that accept a job writing PHP, for Facebook or otherwise? If I had to hazard a guess, I'd guess that HHVM exists because of a large amount of political inertia in the organization that has everything to lose by PHP being eliminated entirely, and the lack of a strong hand by upper management. I'd guess that lack of a strong hand by upper management came in no small part from hiring straight-out-of-college graduate Adam D'Angelo -- who had literally zero experience -- to serve as CTO from 2006-2008. By the time FriendFeed was acquired and Bret Taylor along with it (2009), my guess would be entrenched interests made for a very difficult position for anyone wanting to change the ship's course.
- Lennie 13y ago
  "the intervening years allowed the PHP mess to become only more entrenched" They specifically choose to do it and not move to an other language. One of the reasons was: PHP programmers are cheap and plenty full and can do quick iterations. Sounds a bit like you are complaining about other peoples choices, it really is their choice. :-) I'm not saying it is possible to move to an other language. Just look at Paypal they moved their customer-facing code from Java to node.js and got a very large productivity increase: http://www.youtube.com/watch?v=V5yk5SZxWX4 http://www.youtube.com/watch?v=V5yk5SZxWX4 Obviously the reason Paypal choose node.js are similar to why Facebook choose PHP: quick iterations, means more iterations, which means more experimentation and better results.

bluesnowmonkey 13y ago

> Using PHP to begin with was a bad technical decision. Failing to establish a reasonable migration strategy was a bad business decision likely rooted in bad engineering management that fell out of starting with bad technical decisions. Facebook is a ten year old company with a market capitalization of $170 billion, so "bad" is probably not the most accurate word to describe their technical and business decisions.

teacup50 13y ago
> Facebook is a ten year old company with a market capitalization of $170 billion, so "bad" is probably not the most accurate word to describe their technical and business decisions. How does that follow, exactly? They haven't failed, so any inefficient or sub-optimal decisions were the correct decisions?
- curiousquestion 13y ago
  I guess we should consider your decisions as the "optimal and correct decisions"? Are you a billionaire too?