5 ms·

(I work on the HipHop compiler.) You start by compiling PHP source. Simple PHP statements take a lot more space in the binary than intuition suggests. E.g.: i

• cookiecaper 15y ago
  Based on your work heretofore, do you think it's wise for Facebook to continue on the PHP path instead of working on a backend rewrite in C# or some other, saner language? I find it odd that Facebook is still using PHP and pouring lots of effort and cash into things like HipHop when they're obviously hiring people smart enough to use another language, and when they obviously have the runway to perform a dark horse rewrite into a much cleaner, saner backend.
  • kmavm 15y ago
    This is a long and deep subject. I wouldn't say that we've "continued on the PHP path." I'd say that we've refused to throw out the precious PHP parts of our application, while not being afraid to use more appropriate languages across Thrift boundaries when needed. Our search engine, newsfeed, and ad serving infrastructure, for instance, are in C++. A drop-everything-and-rewrite of the PHP code is entirely out of the question, for all the reasons covered in Spolsky's 12-year-old classic on the subject: http://www.joelonsoftware.com/articles/fog0000000069.html http://www.joelonsoftware.com/articles/fog0000000069.html. Those of us working on making PHP perform better are a tiny fraction of Facebook engineering as a whole; this small overhead cost is nothing compared to the risks inherent in a ground-up rewrite. Most of PHP's language-level faults can be engineered around. For instance, we have a code-review-time script that parses (really parses) the code to warn engineers (and reviewers) about dangerous or deprecated idioms. PHP also has some affirmative virtues. The programming model is more productive than that of compiled languages, and even many interpreted languages; save/reload the web page is just a better, tighter loop to get work done in than save/compile/restart my server/reload the web page. I'm actually a fan of PHP's concurrency model, which naifs often mistake as "no concurrency allowed"; PHP's concurrency primitive is curl[1], and if you wrap a tiny bit of library around it, you can make it behave like actors. [1] Seriously. curl provides a shared-nothing way to asynchronously run code, and has the virtue of not caring what language the other side is written in to boot.
    • mrich 15y ago
      Thanks for your insights, but this makes me wonder: how much PHP code is there to make a rewrite unfeasible? At least the public part shouldn't be so big, are there huge administrative interfaces that somehow can't be separated?
    • cookiecaper 15y ago
      Right, I'm familiar with Spolsky's piece, but I think there are times when a rewrite is legitimate. I think that a situation where you must roll a completely custom in-house compiler that generates binaries which exceed 1 GB in size in order to get adequate performance of your app is a good candidate for a new architecture, despite Spolsky's claims. Spolsky's article discusses throwing out pages of code because the programmers "don't know what half of these API calls are for" and "[wanting] to build something grand" -- these are quite different impulses than the real-world problems staring Facebook in the face by its continued usage of PHP. I think also that there is a difference between writing a new backend for something that is solid and in place and just throwing the whole product out the window and re-imagining it from the ground up, and I think the latter is the kind of rewrite that should be avoided and considered dangerous. When you could throw 4-5 guys on a real C# or C++ rewrite and tell them the final product has to behave identically to the PHP version, you have a much less volatile situation. As for the PHP workflow, I agree it's nice not to have an intermediate step, but that intermediate step can usually be circumvented pretty rapidly by throwing a script or two (or just flipping a config option) into your development environment.
      • TeeWEE 15y ago
        PHP is not such a bad language. Of course it has its quirks. However bad code is written by a bad developer, not by a bad language. You can write beautiful code in PHP too. And the huge code base from Facebook is impossible to rewrite it in even a few years. Throwing away knowledge, tools, and tested code.
      • nbm 15y ago
        There is pretty much no way 4 to 5 guys could rewrite "www" in another language in parallel to continuing development from a hundred developers, no matter how long they had. However, 4 to 5 guys could write HipHop. What real-world problems do you see staring Facebook in the face in its continued usage of PHP? I'm no PHP fan, but many of the common PHP problems are avoided or mitigated in the code base.
  • underwater 15y ago
    To add to what others have said most Facebook hires learn PHP during the boot camp process. The continued use of the PHP is definitely not due to lack of experience with other languages or some misplaced fondness for the language. When you work on a codebase the size of Facebook's the core language becomes less important than the abstractions and framework that have been built around it. As well as having sane internal libraries Facebook have added extensions to the language like XHP and the yield keyword (http://www.serversidemagazine.com/news/10-questions-with-facebook-research-engineer-andrei-alexandrescu/ http://www.serversidemagazine.com/news/10-questions-with-fac...) which make it much nicer to use.
    • sciurus 15y ago
      How large is the "www" codebase?
      • underwater 15y ago
        To be honest I've never checked. Ola's answer here http://www.quora.com/How-many-lines-of-code-is-Facebook http://www.quora.com/How-many-lines-of-code-is-Facebook sounds about right (he seems to be talking about all code). I should amend my previous comment though; most place I've worked at that are within a couple of orders of magnitude of Facebook's size have developed or adopted abstractions that constitute the majority of code engineers interact with.
• dbarry 15y ago
  I'd be interested in learning how the HipHop JIT is fitting into this picture, if you care to comment.
  • kmavm 15y ago
    The HHVM JIT isn't in (much of) production yet, because it isn't fast enough yet. Once we have high confidence it can provide some perf wins, we'll go ahead and move over to it. Even if it were perf neutral, there is some virtue to having developers use the same environment as production.
• pbiggar 15y ago
  How good is the type inference? I did some research on PHP back in the day, and reckoned that a huge number of types (80%-90%) could be inferred - but I didn't run it on anything like Facebook's codebase. Also, more info on the type inference please :) Context-sensitive? Do you model reference counts in your analysis? If so, do you get optimizations out of that? Did the article really say a compile is 15 minutes? That's insane - how is it so fast? Does it limit the optimizations? You mustn't be able to do very deep analysis - or do you cache it somehow? What about FDO? I've argued for years that the PHP interpreter's implementation leak into the language semantics, but you seem to have cast iron proof about the refcounts. Is it just destructors, or do you see errors due to copy-on-write semantics (like the weird array semantics when a reference is in a copy-on-writed array)? (Sorry for all the questions, it's just very interesting)
  • kmavm 15y ago
    I haven't done any work on the type inference stuff, so my understanding is second-hand. My peers who actually know what they're talking about are working on a more complete write-up, so stay tuned. 1. (Type inference.) The compiler's type inference is basically a first-order symbolic execution without inlining; we don't iterate to a fixed point or anything, and any union types at all cause us to give up and call it a variant. Many other PHP surprises mess up type inference; for example, function parameters by reference aren't marked in the caller, so if anywhere in the codebase there exists a unary function that takes its first parameter by reference, $f($a) will spoil our inference of $a: $f might mutate it. Still, simple things work pretty well. One major area we do well on is inferring the return types of function calls. It's hard to say how much we're missing out by not doing more sophisticated type inference; while we can count the sites at compile time, this doesn't tell us which sites are dynamically important. Our anecdotal experience has been that type inference is one of the more powerful optimizations we can perform. We don't model refcounts, and I'm certain there are opportunities there. 2. (Compile times.) 15 minutes sounds about right to me, and chuckr would certainly know better. Compiler speed itself is something we pay attention to, since it inhibits our ability to turn pushes around. The actual global AST analysis is multithreaded, so we keep the whole build machine busy. So far the trade-offs between optimizations and compile time have not been too dire; there haven't been things that would make the binary twice as fast if only we could compile twenty times as long, e.g. 3. (PHP implementation/semantics). Who've you been "arguing" against? :). It's a manifest fact that accidents of PHP's implementation have been sunk into the language. Destructors are one of the ways to witness refcounts, but there are others. var_dump() of arrays exposes the refcount of internal data; things whose refcount is >1 are displayed with a '&' preceding them, e.g.: <? $a[0] = 1; var_dump($a); $data =& $a[0]; var_dump($a); unset($data); var_dump($a); ?> produces array(1) { [0]=> int(1) } array(1) { [0]=> &int(1) } array(1) { [0]=> int(1) } Less trivially, it is hard to efficiently support the copy semantics for arrays without reference counting; other systems have overcome this, but it is less trivial than with refcounting.