17 ms·
I am baffled as to why you'd build your castle atop a crumbling foundation. I have wondered why FB didn't use a proper language with proper typing to begin wit
by reikonomusha 13y ago
I am baffled as to why you'd build your castle atop a crumbling foundation.
I have wondered why FB didn't use a proper language with proper typing to begin with. I mean, I "understand" logistically: they already had a giant codebase in PHP, migrating a codebase is expensive, and it's difficult to hire and train 1000s of hackers in e.g., OCaml. (They do have some OCaml people, but they are outliers. OCaml was my favorite thing to write there, though it didn't afford some of the same niceties and interactivity as the PHP code they had, only because the support was down by several orders of magnitude.)
But at the same time, layering FP with a home rolled static type checking server (??) is bug prone and is certainly yak shaving (which they have time and money to do). Now they've written (1) a compiler to C++, (2) a compiler to VM byte code, (3) a corresponding runtime for each, (4) extensions to PHP, (5) a type checker, and (6) an inference engine. That's a lot of stuff. And in the end, it's still PHP, which is duly disliked. (Though Facebookers don't seem to care. The prevalent attitude toward it is that "PHP, as it's coded here, is mostly like C++, and that's OK.")
Writing correct type checkers and inference engines is kind of difficult. They seemed to take the approach of just building onto it incrementally until it just seems to work. That approach led to many bugs in many cases that just simply aren't thought of when one is trying to build inference engines by hand, as opposed according to theory. Type checking and inference is an area ripe with theory and attached formal, mathematical semantics. Standard ML's standard is perhaps the most infamous; it's a collection of mathematical statements about the language. That way, the compiler is now almost an engine to prove your code is correct. I don't see how the same guarantee can be made with something that is just cobbled together.
- itafroma 13y ago> I am baffled as to why you'd build your castle atop a crumbling foundation. > I have wondered why FB didn't use a proper language with proper typing to begin with. I would suggest watching Keith Adams's talk, "Taking PHP Seriously": http://www.infoq.com/presentations/php-history http://www.infoq.com/presentations/php-history He goes through why Facebook uses PHP and decided to build upon it to create Hack. I highly recommend watching the whole thing, but the main three things he points to are: 1. Frictionless programmer workflow with a short feedback cycle 2. All PHP requests start out with the same consistent state by default 3. Rigid style of concurrency
- eevee 13y agoWhich is nonsense, because: 1. I have never seen a framework that didn't go to great lengths to update to new changes quickly in development 2. Which means you lose resources between requests, unless you stuff them into the interpreter/httpd itself. And anyway this is only a problem for PHP, where by default everything runs in the top-level namespace, versus in separate functions. 3. That's a funny way to spin "no concurrency", but you can get that in any language by just not deploying it threaded.
- m3mnoch 13y agoblink blink um. have you used java?
- eevee 13y agonope
- Terr_ 13y agoWhat, you mean with "hot deployment" of your changes?
- eloisant 13y agoWell, even in Play Framework 1.0 in 2009 there was hot reloading. Edit a file, refresh your browser => it's there.
- Moru 13y agoOn the Atari ST you didn't even have to hit reload, the webpage reloaded automatically as soon as you saved your file. No need to alt-tab even, just keep webpage open next to your editor. Now get off my lawn.
- encoderer 13y agoSomething I didn't really learn in school, that I only picked up much later in my career, is that when applicable, applications should be built on top of mathematical models. A very contrived example would be.. would you build an ad rotator by keeping counts of all banners served and picking the next banner based on those counts and your weighting rules, or would you build the rotator on a foundation of statistics and probability with a little extra logic for handling caps and edge cases?
- yid 13y ago> would you build an ad rotator by keeping counts of all banners served and picking the next banner based on those counts and your weighting rules, or would you build the rotator on a foundation of statistics and probability with a little extra logic for handling caps and edge cases Actually, building on a foundation of solid statistics and probability will often result in an algorithm that essentially counts views and applies weighting rules. As another example, Pagerank has a nice theoretical basis, but power iteration reduces to perhaps 5-10 lines of C.
- linc01n 13y agoI built an ad engine base on probability. But I am not sure which way is better.
- anaphor 13y agoAs far as I can tell they are using local inference, which is basically just unification. The set of possible types seems pretty narrow[1] as well so I don't see much room to go wrong. You're right that there is a lot of mathematical theory about type systems and that inference can easily go wrong (be undecidable) but that is mostly for type systems that try to do inference for higher rank polymorphism and other things, which it doesn't seem like Hack is. Also I guess the language is supposed to be a superset of "valid" PHP, although I don't know whether this is true without modifying the PHP program much. http://hacklang.org/manual/en/hack.annotations.types.php http://hacklang.org/manual/en/hack.annotations.types.php
- samth 13y agoLocal inference isn't just unification. In particular, most local inference algorithms are designed to work with subtyping, which doesn't work in ML-like type systems.
- anaphor 13y agoTrue, I didn't see any mention of subtyping on there but since it's PHP I guess they have to deal with it somehow.
- DeGuerre 13y agoI assume that "local inference" means, in practice, "no let-polymorphism". That's what causes most of the headaches with extensions to Hindley-Milner (including the undecidability of subtyping).
- lbrandy 13y ago> I am baffled as to why you'd build your castle atop a crumbling foundation. Because perfect is not the enemy of the good? Because "build atop a crumbling foundation" has demonstrated time and again to be, by far, the most successful way to accomplish anything in computing? Unless you have some example of perfect, now dominant, technologies that have been created ex nihilo that I'm missing? I mean we (facebook) are still using PHP and MySql, improving both. And when we need to break things out, we head into C++, the queen mother of castles on broken foundations. > But at the same time, layering FP with a home rolled static type checking server (??) is bug prone and is certainly yak shaving (which they have time and money to do). Now they've written (1) a compiler to C++, (2) a compiler to VM byte code, (3) a corresponding runtime for each, (4) extensions to PHP, (5) a type checker, and (6) an inference engine. That's a lot of stuff. All languages, runtimes, and standard libraries (and databases, and source control, and on and on) are "broken" at sufficient scale. You're going to be spending time rebuilding things other people take for granted no matter who you are and what language and technology you are working in. The underlying assumption that a "proper language" gives you these things for free is completely false. > Writing correct type checkers and inference engines is kind of difficult. Just so we are clear, you ask why Facebook didn't rewrite 10,000 human-years of code into a mythical unnamed "proper" language, but you consider writing a type checker to be "difficult". I think you might have vastly inaccurate pictures of what is and isn't "difficult". > That way, the compiler is now almost an engine to prove your code is correct. I don't see how the same guarantee can be made with something that is just cobbled together. Computing history is littered with dead projects from people who believed that anything less than perfect is unworkable or non-valuable.
- goalieca 13y agoFacebook is a relatively mature company now that is in the business of making money. In fact, I would argue that [framework of the week] is more buggy and broken because it less well understood.
- derefr 13y ago> Just so we are clear, you ask why Facebook didn't rewrite 10,000 human-years of code into a mythical unnamed "proper" ... You don't need to rewrite anything (at least, not all at once.) Personally, I'd have expected you to make something akin to CoffeeScript or ClojureScript that targets PHP, and can "link with" your existing PHP modules (or rather, with their HHVM bytecode representations.) Then treat the PHP code as a constantly-dwindling Big Ball of Mud (http://laputan.org/mud/ http://laputan.org/mud/).
- camus2 13y agoI think PHP has some goodness that makes it the poor's man java. Neither Ruby or Python provide that for instance. Though Ruby is in my opinion well designed,and with duck typing,you might not need all that Java like OOP, I dont know,I feel like having interfaces makes me understand code faster and better, Want to understand what an object does? just read the interface,no magics,no bullshit,it's self documenting and that's important,that's why you can have these huge codebases like Zend,Symfony or Doctrine and still understand how complex elements work together. And even figure out how to use them without a doc,just like Java. I feel I cant go to Django's source code or Rails source code and understand everything easily just by reading function signatures. And the write/refresh cycle makes iteration pretty fast during development. But yeah,PHP basic syntax sucks for sure.
- toast0 13y agoThe HN crowd seems to dislike (or despise perhaps?) PHP, but it's really not that bad. Yes it has a lot of warts, but it has a lot of things that make it nice for web development. a) try your new code by saving in your editor, and hitting reload in your web browser. b) it's very approachable. People who only know HTML and CSS can be expected to do a little bit of PHP work to integrate their changes. If you setup the right network mounts, they just need to edit files and reload (see a) c) it's not super high overhead at runtime. If you're not using a framework, and you don't build up a crazy object hierarchy, it's not too hard to get your page out with about 10 ms of overhead beyond data fetching. For very simple webservices (fetch data, possibly from multiple sources, and do a little formatting for the consumer), I was able to get the overhead down to 2ms. You can certainly do better with other languages, but you can usually get better throughput improvement by working on getting data quickly. Btw, all the frameworks are terrible; many of them add 100 ms to the page just for the privilege of loading the includes; PHP is a framework for web programming thank you very much. d) cleanup; you don't have to worry about it. If you don't do anything weird (c extensions, with non-preferred malloc), at the end of the request, everything is thrown away. That said, there are plenty of things PHP isn't good at: I wouldn't run a long running process in PHP; and multithreaded PHP sounds like a bad idea.
- 1ris 13y agoa) WTH? I hope you do not test your code in production. If you don't that feature (a ide that constantly compiles your code) is a common feature. b) Theorem: Leting your web designers program is a way worse than letting your programmers do web design. c) Every simple project can become complex at some point. Facebook certainly is.
- ChuckMcM 13y agoAre not coral reefs both beautiful and built upon the dead bodies of those who came before?
- gtirloni 13y agoPeople also cut themselves easily with them.
- tripzilch 13y agoYes, but unlike Facebook, they are largely submerged in seawater.
- chenelson 13y agoFacebook is literally 60% water, whence they came. Pretty wild when you think about it.
- solipsism 13y agoThat 60% figure pertains to average humans. Those skinny hipsters are mostly calcium :)
- tripzilch 13y agoLike coral reefs! And hipsters are built from the dead bodies of vintage things (that you probably never heard of because they're pretty underwater/ground/whatever). Ok, I cede the point. PHP has the retro chic and Hack's type system is fixed-gear.
- andralex 13y ago(I work at Facebook but not on Hack.) There's a lot to be said about backward compatibility, and much of Hack's virtuosity stems in its smooth interoperability with PHP - many millions of lines of it. There's nothing like working on such a large codebase to convince one how difficult disruption of any kind is. The language definition and semantic checker are difficult, but are stereotypically tasks that cannot be distributed to many engineers; instead, a few senior engineers took that task to the benefit of all others.
- dcc1 13y agoWhat editor do facebook recommends for hack? Pho storm doesn't support it yet unfortunately
- zmanian 13y agoFacebook is doing an admirable job replacing rotting pieces of their infrastructure with more robust replacements. HHVM replaced the execution environment for their code with a more robust code generation/ runtime system. Hack allows them to bootstrap their code base into higher degrees of reliability without a mass rewrite. Also Brian O'Sullivan is one of the best people on the face of the earth to be trying to find practical ways on integrating PL research into practical engineering.
- jimbrusstar 13y agoThis seems an unequivocal improvement for Facebook, since they're unwilling to move away from PHP. The better question is why would anyone else choose to build their company on this?
- hub_ 13y agoFor the same reason there are companies writing software for Windows. Or companies putting Windows on all their workstations.
- wes-exp 13y agoBecause http://en.wikipedia.org/wiki/Worse_is_better http://en.wikipedia.org/wiki/Worse_is_better
- gfodor 13y ago"I am baffled as to why you'd build your castle atop a crumbling foundation." Congratulations for admitting your ignorance, and lending open ears to experts as to why they made certain engineering decisions. Oh wait, you weren't doing that, you were just warming up to go on a diatribe about how stupid Facebook engineers must be.
- teacup50 13y ago> Congratulations for admitting your ignorance, and lending open ears to experts as to why they made certain engineering decisions. Yes, they made certain engineering decisions now because the decisions they made back then were stupid, and they have to dig themselves out.
- gfodor 13y agoYes and the OP is criticizing the decisions being made now as "yak shaving." Could it perhaps be that the "yak shavers" made a conscious and well-reasoned decision to go in the direction of "extend PHP" vs "throw it all out and rewrite everything in language-of-the-month?"
- teacup50 13y agoI'm sure it was conscious and well-reasoned, but the OP's point still stands. Trying to build a reasonable forward-looking high-performance managed language runtime platform is hard. Trying to build it atop of PHP is harder. The only way I could see this as being a smart long-term strategy is if the eventual goal is to isolate and retire PHP projects entirely (and PHP usage itself) over time. However, even then, with PHP gone, and Hack no longer necessary, you're still stuck maintaining your own incompatible VM / runtime. Is Facebook signing up to reproduce the CLR? Or do they have long term plans to somehow bridge the gap between HHVM and more established VMs/runtimes, where they can better share the maintenance load with the wider industry.
- sanxiyn 13y agoMaintaining your own VM is no big deal. Compared to Facebook, HHVM is a tiny codebase. A team made of relatively small number of people (high quality, but low quantity) can and do maintain VMs like HotSpot and V8. LuaJIT is maintained by a single person.
- ahomescu1 13y ago> (3) a corresponding runtime for each That's not really true, HPHP and HHVM share the runtime (mostly).
- zaidf 13y agoBut at the same time, layering FP with a home rolled static type checking server (??) is bug prone Clearly the home rolled stuff is working out for facebook. Twitter was/is on RoR and continues(!!) to have major downtime issues.
- Xorlev 13y agoTwitter might have some legacy Rails hanging around, but it's a beastly Scala system these days.
- vidarh 13y agoTwitter could have been written in any language and worked, but given their old architecture decisions, they'd been just as likely to mess it up in every language. They blamed RoR a lot because they needed an explanation for their problems, but "many to many" messaging even at their scale is a "solved" problem and has been for decades and is fairly easy to scale. (Think of Twitter as a bunch of mailing lists and mailboxes; you scale it by decomposing it: map accounts to virtual "buckets" that becomes the domain part, and map tweets to messages; break apart large follower lists into smaller ones and introduce a forwarding reflector; break apart large following lists by splitting "mailboxes" and doing zipper merges on reading it; add a caching layer -- this is not rocket science, and you could do it properly in any language) Note: I think RoR was a horrible choice for them, though I love Ruby, but I also don't for a second believe RoR was their real problem. Edit: Your overall point stands, though. Especially given that Facebook is a far more complicated application.
- United857 13y ago(Disclaimer: FB employee.) Do you also argue that C++ is built on top of a crumbling foundation, because it's based on C, a ancient language with almost no type-safety?
- bluetech 13y agoWhy do you say C has no type safety? Only way around the type system is void, explicit casts and I guess unions. It admittedly doesn't have a very advanced type system, that's true.
- munificent 13y ago> Only way around the type system is void, explicit casts and I guess unions. And typedefs. Given that there's no parametric types, you run into void* quite frequently as well, so saying "only" inaccurately minimizes the scale of how much C code isn't strictly type safe.
- bluetech 13y agoRight, void* is used quite a lot for "generic" data structures. But I'm not sure that's what he meant. The reason I said "only" was that from my own experience, most C data structures are tailored for a specific use and so I don't see void* too much in this context. And may I ask why you say "typedef" is unsafe? It is merely a type alias, like e.g. Haskell's and ML's "type", or isn't it?
- munificent 13y ago> It is merely a type alias, like e.g. Haskell's and ML's "type", or isn't it? It is, but it freely allows conversion between the aliased types. typedef int Feet; typedef int Meters; int main() { Feet height = 6; Meters inEngland = height; // <-- OK. :( }
- anaphor 13y ago
- ausjke 13y agoI have seen many python programmers that are PHP haters, are you one of them? just curious, no offence. I like both Python and PHP, but use PHP for commercial applications.
- reikonomusha 13y agoI very occasionally write Python code if my job calls for it, but I am not a fan.
- Fasebook 13y agoWhat if you could build a castle on top of a highway?
- coldtea 13y ago>I have wondered why FB didn't use a proper language with proper typing to begin with. Because there are factors like an existing codebase that works that trump BS idealism.
- reikonomusha 13y agoSpelling correction: I said "layering FP" but I meant "layering PHP". The edit window has since come and gone.
- wbsun 13y agoBecause your perfect OCaml, ML, Haskell, and any other fancy, magical, fabulous, eternal, fantastical, simplest, elegant ... languages are all atop crumbling foundations implemented in ugly, stupid, out-dated, evil, chaffy ... C, C++, and assembly running on inefficient, silly and fragile digital logical CPUs. And when Facebook uses this stupid technique to build the world's largest social network for more than one billion users, those elegant and perfect solutions are serving ... how many?
- joesmo 13y ago"And in the end, it's still PHP, which is duly disliked." You might dislike it, but that doesn't mean it's disliked. PHP has a giant base of programmers, scales, is easy to learn, is extremely versatile and powerful, and as you point out, the code was already in PHP. Only an idiot would rewrite a giant working codebase simply to have it in a language that's "difficult to hire and train" in. Or any language for that matter. Perhaps they could have pulled a netscape but instead decided to serve billions.
- vishnugupta 13y agoIf I recall correctly PHP is banned at Amazon, even for internal tools, mainly for security reasons. The team that gives security clearance won't even take a cursory look at the service if it depends on PHP in anyway. Disclaimer: I have no zero experience with PHP.
- cyberneticcook 13y ago>> I am baffled as to why you'd build your castle atop a crumbling foundation I think you're using a wrong metaphor. Facebook foundations can't be crumbling just because they're made in PHP. There wouldn't be Facebook as we know it today otherwise. You might say they used a "low quality" material to build them. I see Hack more as a better material, that can also bind with the previous one and make it stronger.
- rafekett 13y agoI think the short story is that the engineers at Facebook feel pretty productive with their stack today. Making a new language that basically fixes up PHP is ideal for them because it gives them good confidence they can get the benefits of static analysis without sacrificing much productivity. That level of productivity + the sheer numbers they have make Hack a more attractive option.