4 ms·

Migrating to Hack from PHP was huge undertaking requiring years to conclude, and that almost kept the same syntax and semantics where possible. What would the

by yuliyp 5y ago

Migrating to Hack from PHP was huge undertaking requiring years to conclude, and that almost kept the same syntax and semantics where possible. What would the benefit of spending thousands of engineer years (read: billions of dollars) on a rewrite be? Especially when it would nearly certainly come with a huge performance regression compared to continuing to use HHVM which has come with years of optimization for working especially with Facebook's web codebase. In addition to the need to rewrite a gargantuan codebase and loss of the optimized runtime, it would mean throwing away a huge amount of expertise, tooling, and basically all of the source control history context. There really isn't much to gain. That said, the fraction of Facebook's code that is Hack continues to shrink, thanks to the continued growth of backend services and the clients being written in Hack (before there was server-rendered XHP, but now it's predominantly React).

TekMol 5y ago
Do you know if Facebook uses Composer to manage PHP dependencies?
- klodolph 5y ago
  Facebook uses a monorepo, similar to the way Google does it. Dependencies are vendored and checked into the repo. Facebook has released a tool for migrating software between repos, which is much like Google's Copybara: Facebook's fbshipit: https://github.com/facebook/fbshipit https://github.com/facebook/fbshipit Google's copybara: https://github.com/google/copybara https://github.com/google/copybara With these tools, you can make fairly sophisticated choices about how you do vendoring. You can make the internal version look just like the public version, in terms of commit history. And you can export internal commits to public commits, stripping out confidential information along the way (or integrations with internal systems and tooling).
  - TekMol 5y ago
    Thanks for the infos! What does "Dependencies are vendored and checked into the repo" mean?
    - bananapub 5y ago
      https://stackoverflow.com/questions/26217488/what-is-vendoring https://stackoverflow.com/questions/26217488/what-is-vendori... ie copy software source into your own repository, ideally in such a way that you can keep it up to date (which is what shipit/copybara are for)
    - dkasper 5y ago
      It means you literally commit the code from the dependency into your own repo. https://stackoverflow.com/questions/26217488/what-is-vendoring https://stackoverflow.com/questions/26217488/what-is-vendori...
- deleted 5y ago
  [deleted]
- fredemmott 5y ago
  Facebook doesn't. Open source users of Hack do, and it uses packagist.org - however if a package is written in PHP, Hack code can't use it. Not 100% accurate, but if a package's composer.json does not require HHVM, it will most likely install into a hack project, but be unusable.
fragmede 5y ago
> There really isn't much to gain. We (external parties) really can't say. We don't know what's holding Facebook's system back, we don't know the challenges it faces both internally and externally. We don't know what VM limits the code is hitting, how well the GC/memory allocation story is for HHVM with Facebook's code (other than what we're told). We don't know how well Facebook's database is or isn't keeping up with the backend, and what future plans the database team has that would change the query profile which could then cause the backend fleet to hit limits of HHVM's tunables with current growth. We also don't know HHVMs team is planning. If there is a problem that Hack and HHVM hasn't been able to solve, one would hope that the spending of billions of dollars is properly considered. Or to borrow from another expression, I'd hate to be billion-dollar-wise and future-of-the-company foolish. I don't begrudge the exec team for having to make that call and other calls like it. (Choosing to do Hack & HHVM in the first place was a similar judgement call.)
- yuliyp 5y ago
  Although currently an external party, I did work on these things at Facebook in the past. And you're right, that any system does have scale limits on a given type of hardware. Regarding memory allocation: HHVM has per-request memory arenas which get thrown away after each request. That combined with memory and time limits serves to compartmentalize the amount of memory pressure requests can place on the web server. Tuning of concurrent requests and workload mixes allows for some amount of exchanging memory pressure for throughput. I'm not sure what the comments about the database are: queries to databases and other backend systems are fairly indepent of the language the web server is written in (Instagram is written in Python, and hits many similar or the same systems and those systems don't really care which one is making the request). For specific components where HHVM is not able to handle them, those pieces can be extracted to separate services written in a different language (C++ or Rust, I think would be the go-tos probably?), with a cost of being unable to depend on libraries written in Hack.

by yuliyp 5y ago

TekMol 5y ago

Do you know if Facebook uses Composer to manage PHP dependencies?

klodolph 5y ago
Facebook uses a monorepo, similar to the way Google does it. Dependencies are vendored and checked into the repo. Facebook has released a tool for migrating software between repos, which is much like Google's Copybara: Facebook's fbshipit: https://github.com/facebook/fbshipit https://github.com/facebook/fbshipit Google's copybara: https://github.com/google/copybara https://github.com/google/copybara With these tools, you can make fairly sophisticated choices about how you do vendoring. You can make the internal version look just like the public version, in terms of commit history. And you can export internal commits to public commits, stripping out confidential information along the way (or integrations with internal systems and tooling).
- TekMol 5y ago
  Thanks for the infos! What does "Dependencies are vendored and checked into the repo" mean?
  - bananapub 5y ago
    https://stackoverflow.com/questions/26217488/what-is-vendoring https://stackoverflow.com/questions/26217488/what-is-vendori... ie copy software source into your own repository, ideally in such a way that you can keep it up to date (which is what shipit/copybara are for)
  - dkasper 5y ago
    It means you literally commit the code from the dependency into your own repo. https://stackoverflow.com/questions/26217488/what-is-vendoring https://stackoverflow.com/questions/26217488/what-is-vendori...
deleted 5y ago
[deleted]
fredemmott 5y ago
Facebook doesn't. Open source users of Hack do, and it uses packagist.org - however if a package is written in PHP, Hack code can't use it. Not 100% accurate, but if a package's composer.json does not require HHVM, it will most likely install into a hack project, but be unusable.

fragmede 5y ago

> There really isn't much to gain. We (external parties) really can't say. We don't know what's holding Facebook's system back, we don't know the challenges it faces both internally and externally. We don't know what VM limits the code is hitting, how well the GC/memory allocation story is for HHVM with Facebook's code (other than what we're told). We don't know how well Facebook's database is or isn't keeping up with the backend, and what future plans the database team has that would change the query profile which could then cause the backend fleet to hit limits of HHVM's tunables with current growth. We also don't know HHVMs team is planning. If there is a problem that Hack and HHVM hasn't been able to solve, one would hope that the spending of billions of dollars is properly considered. Or to borrow from another expression, I'd hate to be billion-dollar-wise and future-of-the-company foolish. I don't begrudge the exec team for having to make that call and other calls like it. (Choosing to do Hack & HHVM in the first place was a similar judgement call.)

yuliyp 5y ago
Although currently an external party, I did work on these things at Facebook in the past. And you're right, that any system does have scale limits on a given type of hardware. Regarding memory allocation: HHVM has per-request memory arenas which get thrown away after each request. That combined with memory and time limits serves to compartmentalize the amount of memory pressure requests can place on the web server. Tuning of concurrent requests and workload mixes allows for some amount of exchanging memory pressure for throughput. I'm not sure what the comments about the database are: queries to databases and other backend systems are fairly indepent of the language the web server is written in (Instagram is written in Python, and hits many similar or the same systems and those systems don't really care which one is making the request). For specific components where HHVM is not able to handle them, those pieces can be extracted to separate services written in a different language (C++ or Rust, I think would be the go-tos probably?), with a cost of being unable to depend on libraries written in Hack.