4 ms·
For those looking for a technical explanation, the PHP garbage collector in this case is probably wasting a ton of CPU cycles trying to collect thousands of obj
by Seldaek 12y ago
For those looking for a technical explanation, the PHP garbage collector in this case is probably wasting a ton of CPU cycles trying to collect thousands of objects (a LOT of objects are created to represent all the inter-package rules when solving dependencies) during the solving process. It keeps trying and trying as objects are allocated and it can not collect anything but still has to check them all every time it triggers.
Disabling GC just kills the advanced GC but leaves the basic reference counting approach to freeing memory, so Composer can keep trucking without using much more memory as the GC wasn't really collecting anything. The memory reduction many people report is rather due to some other improvements we have made yesterday.
As to why the problem went unnoticed for so long, it seems that the GC is not able to be observed by profilers, so whenever we looked at profiles to improve things we obviously did not spot the issue. In most cases though this isn't an issue and I would NOT recommend everyone disables GC on their project :) GC is very useful in many cases especially long running workers, but the Composer solver falls out of the use cases it's made for.
- haaaaaaryf 12y agoCouldn't I also speed up Composer by using Toran Proxy? ( https://toranproxy.com/ https://toranproxy.com/ )
- Seldaek 12y agoThat would help speed up the network part of the install/update yes. The linked patch on the other hand hopefully gets the CPU part to a decent level for most people.
- sillysaurus3 12y agoAs to why the problem went unnoticed for so long, it seems that the GC is not able to be observed by profilers, so whenever we looked at profiles to improve things we obviously did not spot the issue. That sounds like a bug in the profiler, not with Composer. Observing internal time is pretty important for any profiler.
- Seldaek 12y agoYes it is definitely a failure of the tooling and I hear it is actually being worked on.
- icehawk219 12y agoOut of curiosity what tools do you use for profiling and finding these sorts of things? Plain old xdebug, xhprof, or other things? I'm going to have to jump into debugging a fairly large Symfony application within the next couple months and am on the look out for good tools to help me along.
- Seldaek 12y agoTraditionally xdebug/xhprof were pretty decent, but xhprof has been a bit abandoned since facebook uses HHVM now. There are two new commercial condenters though that came out in the last few months: Blackfire.io and QafooLabs.com Both have announced support for showing GC time in profiles as a result of today's noise :) https://twitter.com/beberlei/status/539816149303955456 https://twitter.com/beberlei/status/539816149303955456 https://twitter.com/symfony_en/status/539815082881187841 https://twitter.com/symfony_en/status/539815082881187841
- icehawk219 12y agoThanks for the info, those two will be very helpful.
- jexus2 12y agoXhprof is maintained by Phacility. They did pick up on this[1], but it's not as easy as one might think. [1] https://secure.phabricator.com/T6675 https://secure.phabricator.com/T6675
- Seldaek 12y agoThanks for the correction, I wasn't aware that was part of what phabricator extracted out of facebook :)
- munificent 12y ago> It keeps trying and trying as objects are allocated Worse. GC gets triggered just by assigning references, allocation isn't even needed.