3 ms·
Maybe he should have split his 120Mb executable into more executables ... or using a sane number of processes. It would be bad to have the OS cover this use ca
by rightbyte 8y ago
Maybe he should have split his 120Mb executable into more executables ... or using a sane number of processes.
It would be bad to have the OS cover this use case by default.
- deleted 8y ago[deleted]
- AstralStorm 8y agoYou do understand that windows creates about 1000 processes on boot alone? A clean start? (Of those about 100 survive as services.) In short this fail adds as much as 10 seconds to boot time? (Sort of hidden on most hardware by parallelism, but if you have something not super recent, well...)
- rightbyte 8y agoYes, but he creates 1000 processes of the same executable with different arguments to run different tests as I understand it. With a really big executable. That's not the same thing as it's slow to add normal sized processes.
- mort96 8y agoThat's not out of the ordinary though? Any time you compile C or C++ code, you're creating a process from the same GCC/Clang/whatever executable but with different arguments for each source file. For a big project, it's not uncommon for there to be thousands or tens of thousands of small source files. Creating and maintaining processes is the core responsibility of an operating system.
- rightbyte 8y agogcc is like 1Mb. It would be interesting if he did the same test on Debian or something with default options. I would personally link in the tests as a shared lib or something. chrome.exe on my laptop is around 1Mb. That would probably speed up his tests more than his hacks to the validation routine.
- leeter 8y agoI have a sneaky suspicion you're right, windows generally assumes any 'shared' code will be in DLLs and is optimized around ensuring they are shared as much as possible in memory. It's likely that if the tests were in a DLL vs. the EXE then windows would just share the CFG page for that DLL read only between all the processes and call it done.
- brucedawson 8y agoI have been told that moving the meat of unit_tests.exe to a DLL would avoid the repeated CFG initialization costs (they would be paid on the first CreateProcess call and then the CFG data would be shared). This wouldn't speed up the tests "more" than my change did (it would speed up the tests slightly less, actually), but it would be a possible fix. But, such a change would be more work. And, honestly, it shouldn't be necessary. Windows could avoid this problem by using an O(n) algorithm for the initialization, or by sharing the CFG data between .exe files as well as .dll files. I am content with my current workaround and I'll consider reverting it when the underlying OS issue is fixed. Note that the chrome executable on Linux contains all of the code so it is 50-100 MB. It is only an accident of history that chrome.exe on Windows puts all the code in chrome.dll/chrome_child.dll
- KyleJ61782 8y agoThe size of the executable doesn't really matter after the first load as subsequent loads share the mapped section so it doesn't need to be loaded from disk again. Loading an executable multiple times after the first time really shouldn't take all that long from an OS perspective...just some mapping of shared pages.
- brucedawson 8y ago> The size of the executable doesn't really matter > after the first load shouldn't really matter. If you have CFG enabled then it does. If they fix CFG's initialization then it will go back to not mattering, and even this huge process will be created in just a few ms.
- brucedawson 8y agoThis OS performance bug does not add 10 seconds to boot time. That would only be the case if those startup processes were enormous .exe files, which they generally are not. The O(n^2) chart in the blog post shows that the cost of this issue is modest for most executables.
- chmod775 8y agoYou don't think an OS/kernel should be expected to perform the most basic of kernel tasks well? Also creating a lot of processes to run your tests is a perfectly normal use case. In this case they're running 10 per process, but creating 1 process/test (so as to not leave any corrupt memory/state behind after a failed one) would also be perfectly reasonable.
- rightbyte 8y agoAm I misunderstanding this; there are 1000 instances of a process stemming from a 120Mb executable, and initializing it 1000 times is kind off slow unless you chose to turn off some branching integrity check.
- nixpulvis 8y agoI don't think other OSs have this much trouble, but I haven't performed my own testing, so take this statement with a grain of salt.
- rightbyte 8y agoDunno. But reading the test readme it seems like gLinux (GNU/Linux? RS' new angle?) has other problems that need dehacking. I mean running test like this is fine who cares. But it's probably bad architecture for the unit test that makes it take time rather than the OSes. And that's fine because it's not production code. But not ground for a condescending snarky blog post. https://www.chromium.org/developers/testing/running-tests https://www.chromium.org/developers/testing/running-tests 'Run tests 5x faster on gLinux Browser tests on gLinux start up extremely slowly due to idiosyncratic NSS configurations. If you need to regularly run browser tests on gLinux, consider using the run_with_dummy_home.py helper script: testing/run_with_dummy_home.py testing/xvfb.py out/Default/browser_tests This can speed tests up by 5x or more.'
- chmod775 8y agoThat is something specific to gLinux, not Linux. Also it's not even related to the kernel. They're likely using some NSS configuration that does network lookups (maybe they're using NIS, LDAP or something else).