4 ms·
Nitpick: > GitLab no longer loads large Git blobs (e.g. binary files) into memory when browsing a Git repository. This prevents timeouts and memory leaks. Nop
by pilif 11y ago
Nitpick:
> GitLab no longer loads large Git blobs (e.g. binary files) into memory when browsing a Git repository. This prevents timeouts and memory leaks.
Nope. Not loading something doesn't prevent memory leaks. It might make existing leaks not as bad (because you're leaking less).
Either you're not leaking at which point it doesn't matter how big the thing you load is, it will get freed once it's not used, or you're leaking at which point, yes, if you only load small things, you get to run for a longer time before you die, but you will still die eventually.
Only loading smaller things doesn't plug leaks.
Aside of that: This looks like a very impressive release. Congratulations!
- jobvandervoort 11y agoThanks for the nitpick, always welcome. I'm asking the author of the change what the background is on this. Will update the post if I can be more accurate. Thanks! UPDATE: Technically we're not fixing the leak, but this fix is reducing the impact significantly.
- Ensorceled 11y agoNitpick on the nitpick. If the Git blobs were the source of the memory leak then this would, in fact, fix the memory leaks. Pretty aggressive stance when you may, in fact, be wrong.
- pilif 11y ago> If the Git blobs were the source of the memory leak then this would, in fact, fix the memory leaks. no. Because they are still reading git blobs - just only for the smaller files. > GitLab no longer loads large Git blobs (e.g. binary files) into memory [emphasis mine] So if their handling of Git blobs is leaking memory, then not reading the bigger blobs just gives them more leeway before they crash.
- Ensorceled 11y agoAgain, you may still be wrong. Many systems have different strategies for large blobs vs. small blobs (I know it's many because I've written several myself) including chunking strategies, separate memory pools etc. I'm taking this stand because I've had the exact issue we are arguing about: my large file handler had a leak in the memory pool for large blobs. Basically it was shared memory space for multiple processes so large objects could be passed out of band instead of using IO.
- pg_is_a_butt 11y agoyeah, pilif way dum. "if you're still loading small then you're still leaking"... moron doesn't understand how caches and paging work and can lead to leaks. what a moron.
- ones_and_zeros 11y agoI think you can be a bit more charitable in your interpretation. Think of it this way: By no longer loading large Git blobs, it prevents known memory leaks and prevents known timeouts.