4 ms·
This is someone angrily filing a revert-all PR due to a performance regression, rather than helping diagnose the issue or make it configurable. Don't bother rea
by NortySpock 4y ago
This is someone angrily filing a revert-all PR due to a performance regression, rather than helping diagnose the issue or make it configurable. Don't bother reading.
It sounds like one person experienced a performance regression as a result of the llama.cpp MMAP changes, and decided to create a pull request to revert all of those changes. While they propose wrapping the mmap changes behind a feature flag / command-line flag, that's not what this PR does -- it just reverts the original commits. It's a "tear it down NOW" reaction rather than a "how can I improve this" reaction.
jart and a few other people in the PR have now proposed a variety of feature-flags or forked versions to address the issue in a more nuanced way.
- eska 4y agoThat’s not what’s happening. File format compatibility is broken while performance degrades by 10x for some people.