4 ms·

A project that has been generating a lot of buzz lately (CPU-based inference for Facebook's LLaMa model that works on commodity hardware) has attracted contribu

by 4bpp 4y ago

A project that has been generating a lot of buzz lately (CPU-based inference for Facebook's LLaMa model that works on commodity hardware) has attracted contributions from a tech/activist celebrity (https://en.wikipedia.org/wiki/Justine_Tunney https://en.wikipedia.org/wiki/Justine_Tunney). Their somewhat overly self-assured/-aggrandizing style (e.g. Github posts written in a tone like they run the place, changing the file format magic number to include their own initials along those of the project's originator) rubbed many people the wrong way, and a sweeping change they introduced may have resulted in performance regressions for several users (while also being hugely oversold: a fantastical and quickly disproven claim about significantly reduced memory usage sat at >1k upvotes on HN yesterday). Then another long-standing contributor made a PR just seeking to flat out revert the patch in question. The discussion quickly turned toxic, with a (thankfully) not-yet-quite-vocalised US culture war undercurrent and people bandwagoning based on their personal disposition towards the person at the core of it.

fastball 4y ago
Also worth mentioning that there is some level of controversy over how much of this work involving mmap should be attributed to jart vs slaren. Slaren originally authored a PR using mmap which some people are claiming was the much better implementation of the feature (including not needing to change the model format) and jart basically re-wrote it so that she could take credit for it.
- detaro 4y ago
  > basically re-wrote it so that she could take credit for it. compare the actual PR: https://github.com/ggerganov/llama.cpp/pull/613 https://github.com/ggerganov/llama.cpp/pull/613 You'll note that it includes the original commits from Slaren, explicitly mentions the collaboration with Slaren and explicitly requests to preserve these commits on merge.

by 4bpp 4y ago

fastball 4y ago

Also worth mentioning that there is some level of controversy over how much of this work involving mmap should be attributed to jart vs slaren. Slaren originally authored a PR using mmap which some people are claiming was the much better implementation of the feature (including not needing to change the model format) and jart basically re-wrote it so that she could take credit for it.

detaro 4y ago
> basically re-wrote it so that she could take credit for it. compare the actual PR: https://github.com/ggerganov/llama.cpp/pull/613 https://github.com/ggerganov/llama.cpp/pull/613 You'll note that it includes the original commits from Slaren, explicitly mentions the collaboration with Slaren and explicitly requests to preserve these commits on merge.