3 ms·
Is that similar to what ggerganov is talking about here? https://x.com/ggerganov/status/2089214161884414147 https://x.com/ggerganov/status/2089214161884414147
by DarmokJalad1701 2mo ago
Is that similar to what ggerganov is talking about here?
https://x.com/ggerganov/status/2089214161884414147 https://x.com/ggerganov/status/2089214161884414147
- hellajack3d 2mo agoYes - it would seem so :) I did make a PR to the official llama-cpp repo some time back (about a month or so), but abandoned it as there seemed to be too much community concern that the mechanism would degrade model performance... Perhaps i'll polish it up and put some effort into benchmarking and revive the project in the near future.
- mobelkh 2mo agowould be great as an opt-in though