4 ms·
Did this end up working? It sounds plausible but it needs some empirical validation.
by markisus 1y ago
Did this end up working? It sounds plausible but it needs some empirical validation.
- serialx 1y agoYeah, attention sinks were applied to gpt-oss
- Maxious 1y agoThere was skepticism last time this was posted https://news.ycombinator.com/item?id=37740932 https://news.ycombinator.com/item?id=37740932 Implementation for gpt-oss this week showed 2-3x improvements https://github.com/ggml-org/llama.cpp/pull/15157 https://github.com/ggml-org/llama.cpp/pull/15157 https://www.reddit.com/r/LocalLLaMA/comments/1mkowrw/llamacpp_just_added_a_major_3x_performance_boost/ https://www.reddit.com/r/LocalLLaMA/comments/1mkowrw/llamacp...