4 ms·
Perhaps not widely known but certainly known in LLM research. There was a bunch of these experiments done 2 years ago and what's interesting is that it still se
by vibe42 7mo ago
Perhaps not widely known but certainly known in LLM research. There was a bunch of these experiments done 2 years ago and what's interesting is that it still seems to work on the latest models.
Though beware that the increased score on math and EQ could lead to other areas scoring less well; would love to see how these models score on all open benchmarks.
- v9v 7mo agoThe author claimed that the models he modified with this layer repetition method topped the huggingface open llm leaderboard in his first post: https://dnhkng.github.io/posts/rys/ https://dnhkng.github.io/posts/rys/ Do you remember the names of the previous experiments done on this? Would love to take a look.
- vibe42 7mo agoJust learned about it the other day from this thread from Feb, 2024: https://old.reddit.com/r/LocalLLaMA/comments/1aqrd7t/i_made_an_inference_sever_that_supports_repeating/ https://old.reddit.com/r/LocalLLaMA/comments/1aqrd7t/i_made_... Has some interesting github links.