3 ms·
Open researchers are trying to shrink and speed up 138K models e.g. YaRN https://github.com/jquesnelle/yarn https://github.com/jquesnelle/yarn It's very compel
by fancy_pantser 3y ago
Open researchers are trying to shrink and speed up 138K models e.g. YaRN https://github.com/jquesnelle/yarn https://github.com/jquesnelle/yarn
It's very compelling and opens up a lot of use cases, so I've been keeping an eye out for advancements. However, inferencing on 4xA100s would be the target today for YaRN and 128K to get a reasonable token rate on their version of Mistral.