2 ms·
They're using GTPQ -- here you go: https://arxiv.org/abs/2210.17323 https://arxiv.org/abs/2210.17323 . The authors benchmarked two families of models over a wid
by sottol 4y ago
They're using GTPQ -- here you go: https://arxiv.org/abs/2210.17323 https://arxiv.org/abs/2210.17323 . The authors benchmarked two families of models over a wide range of numbers of params.
- ddren 4y agollama.cpp is using RTN at the moment.