2 ms·
70B llama.cpp works now. You need the temporary `-gqa 8` flag for 70B. You can even extend context with RoPE!
by vczf 3y ago
70B llama.cpp works now. You need the temporary `-gqa 8` flag for 70B.
You can even extend context with RoPE!
loading thread…