4 ms·
You used to be able to just start a correct looking output but they got really good at detecting that. llama.cpp ftw. It's not hard for it to be more producti
by nullc 3y ago
You used to be able to just start a correct looking output but they got really good at detecting that.
llama.cpp ftw. It's not hard for it to be more productive than fighting with the absurd openai censorware... Sadly many of the instruct trained models are tainted with openai censorship because they used GPT4 output in the fine tuning-- but at least on those the trick of starting the correct output yourself works!
- KirillPanov 3y agoThis. Also, llama.cpp now works (really well) with Radeon Instinct cards, which are stupid cheap because everybody thinks you need to buy nvidia stuff. Pcie bifurcation FTW!