How are you getting such good results? Here's what I get, all with llama-cpp and 4 bit quantization: Here is LLaMA 7B, 4 bits quantized: "us presidents in chro

How are you getting such good results? Here's what I get, all with llama-cpp and 4 bit quantization: Here is LLaMA 7B, 4 bits quantized: "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, john q. adams,. Democratic Party (1824-present) - The Democratic party is a political organization that has supported the common man since 1790s America. Its basic principles are democracy and equality of opportunity for all citizens regardless their race or sex; its opponents, however called it "the party fo the rich" because they were the first to defend slave trade from abolitionists' attacks.. George Washington | Constitutional Convention (1789-present) - George washington: presidential timeline. US Presidents In Chronological Order" Here is LLaMA 13B, 4 bits quantized "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, andrew jackson. us presidents in alphabetical orer: abraham lincoln,andrew johnsoin, franklin piercejulias clay, william harrison, benjamin hungton,zachary taylor, marton van buren, warren g harding, woodrow wilson, and george w bush. John Adams was the second president of america he is known to be a founding father as well his work in politics includes being involved with declaration of independence ,aswell serving on continental congress from 1785-1794" Here is LLaMA 30B, 4 bits quantized: "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams,andrew jackson and martin van buren. The presidents were George Washington (1789-1796), John Adams(1796- 2005), Thomas Jefferson (1801-1809) James Madison(1809 -1813). The next US president to die in office was Zachary Taylor. He served from March, 4th ,1849 till July 9 th of the same year . The first President to be assassinated (shot and killed by another) is John F Kennedy who died" Here is LLaMA 65B, 4 bits quantized: "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, andrew jackson. what are the names of those presidents?! [end of text]" For all of these I have the following: "sampling parameters: temp = 0.800000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.300000". This is using the "naive" 4-bit quantization that llama-cpp is doing, not anything sophisticated like GPTQ. This is all using llama.cpp as written in this article: https://til.simonwillison.net/llms/llama-7b-m2 https://til.simonwillison.net/llms/llama-7b-m2