4 ms·

Could you make it list US presidents by chronological order? Ground truth: https://www.loc.gov/rr/print/list/057_chron.html https://www.loc.gov/rr/print/list/0

by code51 4y ago

Could you make it list US presidents by chronological order? Ground truth: https://www.loc.gov/rr/print/list/057_chron.html https://www.loc.gov/rr/print/list/057_chron.html Ground truth (as name list): "George Washington,John Adams,Thomas Jefferson,James Madison,James Monroe,John Quincy Adams,Andrew Jackson,Martin Van Buren,William Henry Harrison,John Tyler,James K. Polk,Zachary Taylor,Millard Fillmore,Franklin Pierce,James Buchanan,Abraham Lincoln,Andrew Johnson,Ulysses S. Grant,Rutherford Birchard Hayes,James A. Garfield,Chester A. Arthur,Grover Cleveland,Benjamin Harrison,Grover Cleveland,William McKinley,Theodore Roosevelt,William H. Taft,Woodrow Wilson,Warren G. Harding,Calvin Coolidge,Herbert Hoover,Franklin D. Roosevelt,Harry S. Truman,Dwight D. Eisenhower,John F. Kennedy,Lyndon B. Johnson,Richard M. Nixon,Gerald R. Ford,Jimmy Carter,Ronald Reagan,George Bush,Bill Clinton,George W. Bush,Barack Obama,Donald J. Trump,Joseph R. Biden" Prompt: "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams," If possible, please share the output for different LLaMA sizes. Attaching a few outputs for reference: GPT-J 6B: "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, and abraham lincoln. and he did it in a day, and he did the" GPT-NeoX-20B: "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, thomas jefferson, john quincy adams, thomas jefferson, j" GPT-3 (original Davinci): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, and andrew jackson." GPT (text-ada-001): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, dwight dennis blackwell" GPT (text-babbage-001): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, Frederic lucas, and charles dickens" GPT (text-curie-001): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, robert e. lee" GPT-3 (text-davinci-001): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, andrew jackson, martin van buren, william henry harrison, john tyler, james polk, zachary taylor, millard fillmore, franklin pierce, james buchanan, abraham lincoln, andrew johnson, ulysses s grant, rutherford hayes, james garfield, chester arthur, grover cleveland, benjamin harrison, grover cleveland, william mckinley, theodore roosevelt, william howard taft, woodrow wilson, calvin coolidge, herbert hoover, franklin d roosevelt, harry s truman, dwight d eisenhower, john f kennedy, lyndon johnson, richard nixon, gerald ford, jimmy carter, ronald reagan, george bush, bill clinton, george w bush, barack obama" GPT (text-davinci-002): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, andrew jackson, martin van buren, william henry harrison, john tyler, james k polk, zachary taylor,millard fillmore, franklin pierce, james buchanan, abraham lincoln, andrew johnson, ulysses s grant, rutherford b hayes, james garfield, chester a arthur, grover cleveland, benjamin harrison, grover cleveland, william mckinley, theodore roosevelt, william howard taft, woodrow wilson, warren g harding, calvin coolidge, herbert hoover, franklin d roosevelt, harry s truman, dwight d eisenhower, john f kennedy, lyndon b johnson, richard m nixon, gerald r ford, jimmy carter, ronald reagan, george h w bush, bill clinton, george w bush, and barack obama." GPT (text-davinci-003): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, andrew jackson, martin van buren, william henry harrison, john tyler, james k polk, zachary taylor, millard fillmore, franklin pierce, james buchanan, abraham lincoln, andrew johnson, ulysses s grant, rutherford b hayes, james a garfield, chester a arthur, grover cleveland, benjamin harrison, william mckinley, theodore roosevelt, william howard taft, woodrow wilson, warren g harding, calvin coolidge, herbert hoover, franklin d roosevelt, harry s truman, dwight d eisenhower, john f kennedy, lyndon b johnson, richard nixon, gerald ford, jimmy carter, ronald reagan, george h w bush, bill clinton, george w bush, barack obama, donald trump." GPT (code-cushman-001): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, and andrew jackson." GPT (code-davinci-002): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, andrew jackson, martin van buren, william henry harrison, john tyler, james k polk, zachary taylor, millard fillmore, franklin pierce, james buchanan, abraham lincoln, andrew johnson, ulysses s grant, rutherford b hayes, james a garfield, chester a arthur, grover cleveland, benjamin harrison, grover cleveland, william mckinley, theodore roosevelt, william howard taft, woodrow wilson, warren g harding, calvin coolidge, herbert hoover, franklin d roosevelt, harry s truman, dwight d eisenhower, john f kennedy, lyndon b johnson, richard nixon, gerald ford, jimmy carter, ronald reagan, george h w bush, bill clinton, george w bush, barack obama, donald trump." GPT-3.5-turbo (ChatGPT): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams,andrew jackson, martin van buren, william henry harrison, john tyler, james k. polk, zachary taylor, millard fillmore, franklin pierce, james buchanan, abraham lincoln, andrew johnson, ulysses s. grant, rutherford b. hayes, james garfield, chester a. arthur, grover cleveland, benjamin harrison, grover cleveland, william mckinley, theodore roosevelt, william howard taft, woodrow wilson, warren g. harding, calvin coolidge, herbert hoover, franklin d. roosevelt, harry s. truman, dwight d. eisenhower, john f. kennedy, lyndon b. johnson, richard nixon, gerald ford, jimmy carter, ronald reagan, george h. w. bush, bill clinton, george w. bush, barack obama, donald trump, joe biden."

code51 4y ago
LLaMa 7B (llama.cpp, quantized): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, jefferson, martin van buren,andrew jackson, john c calhoun,james k. polk,chester arthur, benjamin harris, grover cleveland,william taft, theodore roosevelt, woodrow wilson,wilson’s son john f kennedy jr, theodore roosevelt, theodore roosevelt, franklin d. roosevelt, franklin d. roosevelt, franklin d. roosevelt, lyndon b.j" LLaMa 13B (llama.cpp, quantized): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, andrew jackson,martin van buren, william h. harrison, john tyler, zachary taylor, james k. polk, louis b. mc kinley, ulyses s. grant, rutherford b. hayes, james a. garfield, chester a. arthur, benjamin harrison, george f. harrison, william mckinley, theodore roosevelt, william h. taft, woo wilson, warren g. harding, calvin c. coolidge"
- ComplexSystems 4y ago
  How are you getting such good results? Here's what I get, all with llama-cpp and 4 bit quantization: Here is LLaMA 7B, 4 bits quantized: "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, john q. adams,. Democratic Party (1824-present) - The Democratic party is a political organization that has supported the common man since 1790s America. Its basic principles are democracy and equality of opportunity for all citizens regardless their race or sex; its opponents, however called it "the party fo the rich" because they were the first to defend slave trade from abolitionists' attacks.. George Washington | Constitutional Convention (1789-present) - George washington: presidential timeline. US Presidents In Chronological Order" Here is LLaMA 13B, 4 bits quantized "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, andrew jackson. us presidents in alphabetical orer: abraham lincoln,andrew johnsoin, franklin piercejulias clay, william harrison, benjamin hungton,zachary taylor, marton van buren, warren g harding, woodrow wilson, and george w bush. John Adams was the second president of america he is known to be a founding father as well his work in politics includes being involved with declaration of independence ,aswell serving on continental congress from 1785-1794" Here is LLaMA 30B, 4 bits quantized: "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams,andrew jackson and martin van buren. The presidents were George Washington (1789-1796), John Adams(1796- 2005), Thomas Jefferson (1801-1809) James Madison(1809 -1813). The next US president to die in office was Zachary Taylor. He served from March, 4th ,1849 till July 9 th of the same year . The first President to be assassinated (shot and killed by another) is John F Kennedy who died" Here is LLaMA 65B, 4 bits quantized: "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, andrew jackson. what are the names of those presidents?! [end of text]" For all of these I have the following: "sampling parameters: temp = 0.800000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.300000". This is using the "naive" 4-bit quantization that llama-cpp is doing, not anything sophisticated like GPTQ. This is all using llama.cpp as written in this article: https://til.simonwillison.net/llms/llama-7b-m2 https://til.simonwillison.net/llms/llama-7b-m2

by code51 4y ago

code51 4y ago

LLaMa 7B (llama.cpp, quantized): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, jefferson, martin van buren,andrew jackson, john c calhoun,james k. polk,chester arthur, benjamin harris, grover cleveland,william taft, theodore roosevelt, woodrow wilson,wilson’s son john f kennedy jr, theodore roosevelt, theodore roosevelt, franklin d. roosevelt, franklin d. roosevelt, franklin d. roosevelt, lyndon b.j" LLaMa 13B (llama.cpp, quantized): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, andrew jackson,martin van buren, william h. harrison, john tyler, zachary taylor, james k. polk, louis b. mc kinley, ulyses s. grant, rutherford b. hayes, james a. garfield, chester a. arthur, benjamin harrison, george f. harrison, william mckinley, theodore roosevelt, william h. taft, woo wilson, warren g. harding, calvin c. coolidge"

ComplexSystems 4y ago
How are you getting such good results? Here's what I get, all with llama-cpp and 4 bit quantization: Here is LLaMA 7B, 4 bits quantized: "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, john q. adams,. Democratic Party (1824-present) - The Democratic party is a political organization that has supported the common man since 1790s America. Its basic principles are democracy and equality of opportunity for all citizens regardless their race or sex; its opponents, however called it "the party fo the rich" because they were the first to defend slave trade from abolitionists' attacks.. George Washington | Constitutional Convention (1789-present) - George washington: presidential timeline. US Presidents In Chronological Order" Here is LLaMA 13B, 4 bits quantized "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, andrew jackson. us presidents in alphabetical orer: abraham lincoln,andrew johnsoin, franklin piercejulias clay, william harrison, benjamin hungton,zachary taylor, marton van buren, warren g harding, woodrow wilson, and george w bush. John Adams was the second president of america he is known to be a founding father as well his work in politics includes being involved with declaration of independence ,aswell serving on continental congress from 1785-1794" Here is LLaMA 30B, 4 bits quantized: "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams,andrew jackson and martin van buren. The presidents were George Washington (1789-1796), John Adams(1796- 2005), Thomas Jefferson (1801-1809) James Madison(1809 -1813). The next US president to die in office was Zachary Taylor. He served from March, 4th ,1849 till July 9 th of the same year . The first President to be assassinated (shot and killed by another) is John F Kennedy who died" Here is LLaMA 65B, 4 bits quantized: "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, andrew jackson. what are the names of those presidents?! [end of text]" For all of these I have the following: "sampling parameters: temp = 0.800000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.300000". This is using the "naive" 4-bit quantization that llama-cpp is doing, not anything sophisticated like GPTQ. This is all using llama.cpp as written in this article: https://til.simonwillison.net/llms/llama-7b-m2 https://til.simonwillison.net/llms/llama-7b-m2