16 ms·
From my experience when using llama.cpp, using min-p sampling gives far better results than the defaults, for example (note this is using deepseek-coder 33b) P
by azeirah 3y ago
From my experience when using llama.cpp, using min-p sampling gives far better results than the defaults, for example (note this is using deepseek-coder 33b)
Prompt: "Write the fibonacci function in Python3"
min_p 0.05
def fibonacci(n:int, memo={}) -> int:
if n <=2 : return 1 # base case 1, 2 => 1
if n not in memo:
memo[n] = fibonacci(n-1, memo) + fibonacci(n-2,memo) # store the result
return memo[n]
# Test cases
print("Test case 1")
print(f"The 5th Fibonacci is {fibonacci(5)} ")
print(f"The 9th Fibonacci is {fibonacci(9)} ")
Default sampling settings
def fibonacci(n):
if n == 0:
return 0
elif n ==1:
return 1
else :
return (fibonacci(n - 1) + fibonacci(n -2))
n = int(input())
print(fibonacci(n))
Seed is the same.
So if you're using these models locally, make sure you're using min-p sampling. All other samplers are genuinely very suboptimal.
For more in-depth info about _why_: https://www.reddit.com/r/LocalLLaMA/comments/17vonjo/your_settings_are_probably_hurting_your_model_why/ https://www.reddit.com/r/LocalLLaMA/comments/17vonjo/your_se...