4 ms·
Why can't LLM's solve this coding problem?
GPT4, Claude 3 Opus, Gemini Advanced, and Meta AI all fail at this, which surprised me. Even after pointing out the flaws in their initial solutions, I can't get them to give me a correct one. They clearly understand the problem specification, and the correct solution is only 8 lines of code. Curious if anyone has a theory why they find this so hard.
Here is the prompt:
I need a function that takes two integer parameters named TOTAL and MAX. The function should return a list of integers, each between 1 and MAX inclusive, that sum to TOTAL. The list should contain as few elements as possible, but the numbers within it should also be as close to each other as possible, and be in descending order. Examples:
TOTAL = 7, MAX = 1, RESULT = [1, 1, 1, 1, 1, 1, 1]
TOTAL = 7, MAX = 2, RESULT = [2, 2, 2, 1]
TOTAL = 7, MAX = 3, RESULT = [3, 2, 2]
TOTAL = 7, MAX = 4, RESULT = [4, 3]
TOTAL = 7, MAX = 5, RESULT = [4, 3]
TOTAL = 7, MAX = 6, RESULT = [4, 3]
TOTAL = 7, MAX = 7, RESULT = [7]
TOTAL = 7, MAX = 8, RESULT = [7]
Please write the function in python.
They usually return an incorrect solution that adds MAX until the final element, resulting in e.g. [6, 1] for TOTAL=7,MAX=6 instead of the correct [4, 3]. The correct solution is:
def distribute(total, max):
count = total // max
remainder = total % max
if remainder == 0:
return [max] * count
base = total // (count + 1)
left_over = total % (count + 1)
return [(base + 1 if i <= left_over else base) for i in range(1, count + 2)]
- DanAtC 3y agoBecause LLMs are just spicy autocomplete. They can't reason.
- Bostonian 3y agoBy giving your prompt (using "Fortran" instead of "Python") and then repeatedly pointing out the wrong output for TOTAL = 7, MAX = 6 and then reminding ChatGPT4 that the list should be descending order, I got it to generate the following working Fortran code module list_module implicit none contains function get_list(total, max) result(res) integer, intent(in) :: total, max integer, allocatable :: res(:) integer :: n, remaining, i, base_value ! Calculate the base value for each element and how many elements are needed n = (total + max - 1) / max ! This ensures we use the maximum possible times max can fit base_value = total / n remaining = total - base_value * n ! Allocate and initialize the result array allocate(res(n)) res = base_value ! Distribute the remainder starting from the last element, working backwards do i = n, 1, -1 if (remaining > 0) then res(i) = res(i) + 1 remaining = remaining - 1 endif end do ! Reverse the array to ensure descending order res = res(size(res):1:-1) end function get_list end module list_module program test_get_list use list_module implicit none integer, allocatable :: result(:) integer :: total, max, i, j ! Define the test cases integer, parameter :: test_cases(2, 8) = reshape([ & 7, 1, & 7, 2, & 7, 3, & 7, 4, & 7, 5, & 7, 6, & 7, 7, & 7, 8], [2, 8]) ! Loop over each test case do i = 1, size(test_cases, 2) total = test_cases(1, i) max = test_cases(2, i) result = get_list(total, max) ! Print the result write(*, '(A, I2, A, I2, A)', advance='no') "TOTAL = ", total, ", MAX = ", max, ", RESULT = [" if (allocated(result)) then write(*, '(I1, A)', advance='no') result(1), " " do j = 2, size(result) write(*, '(I1, A)', advance='no') result(j), " " end do endif write(*, *) "]" end do end program test_get_list with output TOTAL = 7, MAX = 1, RESULT = [1 1 1 1 1 1 1 ] TOTAL = 7, MAX = 2, RESULT = [2 2 2 1 ] TOTAL = 7, MAX = 3, RESULT = [3 2 2 ] TOTAL = 7, MAX = 4, RESULT = [4 3 ] TOTAL = 7, MAX = 5, RESULT = [4 3 ] TOTAL = 7, MAX = 6, RESULT = [4 3 ] TOTAL = 7, MAX = 7, RESULT = [7 ] TOTAL = 7, MAX = 8, RESULT = [7 ]
- yourfriendben 3y agoEasy to modify that function to do the final loop in the other order and then not need the sort afterward. I saw ChatGPT make that mistake a few times - it really wants to satisfy the requirement that they are in descending order by doing a sort at the end.