3 ms·
I don't disagree with your proof but I would like to add that your example doesn't mean it's not practical to recursively summarise a couple times. One bit of
by azeirah 3y ago
I don't disagree with your proof but I would like to add that your example doesn't mean it's not practical to recursively summarise a couple times.
One bit of information can mean more the more parameters an LLM has. The text "a8ckd" could easily mean the entire Declaration of Independence or whatever to an LLM (it doesn't, of course).
It has memorised a lot of stuff and similar to us "bubble sort" means
// An optimized version of Bubble Sort
void bubbleSort(int arr[], int n)
{
int i, j;
bool swapped;
for (i = 0; i < n - 1; i++) {
swapped = false;
for (j = 0; j < n - i - 1; j++) {
if (arr[j] > arr[j + 1]) {
swap(&arr[j], &arr[j + 1]);
swapped = true;
}
}
// If no two elements were swapped by inner loop,
// then break
if (swapped == false)
break;
}
}
That's 26x as much text (counting comments and whitespace). Without comments and removing most indentation it's about 20x as much.
The more information stored in its parameters, the more information a single bit can hold.
All we need to do is find out how many times can you recur until it breaks down? 3 times? 5 times? Just a few times is still genuinely very useful!