4 ms·

It is interpolation but that is what human thinking is as well. Interpolation is so broad it can cover agi conceptually. But I get it, the interpolation you’re

It is interpolation but that is what human thinking is as well. Interpolation is so broad it can cover agi conceptually. But I get it, the interpolation you’re talking about is limited. But I think you missed this insight: human interpolation is limited too. In the short term everything we do is simply recombination of ideas as you put it. But that’s the short term. In the long term we do things that are much greater. But I think this is just an aggregation of small changes. Change the words in a poem 5000 times: have the LLM do the same task 5000 times. Let it pick a random word. The result is wholly original. And I think in the end this what human cognition is as well.

kingstnap 9mo ago
A chatbot is exactly and only a short term recombination of existing ideas is exactly my point. Even if an LLM came up with a theory of quantum gravity in some random chain of thought via chance, once the context is wiped everything is gone. Expanding the frontier of knowledge (true extrapolation) requires iteration and layering of sinpler ideas. If you loose the layers and have to start from scratch every time then you fundamently will never move further out then what you already know (the interpolation).
- threethirtytwo 9mo ago
  >A chatbot is exactly and only a short term recombination of existing ideas is exactly my point. You missed my point. I'm saying humans have finite context windows as well. Look at how claude keeps passing it's context window down the chain. It creates a summary. It can spend thousands of tokens to coalesce on a conclusion, and only that conclusion needs to be passed on to the next context window. The research can be tossed. That's how human discovery works. We don't need the whole context window, we produce major discoveries because we pass the conclusion down the chain. LLMs can do it too. We just never fully tried it.

kingstnap 9mo ago

A chatbot is exactly and only a short term recombination of existing ideas is exactly my point. Even if an LLM came up with a theory of quantum gravity in some random chain of thought via chance, once the context is wiped everything is gone. Expanding the frontier of knowledge (true extrapolation) requires iteration and layering of sinpler ideas. If you loose the layers and have to start from scratch every time then you fundamently will never move further out then what you already know (the interpolation).

threethirtytwo 9mo ago
>A chatbot is exactly and only a short term recombination of existing ideas is exactly my point. You missed my point. I'm saying humans have finite context windows as well. Look at how claude keeps passing it's context window down the chain. It creates a summary. It can spend thousands of tokens to coalesce on a conclusion, and only that conclusion needs to be passed on to the next context window. The research can be tossed. That's how human discovery works. We don't need the whole context window, we produce major discoveries because we pass the conclusion down the chain. LLMs can do it too. We just never fully tried it.