4 ms·
Just to confirm, as this topic gets "very meta" with levels of indirection, it sounds like you mean the LLM appends a "fitting" document fragment like: Thi
by Terr_ 1y ago
Just to confirm, as this topic gets "very meta" with levels of indirection, it sounds like you mean the LLM appends a "fitting" document fragment like:
This was an unusual task Bot wasn't sure how to solve directly.
Bot decided it needed to execute a program:
[CODE_START]foo(bar(baz())[CODE_END]
Which resulted in
[CODE_RESULT_PLACEHOLDER]
This stage-direction is externally parsed, executed, and substituted, and then the LLM is called upon to generate Bot-character's next reaction.
In terms of how this could go wrong, it makes me think of a meme:
> Thinking quickly, Dave constructs a homemade megaphone, using only some string, a squirrel, and a megaphone.
- QueensGambit 1y agoYes, my understanding is: - finding patterns in data is memorization - finding patterns in metadata is intelligence - finding patterns in meta-metadata is invention For example, if you ask someone to hang a painting in an art gallery 12 feet from the floor using a 13-foot ladder: - a worker will use the safety rule of staying 5 feet away from the wall. This is what GPT-3 does. [1] - an engineer will apply the Pythagorean theorem. This is what o3 does. - Pythagoras, seeing it for the first time, will derive the theorem. GPT-5 is nowhere close to that. This climbing up the ladder of abstraction existed even before LLMs. DeepMind's AlphaGo learned from human games. But AlphaGo Zero and AlphaZero trained entirely through self-play and began uncovering new strategies across Go, chess, and shogi. So whether it's code, a game, or pseudocode, they're all metadata operating at the same level of abstraction. [1] The Nature of Intelligence is Meta - https://manidoraisamy.com/intelligence-is-meta.html https://manidoraisamy.com/intelligence-is-meta.html