I also thought of this as a general idea: intelligence at the sampling level. Broadly you have different tiers of intelligence 1. take highest probability 2.

I also thought of this as a general idea: intelligence at the sampling level. Broadly you have different tiers of intelligence 1. take highest probability 2. based on some light weight code that tracks some state - like number of tokens or some sampling distribution 3. higher level is using a smaller llm to decide which token to sample (just a thought)